GSoC/2020/StatusReports/LeonardoEmanuelSegovia

Dynamic Fill Layers in Krita using SeExpr

Abstract

Layers are one of the core concepts of digital painting. They allow artists to control different parts of their artwork at once, for instance, color, lighting, lineart, as well as texture. A key feature of them is their ability to be resized, composited, renamed, grouped or deleted independently of the rest of the document.

Patterns and textures are also essential components of an artist’s toolbox, allowing them to represent the intricacies of a physical material. They come in two forms: bitmap textures, which are images contained in e.g. PNG or OpenEXR files, or procedural textures, which are generated on the fly using their mathematical representation.

KDE’s Krita painting suite supports using patterns and textures through two types of layers, File or Fill Layers. However, neither of them let artists create dynamically generated content: File Layers are inherently static, and Fill Layers support only color fills (like Paint Layers) or basic pattern rendering.

The goal of this project is to let artists create dynamic content through a new, scriptable Fill Layer. To this effect, I integrated Disney Animation’s SeExpr expression language into Krita.

Components

This project is divided into the following objectives or goals:

• SeExpr-based Fill Layer generator
• Creation and management of SeExpr scripts
• Integration tests
• User documentation

Side goals included:

• Feature support status across all our platforms
• Multithread support for Fill Layers
• ARM support for AppImages (including this project’s deliverables as well)

SeExpr-based Fill Layer generator

Enabling SeExpr as a Fill Layer generator involves two stages.

• Integration of SeExpr as a 3rdparty dependency
  • Commits: 1, 2, 3, 4
• Defining the new generator
• Loading, rendering, and saving the script with the layer
  • This commit implements both items.

The SeExpr generator is divided into three components:

• SeExprExpressionContext and SeExprVariable are the glue between SeExpr and Krita; the former contains the script’s execution context, the latter exposes the necessary variables for rendering the script (width, height, and pixel location in normalized coordinates).
• KisSeExprGenerator is the instance of KisGenerator tasked with rendering the script to the layer’s paint device. It sets up the SeExprExpressionContext with the given script and variables. Then, it iterates over the paint device, rendering the script on each pixel and converting the resulting color back to Krita’s color space.
• KritaSeExprGenerator, WdgSeExpr and WdgSeExprSavePreset comprise the UI of the SeExpr generator. These are described in more detail in the next section.

It must be noted that throughout the development of the SeExpr generator, extensive changes were made to Disney’s work itself. Of particular note are the following:

• Warnings and deprecations were solved across all operating systems I worked on.
  • Commits: 1, 2, 3, 4, 5, 6, 7, 8, 9
• The provided UI widgets were completely restyled for better accessibility.
  • Commits: 1, 2, 3, 4, 5, 6, 7, 8, 9
• Unused or irrelevant items were hidden behind flags to further optimize compilation.
  • Commits: 1, 2
• The ExprEditor text widget was decoupled from ExprControlCollection (the manipulation widgets list). This enabled a complete migration to Krita’s .ui widget system.
• ExprTextEditor now signals when a change is made to the script text field.
• A consistency change was made in the naming of variables.
• By request from Agata Cacko, variable insertion was modified to insert them into a new line, and comment were better differentiated from numbers.

The following bugs in SeExpr were fixed:

• As detailed on these blog posts, SeExpr implicitly assumed that it would be always executed in English-based locales. This was detected by Wolthera van Hövell.
  • Commits: 1, 2, 3
• The autocompletion support was unusable since it blocked numpad entry. This was fixed (commits: 1, 2), along with an extra optimization to use native controls.
• A buffer underflow was fixed in CCurveScene (the UI widget for color curves).

Creation and management of SeExpr scripts

When I implemented the generator, the script was saved along with the layer. David Revoy proposed that scripts should be reusable, by making them bundleable as any other resource in Krita.

The initial implementation, shown in the screenshots below, is based on the existing Pattern chooser widget. The underlying KoResource specialization, which is called KisSeExprScript and located in libs/flake/resources, is an almost-straight port of Anna Medonosová’s Gamut Masks.

• 

• 

Data-wise, a SeExpr preset’s script is stored in a plaintext file named script.se, along with a thumbnail image in preview.png. These are bundled together in a ZIP of extension .kse and MIME type application/x-krita-seexpr-script.

In T13337, I requested feedback from the developers on the UX for creating and editing these presets. Based on the existing workflow for managing brushes, I proposed the following mocks:

• 

• 

Based on suggestions from Agata Cacko in both the Krita Artists feedback thread and the Phabricator task, the final form of the UX is as follows:

• 

• 

• 

By request from Agata, the manipulation widgets and the text editor were placed inside a QSplitter, to ensure users could freely choose how much space they want to assign each component.

By request from Wolthera, support was added for reporting validation errors. Since SeExpr was not designed with internationalization in mind, extensive refactoring was needed to extracts its strings from the UI components and from the render library, allow their future localization by the KDE team, and finally enable their usage in the widgets. Additionally, variables exposed from Krita were added to the autocompletion system.

• 

• 

While there were not many changes in the management UX, it was simplified into 4 actions. The unique contribution here is Render Script to Thumbnail; as its name says, it creates an instance of the SeExpr fill layer generator, points it to the thumbnail image’s QPaintDevice, and renders the given preset’s script into the thumbnail.

Integration tests

There are two testable points in the SeExpr generator:

• successful rendering from the given script
• loading and saving from a given KoResource instance

The first is already implemented in the SeExpr generator (commits: 1, 2). The second is NYI.

User documentation

NYI

Multithread support for Fill Layers

To maximize the usability and performance of SeExpr-based layers, Dmitry Kazakov suggested that Fill Layers themselves should be converted to a Stroke. This lets Krita multithread the rendering process, thus greatly speeding up layer updates.

ARM support for AppImages

This goal is explained in more detail on its blog post.

• 

  The SeExpr AppImage running on a Raspberry Pi 3B+.

• 

  The SeExpr AppImage running on Libre Computer's Le Potato with 2GB RAM.

It was asked on the #krita IRC channel if it was possible to run Krita on ARM-based computers, specifically the Raspberry Pi 3B+. To the best of my knowledge, it has not been tried before.

In the blog post linked above, I prove fully-featured AppImages can be built for ARM-based platforms. Although the original target was 32-bit ARM (also known as armhf and armv7l), my main testing board is a 64-bit, 2GB Le Potato, which forced me to test this process for the aarch64 architecture as well.

The build process was done on a 16-thread Ryzen 7 under QEMU.

The key results were as follows:

• The KDE Sysadmin’s Docker image works almost out-of-the-box.
  • The base image must be changed to one that has QEMU, for instance multiarch/ubuntu-debootstrap.
  • If this path is followed, the universe repository must be enabled manually.
  • xkbcommon-dev is a key dependency to build Qt’s X11Extras module. It’s missing in aarch64 because libegl1-mesa-dev does not bring it in, unless in every other platform.
  • There are no official binaries for CMake, patchelf, and linuxdeployqt, so they must be built from scratch and bundled into the image.
• The build process must be multithreaded as much as possible.
  • The 3rdparty dependencies ext_expat, ext_python, ext_sip, and ext_qt must be forcibly parallelized using the -j${SUBMAKE_JOBS} flag.
• These dependencies need fixes or be disabled altogether:
  • OpenColorIO (ext_ocio) needs this patch applied
  • libx265 inside ext_heif needs to be upgraded to 3.4 to allow aarch64 builds. It works for armhf with -DENABLE_ASSEMBLY=false
  • Python will hang when running its tests, the BUILD_COMMAND must be changed to make build_all.
  • GSL needs the shared library in Unix-based systems, not only Android
  • Vc does not work altogether
• The build scripts assume across the whole process that the platform is x86_64. This must be changed taking into account that arm64 is called aarch64 by AppImageTool.

Status

• SeExpr-based Fill Layer generator: feature complete, awaiting review
• Creation and management of SeExpr scripts: feature complete, awaiting review
• Integration tests: WIP, missing tests for the Resources system
• User documentation: WIP, not yet uploaded
• Feature support status review: blog post published, no fixes proposed
• Multithread support for Fill Layers: merged
• ARM support for AppImages: blog post published, no fixes proposed

Deliverables

• Phabricator task: T13097, see in particular subtask T13337 for the preset management UI
• Krita Artists threads: “First alpha of my GSoC project: procedural texture generator”, “Procedural texture generator (example and wishes)”
• Fixes and improvements to SeExpr are available at my clone on KDE Invent
• Merge requests for the SeExpr generator: !380, !411
• Merge request for multithreading Fill Layers: !412
• Feature support list is available as a Google Docs file
• Blog posts are available at my site