GSoC/2020/StatusReports/LeonardoEmanuelSegovia

From KDE Community Wiki
< GSoC‎ | 2020‎ | StatusReports
Revision as of 14:08, 9 August 2020 by Lsegovia (talk | contribs) (→‎User documentation: Add resource bundle contribution)

Dynamic Fill Layers in Krita using SeExpr

  • Example render by David Revoy.

    Example render by David Revoy.

  • SeExpr presets management UI.

    SeExpr presets management UI.

  • Example render by David Revoy.

    Example render by David Revoy.

Abstract

Layers are one of the core concepts of digital painting. They allow artists to control different parts of their artwork at once, for instance, color, lighting, lineart, as well as texture. A key feature of them is their ability to be resized, composited, renamed, grouped or deleted independently of the rest of the document.

Patterns and textures are also essential components of an artist’s toolbox, allowing them to represent the intricacies of a physical material. They come in two forms: bitmap textures, which are images contained in e.g. PNG or OpenEXR files, or procedural textures, which are generated on the fly using their mathematical representation.

KDE’s Krita painting suite supports using patterns and textures through two types of layers, File or Fill Layers. However, neither of them let artists create dynamically generated content: File Layers are inherently static, and Fill Layers support only color fills (like Paint Layers) or basic pattern rendering.

The goal of this project is to let artists create dynamic content through a new, scriptable Fill Layer. To this effect, I integrated Disney Animation’s SeExpr expression language into Krita.

Components

This project is divided into the following objectives or goals:

  • SeExpr-based Fill Layer generator
  • Creation and management of SeExpr scripts
  • Integration tests
  • User documentation

Side goals included:

  • Feature support status across all our platforms
  • Multithread support for Fill Layers
  • ARM support for AppImages (including this project’s deliverables as well)

SeExpr-based Fill Layer generator

Enabling SeExpr as a Fill Layer generator involves two stages.

  • Integration of SeExpr as a 3rdparty dependency
  • Defining the new generator
  • Loading, rendering, and saving the script with the layer

The SeExpr generator is divided into three components:

  • SeExprExpressionContext and SeExprVariable are the glue between SeExpr and Krita; the former contains the script’s execution context, the latter exposes the necessary variables for rendering the script (width, height, and pixel location in normalized coordinates).
  • KisSeExprGenerator is the instance of KisGenerator tasked with rendering the script to the layer’s paint device. It sets up the SeExprExpressionContext with the given script and variables. Then, it iterates over the paint device, rendering the script on each pixel and converting the resulting color back to Krita’s color space.
  • KritaSeExprGenerator, WdgSeExpr and WdgSeExprSavePreset comprise the UI of the SeExpr generator. These are described in more detail in the next section.

It must be noted that throughout the development of the SeExpr generator, extensive changes were made to Disney’s work itself. Of particular note are the following:

The following bugs in SeExpr were fixed:

  • As detailed on these blog posts, SeExpr implicitly assumed that it would be always executed in English-based locales. This was detected by Wolthera van Hövell.
    • Commits: 1, 2, 3
  • The autocompletion support was unusable since it blocked numpad entry. This was fixed (commits: 1, 2), along with an extra optimization to use native controls.
  • A buffer underflow was fixed in CCurveScene (the UI widget for color curves) (commits: 1)
  • As part of the revision of the documentation, Wolthera van Hövell found a couple of bugs related to incorrect function documentation (commits: [1](https://invent.kde.org/lsegovia/seexpr/-/commit/b807bfebfec9edc4197ca3561f53625895edc96e))
  • A crash when initializing vector variables outside the [0, 1] range (commits: 1, 2)
  • Lack of promotion of vectors to colors after widget initialization (commits: 1)

Creation and management of SeExpr scripts

When I implemented the generator, the script was saved along with the layer. David Revoy proposed that scripts should be reusable, by making them bundleable as any other resource in Krita.

The initial implementation, shown in the screenshots below, is based on the existing Pattern chooser widget. The underlying KoResource specialization, which is called KisSeExprScript and located in libs/flake/resources, is an almost-straight port of Anna Medonosová’s Gamut Masks.

Data-wise, a SeExpr preset’s script is stored in a plaintext file named script.se, along with a thumbnail image in preview.png. These are bundled together in a ZIP of extension .kse and MIME type application/x-krita-seexpr-script.

In T13337, I requested feedback from the developers on the UX for creating and editing these presets. Based on the existing workflow for managing brushes, I proposed the following mocks:

Based on suggestions from Agata Cacko in both the Krita Artists feedback thread and the Phabricator task, the final form of the UX is as follows:

By request from Agata, the manipulation widgets and the text editor were placed inside a QSplitter, to ensure users could freely choose how much space they want to assign each component.

By request from Wolthera, support was added for reporting validation errors. Since SeExpr was not designed with internationalization in mind, extensive refactoring was needed to extracts its strings from the UI components and from the render library, allow their future localization by the KDE team, and finally enable their usage in the widgets. Additionally, variables exposed from Krita were added to the autocompletion system.

While there were not many changes in the management UX, it was simplified into 4 actions. The unique contribution here is Render Script to Thumbnail; as its name says, it creates an instance of the SeExpr fill layer generator, points it to the thumbnail image’s QPaintDevice, and renders the given preset’s script into the thumbnail.

Integration tests

There are two testable points in the SeExpr generator:

  • successful rendering from the given script
  • loading and saving from a given KoResource instance

The first is already implemented in the SeExpr generator (commits: 1, 2). The second is NYI. The first covers the SeExpr generator itself (commits: 1, 2, 3). The second covers serialization and deserialization from a known working KoResource (commits: 1).

User documentation

Up to now, there was no complete reference for creating SeExpr scripts. To build a complete reference for SeExpr scripts, my work created the following documental resources in the Krita user manual:

  • an entry to the Fill Layers section
  • a specific page for SeExpr scripts in Resource Management
  • a tutorial for creating, editing and bundling script presets
  • a Quick Reference page detailing the expression language

The latter is one of the most important productions, as it involved porting the existing User Documentation of Disney's (which is only available in precompiled Doxygen form) to reStructuredText, while adapting it to suit the script specification in Krita.

A fifth resource, this one provided with the Krita binaries, is a resource bundle comprising all examples in the Krita Artists thread, “Procedural texture generator (example and wishes)”. The authors of the examples are Wolthera van Hövell and David Revoy; the latter employed an alpha version of this tool in the episode 33 of his webcomic, Pepper&Carrot. (commits: 1, 2, 3, 4)

Multithread support for Fill Layers

To maximize the usability and performance of SeExpr-based layers, Dmitry Kazakov suggested that Fill Layers themselves should be converted to a Stroke. This lets Krita multithread the rendering process, thus greatly speeding up layer updates.

ARM support for AppImages

This goal is explained in more detail on its blog post.

  • The SeExpr AppImage running on a Raspberry Pi 3B+.
  • The SeExpr AppImage running on Libre Computer's Le Potato with 2GB RAM.

It was asked on the #krita IRC channel if it was possible to run Krita on ARM-based computers, specifically the Raspberry Pi 3B+. To the best of my knowledge, it has not been tried before.

In the blog post linked above, I prove fully-featured AppImages can be built for ARM-based platforms. Although the original target was 32-bit ARM (also known as armhf and armv7l), my main testing board is a 64-bit, 2GB Le Potato, which forced me to test this process for the aarch64 architecture as well.

The build process was done on a 16-thread Ryzen 7 under QEMU.

The key results were as follows:

  • The KDE Sysadmin’s Docker image works almost out-of-the-box.
    • The base image must be changed to one that has QEMU, for instance multiarch/ubuntu-debootstrap.
    • If this path is followed, the universe repository must be enabled manually.
    • xkbcommon-dev is a key dependency to build Qt’s X11Extras module. It’s missing in aarch64 because libegl1-mesa-dev does not bring it in, unless in every other platform.
    • There are no official binaries for CMake, patchelf, and linuxdeployqt, so they must be built from scratch and bundled into the image.
  • The build process must be multithreaded as much as possible.
    • The 3rdparty dependencies ext_expat, ext_python, ext_sip, and ext_qt must be forcibly parallelized using the -j${SUBMAKE_JOBS} flag.
  • These dependencies need fixes or be disabled altogether:
    • OpenColorIO (ext_ocio) needs this patch applied
    • libx265 inside ext_heif needs to be upgraded to 3.4 to allow aarch64 builds. It works for armhf with -DENABLE_ASSEMBLY=false
    • Python will hang when running its tests, the BUILD_COMMAND must be changed to make build_all.
    • GSL needs the shared library in Unix-based systems, not only Android
    • Vc does not work altogether
  • The build scripts assume across the whole process that the platform is x86_64. This must be changed taking into account that arm64 is called aarch64 by AppImageTool.

Status

  • SeExpr-based Fill Layer generator: merged
  • Creation and management of SeExpr scripts: merged
  • Integration tests: merged
  • User documentation: feature complete, awaiting review
  • Feature support status review: merged
  • Multithread support for Fill Layers: merged
  • ARM support for AppImages: merged

Deliverables

Retrieved from "https://community.kde.org/index.php?title=GSoC/2020/StatusReports/LeonardoEmanuelSegovia&oldid=89371"