Krita/Painterly stuff

Painterly stuff in Krita 2.0

Introduction

We've always said Krita was for creating images, not just manipulating them. And, in any case, manipulating images is more and more done with non-destructive apps using masks and filter overlays. In the 1.x series we've had two main attempts and one small attempts at painterly features: the watercolor colorspace, the wet and sticky colorspace and the smeary brush paintop. I don't count the silly little airbrush paintop. In 2.0 we've got the dynamic brush and the chinese brush doing things already -- but they are far from finished.

Design

The basic reason for adding painterly features to Krita is not to create an as-faithful-as-possible physics simulation of real-world artistic media, but rather to allow the artist to do his thing: to create art without getting bothered by the mechanics of the application. As the Pixar artist quoted by Bill Baxter in his thesis says, "My artist's toolbox has oils and acrylics—and a computer. But the computer's not tactile, and I miss that. You can't push junk around." (http://www.pixar.com/artistscorner/tia/interview.html) Our goal is to make it possible to push junk around.

Moreover: we want to make it easy to push junk around. Corel Painter, while fantastically versatile is not easy to use. Not only does the color you deposit in Corel Painter not have any tactile feel -- it's color, not junk -- but the user need to manipulate a multitude of sliders and controls to calibrate his brushes this way and that to get the right effect. Just read an issue of Corel Painter, the Official Magazine. And where Corel Painter does simulate junk, it does it in a severely compartmentalized way. Wet ink or wet watercolors are on their own layer, untouchable by other tools.

So, the design goals:

Make it easy to push junk around
No fence between junk and color manipulation
Use familiar real-world examples, but don't follow them slavishly
Make it possible to work on big canvases

Definitions

Tools: determine where on the canvas the paintops act.
Paintops: determine how the contents of the canvas (pixels, paint) are changed. Paintops simulate a particular type of tool: hard brush, pencil, chalk, palette knife etc.
Canvas: contains the pixels/paint. A layer is a canvas. (Note: Should we make the group layer the canvas with canvas properties like adsorbency and height, instead of each layer?)
Paint: contains the physical properties of the paint (wetness, stickiness etc) Paint can also be gunk like graphite or chalk.
Physics filters: continuously update the canvas and paint according to a physical simulation (drying, dripping)
Color model: defines the chromatic properties of paint. We can use any color model with painterly features, so we can also add more natural color models, like the famous Kubelka-Monk model.

Layer design

In 1.x we mixed painterly features in-band with color. This didn't work out and prevented us from fun features like copying the wetness of a layer onto another layer.

In 2.x we are going to take the painterly features outof-band. We will be following the Wet & Sticky Model by Tunde Cockshott here. This is not the most modern or accurate model, but it feasible and workable even on modest hardware. It will also make it possible to mess with the painterly masks themselves, like converting the face of the Mona Lisa to a wetness mask, adding it to another layer and let that layer dry in the shape of the Mona Lisa.

The overlays are part of the paintdevice, not the layer proper. (Note: is this a good thing or not? How about group layers and canvas height?)

Note: in this design the words "mask" and "overlay" are used interchangeably. The word "overlay" is the new term, but the code still uses the old work "mask".

At the moment we have defined the following types of painterly features:

wetness (kis_wetness_mask.h, simulates the wetness of pixels in the parent layer. Determines how many drying cycles the pixel can undergo before it's done and fixed. Drying also may mean spreading paint to other pixels, possibly creating borders of deeper color. See Salesin et. all.

stickiness kis_viscosity_mask, affects both the spreading and drying of paint. If paint is very viscous, it will not travel easily with gravity or dry very fast to the air.

gravity kis_gravity_mask.h, defines the direction in which wet material is induced to flow, and the rate at which it is pulled.

adsorbency kis_adsorbency_mask.h, defines the rate at which pigment is fixed onto the canvas, the rate at which wetness disappears into the canvas, taking pigment with it.

paint height kis_volume_mask.h, defines the spreading of paint through gravity (more volume means there's more to spread out) and the height of the layer of paint on the canvas. paint height + canvas height can be visualized using bumpmapping

canvas height kis_height_mask.h, defines the height of the canvas material. Together with adsorbency, this describes the features of canvas, paper etc. Other options could be slipperiness, although that's perhaps adequately described by height anyway.

mixability kis_mixability_mask.h, defines how easy paint travels from the canvas to the brush, i.e, how much the paint sticks to the hairs of the brush, and also how easy the pigment mixes with the pigment already on the canvas. Low mixability means that existing color is more completely replaced by the new color.)

Reflectivity kis_reflectivity_mask.h, defines the measure in which paint allows the light to pass through or be reflected by the layer.

Pigment Concentration kis_pigment_concentration_mask.h, defines the intensity of the color in the material. A lot of pigment may make the paint more opaque (but that could depend on the material) and means that the paint can spread more widely without losing its color.

These overlays can be added individually to layers. That means that some layers will not have, for instance, gravity, others won't have wetness. The paintops should therefore never assume that a particular layer has a certain paintelry overlay. See the current draft for a layer definition dialog box:

Note the "Permapaint" option. It should be possible to have paint that never becomes wet again. The permapaint option adds another paint device of the colorspace type of the layer. This allows glazing, which is just the old-fashioned word for alpha-blending.

So, the datamodel of a layer with all painterly features looks like this:

KisLayer
   KisPaintDeviceSP projection   // rendered representation of the layer
   KisPaintDeviceSP paintDevice  // pixels that can be changed by tools
       Map<Type,KisMaskSP> overlays  // painterly overlays
   KisPaintDeviceSP permaPaint   // Color that cannot be picked up by the brush,
   Vector<KisMaskSP> effectMasks // masks that contain filters or transformations

There are three main types of interaction with such a layer:

Paintops that are aware of the painterly overlays
Paintops that are not aware of the painterly overlays
Physics filters

Media types

The given set of overlays is enough to simulate gunk that can be pushed around. One thing that cannot be simulated (easily, one can do it using the permapaint paintdevice) is the difference between types of medium. That's to say, if the crayon paintop deposits dry pigment with a high pigment concentration in a given color, then it won't stay that way when wet paint is added: suddenly it will change into wet paint.

What is possible is to create different types of physics filters that act differently on the material deposited on the substrate.

Visualisation

The advantage is that with OpenGL GLSL it becomes very easy to use a mask to compute a particular effect (like bumpmapping for canvas height and lighting). Adrian Page already has made it possible to use glsl shaders in Krita; we should use that to the fullest for our natural media visualisation.

Physics filters

Physics filters simulate the properties of a particular real-world-like medium, like watercolors, oilpaint or ink. In a future version of krita, the physics filters will run continuously: for now we will add physics filters to the filters menu and allow the user to specify a certain amount of real-world time the filter should simulate (in, hopefully less time than real time).

For example, we could have an oilpaint drying filter, a watercolors drying filter and an ink drying filter, all acting on the same layer, if the user wants.

Canvas design

Ideally, the canvas should be a property of a group layer and affect all layers in that group. The canvas itself should be relatively fixed (let's not simulate erasing a hole in thin paper? Or should we?). What Corel Painter does by bumpmapping a layer and then allowing the painter to smear out the canvas surface as if it were.

Paintop design

When I designed the paintop system I intended each paintop to represent a particular kind of artist's tool -- so we'd have a chinese brush paintop, a filbert paintop, a crayon, felt tip, airbrush, stiff bristle brush, fine brush, lettering brush, charcoal, conté or painter's knife paintop. That goal still stands.

At the same time, we should not limit ourselves to paintops that represent known tools: the current pen paintop is a traditional shoup model aliased brush, the brush paintop ditto but anti-aliased and the dynamic paintop is programmable in ways real life tools will never be.

However, we should not create an uber-paintop that then shows another bit of gui to select various predefined but unrelated real-world analogues, that's too many layers of indirection.

There are two kinds of paintops:

painterly paintops (isPainterly() returns true)
traditional paintops (isPainterly() returns false)

Tool design

This is fine as it is.

Problems

Undo

One problem we have with the watercolor colorspace is that the drying filter creates undo states. Because the physics filter continuously modifies the whole paint device it becomes impossible to undo a user action by simply replacing some old tiles: those tiles will be wetter than the surrounding tiles.

Maybe we should for painterly canvases add a "checkpoint" button that checkpoints the whole canvas and disable undo?

References

Krita's main goal is to enable to people to create images from scratch. An important feature is approaching natural media in simplicity and richness. This goal is subsumed under the name "painterly stuff".

There has been done plenty of research in this area. Most does not come with source code, much runs on the massively parallel supercomputers sold by ATI and Nvidia, but most dissertations and papers are freely available.

Prior Research

Watercolors

Chinese brushes

Strassmann

Two Computation Approaches for creating Chinese Painting, Clara Chan

Moxi

Oils and Acrylics

Bill Baxter

Pen and ink

Fractal/Corel Painter

Corel Painter