Krita/Docs/Unittests in Krita

What a unit test is and why it is needed?

Wiki: Unit test is a piece of code that automatically checks if your class or subsystem works correctly. The goal of unit testing is to isolate each part of the program and show that the individual parts are correct. A unit test provides a strict, written contract that the piece of code must satisfy. As a result, it affords several benefits. In other words unit testing allows the developer to verify if his initial design decisions has been implemented correctly and all the corner-cases are handled correctly.

Comment: In Krita Project we use unit tests for several purposes. Not all of them work equally good, but all together they help developing a lot.

Debugging of new code

Wiki: Unit testing finds problems early in the development cycle. This includes both bugs in the programmer's implementation and flaws or missing parts of the specification for the unit. The process of writing a thorough set of tests forces the author to think through inputs, outputs, and error conditions, and thus more crisply define the unit's desired behavior. The cost of finding a bug before coding begins or when the code is first written is considerably lower than the cost of detecting, identifying, and correcting the bug later; bugs may also cause problems for the end-users of the software.

Comment: Krita is a big project and has numerous subsystems that communicate with each other in complicated ways. It makes debugging and testing new code in the application itself difficult. What is more, just compiling and running the entire application to check a one-line change in a small class is very time-consuming. So when writing a new subsystem we usually split it into smaller parts (classes) and test each of them individually. Testing a single class in isolation helps to catch all the corner-cases in the class public interface, e.g. "what happens if we pass 0 here instead of a valid pointer?" or "what if the index we just gave to this method is invalid?"

Changing/refactoring existing code

Wiki: Unit testing allows the programmer to refactor code or upgrade system libraries at a later date, and make sure the module still works correctly (e.g., in regression testing). The procedure is to write test cases for all functions and methods so that whenever a change causes a fault, it can be quickly identified. Unit tests detect changes which may break a design contract.

Comment: Imagine someone decides to refactor the code you wrote a year ago. How would he know whether his changes didn't break anything in the internal class structure? Even if he/she asks you, how would you know if the changes to a year-old class, whose details are already forgotten, are valid?

Automated regression testing

Ideally, unit tests should also facilitate automated regression catching (ask Jenkins). But at the moment some of Krita unit tests are not stable enough to do the trick. They do straightforward QImage comparisons, so the test results can depend no only on version of the libraries installed, but also on build options and even type of CPU the tests are run on. In the future we plan to split such "unstable" unit test into a separate group and don't run them on Jenkins.

When to write a unit test?

Ideally a unit test should be written for any new class that implements some logic and provides any kind of public interface. It is especially true if this public interface is going to be used more that one client-class.

What should unit test include?

• corner cases. E.g. what happens if we request merging of two layers, one of which has Inherit Alpha option enabled? What properties and composition mode the final layer should have? Answers to these questions should be given and tested in the unit test.

• non-obvious design decisions. E.g. if a paint device has a non-transparent default pixel, then its ```exactBounds()``` returns the rect, not smaller that the size of the image, even though technically the device might be empty.

How to write a unittest?

Suppose you want to write a unittest for kritaimage library. You need to perform just a few steps:

1. Add files for the test class into ./image/tests/ direcotry:
   kis_some_class_test.h

   #ifndef __KIS_SOME_CLASS_TEST_H
   #define __KIS_SOME_CLASS_TEST_H
   
   #include <QtTest/QtTest>
   
   class KisSomeClassTest : public QObject
   {
       Q_OBJECT
   private slots:
       void test();
   };
   
   #endif /* __KIS_SOME_CLASS_TEST_H */
   

   
   kis_some_class_test.cpp

   #include "kis_some_class_test.h"
   
   #include <qtest_kde.h>
   
   void KisSomeClassTest::test()
   {
   }
   
   QTEST_KDEMAIN(KisSomeClassTest, GUI)
   

   
   
2. Modify ./image/tests/CMakeLists.txt to include your new test class
   

   ...
   ########### next target ###############
   set(kis_some_class_test_SRCS kis_some_class_test.cpp )
   kde4_add_unit_test(KisSomeClassTest TESTNAME kritaimage-some_class_test ${kis_some_class_test_SRCS})
   target_link_libraries(KisSomeClassTest  ${KDE4_KDEUI_LIBS} kritaimage ${QT_QTTEST_LIBRARY})
   ...
   

   
   
3. Write your test. You can use any macro commands provided by Qt (QVERIFY, QCOMPARE or QBENCHMARK).

   void KisSomeClassTest::test()
   {
       QString cat("cat");
       QString dog("dog");
   
       QVERIFY(cat != dog);
       QCOMPARE(cat, "cat");
   }
   

   
   
4. Run your test by running an executable in ./image/test/ folder

Krita-specific testing utils

Fetching reference images

All the testing files/images are usually stored in the test's data folder (e.g. ./krita/image/tests/data/). But there are some files which are used throughout all the unit tests (e.g. lena.png). These files are stored in the global folder ./krita/sdk/tests/data/. If you want to access any file, just use TestUtil::fetchDataFileLazy. It first searches the file in the local test's folder and if nothing is found checks the global folder.

Example:

QImage refImage(TestUtil::fetchDataFileLazy("lena.png"));
QVERIFY(!refImage.isNull());

Compare test result against a reference QImage

There are two helper functions to compare a given QImage against an image saved in the data folder.

bool TestUtil::checkQImage(const QImage &image, const QString &testName,
                           const QString &prefix, const QString &name,
                           int fuzzy = 0, int fuzzyAlpha = -1, int maxNumFailingPixels = 0);
bool TestUtil::checkQImageExternal(const QImage &image, const QString &testName,
                                   const QString &prefix, const QString &name,
                                   int fuzzy = 0, int fuzzyAlpha = -1, int maxNumFailingPixels = 0);

MaskParent object

TestUtil::MaskParent is a simple class that, in its constructor, creates an RGB8 image with a single paint layer, which you can use for further testing. The image and the layer can be accessed as simple member variables.

Example:

void KisMaskTest::testCreation()
{
    // create an image and a simple layer
    TestUtil::MaskParent p;

    // create a mask and attach its selection to the created layer
    TestMaskSP mask = new TestMask;
    mask->initSelection(p.layer);

    QCOMPARE(mask->extent(), QRect(0,0,512,512));
    QCOMPARE(mask->exactBounds(), QRect(0,0,512,512));
}