Wiki: A unit test is a piece of code that automatically checks if your class or subsystem works correctly. The goal of unit testing is to isolate each part of the program and show that the individual parts are correct. A unit test provides a strict, written contract that the piece of code must satisfy. As a result, it affords several benefits.[1]
Comment: In other words unit testing allows the developer to verify if his initial design decisions has been implemented correctly and all the corner-cases are handled correctly.
In Krita Project we use unit tests for several purposes. Not all of them work equally good, but all together they help developing a lot.
Wiki: Unit testing finds problems early in the development cycle. This includes both bugs in the programmer's implementation and flaws or missing parts of the specification for the unit. The process of writing a thorough set of tests forces the author to think through inputs, outputs, and error conditions, and thus more crisply define the unit's desired behavior. The cost of finding a bug before coding begins or when the code is first written is considerably lower than the cost of detecting, identifying, and correcting the bug later; bugs may also cause problems for the end-users of the software.[1]
Comment: Krita is a big project and has numerous subsystems that communicate with each other in complicated ways. It makes debugging and testing new code in the application itself difficult. What is more, just compiling and running the entire application to check a one-line change in a small class is very time-consuming. So when writing a new subsystem we usually split it into smaller parts (classes) and test each of them individually. Testing a single class in isolation helps to catch all the corner-cases in the class public interface, e.g. "what happens if we pass 0 here instead of a valid pointer?" or "what if the index we just gave to this method is invalid?"
Wiki: Unit testing allows the programmer to refactor code or upgrade system libraries at a later date, and make sure the module still works correctly (e.g., in regression testing). The procedure is to write test cases for all functions and methods so that whenever a change causes a fault, it can be quickly identified. Unit tests detect changes which may break a design contract.[1]
Comment: Imagine someone decides to refactor the code you wrote a year ago. How would he know whether his changes didn't break anything in the internal class structure? Even if he/she asks you, how would you know if the changes to a year-old class, whose details are already forgotten, are valid?
Ideally, unit tests should also facilitate automated regression catching (ask Jenkins). But at the moment some of Krita unit tests are not stable enough to do the trick. They do straightforward QImage comparisons, so the test results can depend no only on version of the libraries installed, but also on build options and even type of CPU the tests are run on. In the future we plan to split such "unstable" unit test into a separate group and don't run them on Jenkins.
Ideally a unit test should be written for any new class that implements some logic and provides any kind of public interface. It is especially true if this public interface is going to be used more that one client-class.
Suppose you want to write a unittest for kritaimage library. You need to perform just a few steps:
#ifndef __KIS_SOME_CLASS_TEST_H
#define __KIS_SOME_CLASS_TEST_H
#include <QtTest/QtTest>
class KisSomeClassTest : public QObject
{
Q_OBJECT
private Q_SLOTS:
void test();
};
#endif /* __KIS_SOME_CLASS_TEST_H */
kis_some_class_test.cpp
#include "kis_some_class_test.h"
#include <QTest>
void KisSomeClassTest::test()
{
}
QTEST_MAIN(KisSomeClassTest, GUI)
...
########### next target ###############
set(kis_some_class_test_SRCS kis_some_class_test.cpp )
kde4_add_unit_test(KisSomeClassTest TESTNAME kritaimage-some_class_test ${kis_some_class_test_SRCS})
target_link_libraries(KisSomeClassTest ${KDE4_KDEUI_LIBS} kritaimage ${QT_QTTEST_LIBRARY})
...
void KisSomeClassTest::test()
{
QString cat("cat");
QString dog("dog");
QVERIFY(cat != dog);
QCOMPARE(cat, "cat");
}
All the testing files/images are usually stored in the test's data folder (e.g. ./krita/image/tests/data/). But there are some files which are used throughout all the unit tests (e.g. lena.png). These files are stored in the global folder ./krita/sdk/tests/data/. If you want to access any file, just use TestUtil::fetchDataFileLazy. It first searches the file in the local test's folder and if nothing is found checks the global folder.
Example:
QImage refImage(TestUtil::fetchDataFileLazy("lena.png"));
QVERIFY(!refImage.isNull());
There are two helper functions to compare a given QImage against an image saved in the data folder.
bool TestUtil::checkQImage(const QImage &image, const QString &testName,
const QString &prefix, const QString &name,
int fuzzy = 0, int fuzzyAlpha = -1, int maxNumFailingPixels = 0);
bool TestUtil::checkQImageExternal(const QImage &image, const QString &testName,
const QString &prefix, const QString &name,
int fuzzy = 0, int fuzzyAlpha = -1, int maxNumFailingPixels = 0);
The functions search for a png file with path
./tests/data/<testName>/<prefix>/<prefix>_<name>.png
# or without a subfolder
./tests/data/<testName>/<prefix>_<name>.png
The supplied QImage is compared against the saved PNG, and the result is returned to the caller. If the images do not coincide, two images are dumped into the current directory: one with actual result and another with what is expected.
The second version of the function is different. It searches the image in "an external repository". The point is that PNG images occupy quite a lot of space and bloat the repository size. So we decided to put all the images that are big enough (>10KiB) into an external SVN repository. To configure an external test files repository on your computer, please do the following:
# create the tests data folder and enter it
mkdir ~/testsdata
cd ~/testsdata
#checkout the extra repository
svn checkout svn+ssh://svn@svn.kde.org/home/kde/trunk/tests/kritatests
export KRITA_UNITTESTS_DATA_DIR= ~/testsdata/kritatests/unittests
Sometimes you need to test some complex actions like cropping or transforming the whole image. The main problem of such action is that it should work correctly with any kind of layer or mask, e.g. KisCloneLayer, KisGroupLayer or even KisSelectionMask. To facilitate such complex testing conditions, Krita provides a special class QImageBasedTest. It helps you to create a really complex image and check the contents of its layers. You can find the best example of its usage in KisProcessingsTest. Basically, to use this class, one should derive it's own testing class from it, and call a set of callbacks, which do all the work. Let's consider the code from KisProcessingsTest:
// override QImageBasedTest class
class BaseProcessingTest : public TestUtil::QImageBasedTest
{
public:
BaseProcessingTest()
: QImageBasedTest("processings")
{
}
// The method is called by test cases. If the test fails, a set of PNG images
// is saved into working directory
void test(const QString &testname, KisProcessingVisitorSP visitor) {
// create an image and regenerate its projection
KisSurrogateUndoStore *undoStore = new KisSurrogateUndoStore();
KisImageSP image = createImage(undoStore);
image->initialRefreshGraph();
// check if the image is correct before testing anything
QVERIFY(checkLayersInitial(image));
// do the action we are trying to test
KisProcessingApplicator applicator(image, image->root(),
KisProcessingApplicator::RECURSIVE);
applicator.applyVisitor(visitor);
applicator.end();
image->waitForDone();
// check the result, and dump images if something went wrong
QVERIFY(checkLayers(image, testname));
// Check if undo(!) works correctly
undoStore->undo();
image->waitForDone();
if (!checkLayersInitial(image)) {
qWarning() << "NOTE: undo is not completely identical "
<< "to the original image. Falling back to "
<<"projection comparison";
QVERIFY(checkLayersInitialRootOnly(image));
}
}
};
TestUtil::MaskParent is a simple class that, in its constructor, creates an RGB8 image with a single paint layer, which you can use for further testing. The image and the layer can be accessed as simple member variables.
Example:
void KisMaskTest::testCreation()
{
// create an image and a simple layer
TestUtil::MaskParent p;
// create a mask and attach its selection to the created layer
TestMaskSP mask = new TestMask;
mask->initSelection(p.layer);
QCOMPARE(mask->extent(), QRect(0,0,512,512));
QCOMPARE(mask->exactBounds(), QRect(0,0,512,512));
}