Guidelines and HOWTOs/UnitTests

Author: Brad Hards, Sigma Bravo Pty Limited

Abstract

This article provides guidance on writing unittests for Qt4 and KDE4, based on the QtTestLib framework provided with Qt4.1. It provides an introduction to the ideas behind unit testing, tutorial material on the QtTestLib framework, and suggestions for getting the most value for your effort. This document is matched to Qt 4.1 and KDE 4.

About Unit Testing

A unit test is a test that checks the functionality, behaviour and correctness of a single software component. In Qt4 code (including KDE4 code) unit tests are almost always used to test a single C++ class (although testing a macro or C function is also possible).

Unit tests are a key part of Test Driven Development, however they are useful for all software development processes. It is not essential that all of the code is covered by unit tests (although that is obviously very desirable!). Even a single test is a useful step to improving code quality.

Even if they don't call them "unit tests", most programmers have written some "throwaway" code that they use to check an implementation. If that code was cleaned up a little, and built into the development system, then it could be used over and over to check that the implementation is still OK. To make that work a little easier, we can use test frameworks.

Note that it is sometimes tempting to treat the unit test as a pure verification tool. While it is true that unit tests do help to ensure correct functionality and behaviour, they also assist with other aspects of code quality. Writing a unit test requires a slightly different approach to coding up a class, and thinking about what inputs need to be tested can help to identify logic flaws in the code (even before the tests get run). In addition, the need to make the code testable is a very useful driver to ensure that classes do not suffer from close coupling.

Anyway, enough of the conceptual stuff - lets talk about a specific tool that can reduce some of the effort and let us get on with the job.

About QtTestLib

QtTestlib is a lightweight testing library developed by Trolltech and released under the GPL (a commercial version is also available, for those who need alternative licensing). It is written in C++, and is cross-platform. It is provided as part of the tools included in Qt 4.1, and earlier versions were provided separately. Note that earlier versions had a different API, and the examples will not work with that earlier version.

In addition to normal unit testing capabilities, QtTestLib also offers basic GUI testing, based on sending QEvents. This allows you to test GUI widgets, but is not generally suitable for testing full applications.

Each testcase is a standalone test application. Unlike CppUnit or JUnit, there is no Runner type class. Instead, each testcase is an executable which is simply run.

Tutorial 1 - A simple test of a date class

In this tutorial, we will build a simple test for a class that represents a date, using QtTestLib as the test framework. To avoid too much detail on how the date class works, we'll just use the QDate. class that comes with Qt. In a normal unittest, you would more likely be testing code that you've written yourself.

The code below is the entire testcase.

Example 1. QDate test code

#include <QtTest>
#include <QtCore>

class testDate: public QObject
{
    Q_OBJECT
private slots:
    void testValidity();
    void testMonth();
};

void testDate::testValidity()
{
    // 11 March 1967
    QDate date( 1967, 3, 11 );
    QVERIFY( date.isValid() );
}

void testDate::testMonth()
{
    // 11 March 1967
    QDate date;
    date.setYMD( 1967, 3, 11 );
    QCOMPARE( date.month(), 3 );
    QCOMPARE( QDate::longMonthName(date.month()),
              QString("March") );
}


QTEST_MAIN(testDate)
#include "tutorial1.moc"

Stepping through the code, the first line imports the header files for the QtTest namespace. The second line imports the headers for the QtCore namespace (not strictly necessary, since QtTest also imports it, but it is robust and safe). Lines 3 to 9 give us the test class, testData. Note that testDate inherits from QObject and has the Q_OBJECT macro - QtTestLib requires specific Qt functionality that is present in QObject.

Lines 10 to 15 provide our first test, which checks that a date is valid. Note the use of the QVERIFY macro, which checks that the condition is true. So if date.isValid() returns true, then the test will pass, otherwise the test will fail. QVERIFY is similar to ASSERT in other test suites.

Similarly, lines 16 to 24 provide another test, which checks a setter, and a couple of accessor routines. In this case, we are using QCOMPARE, which checks that the conditions are equal. So if date.month() returns 3, then that part of that test will pass, otherwise the test will fail.

'Note: As soon as a QVERIFY evaluates to false or a QCOMPARE does not have two equal values, the rest of that test will be skipped over. So in the example above, if the check at line 21 fails, then the check at lines 22 and 23 will not be run.

In a later tutorial we will see how to work around problems that this behaviour can cause.

Line 25 uses the QTEST_MAIN which creates an entry point routine for us, with appropriate calls to invoke the testDate unit test class.

Line 26 includes the Meta-Object compiler output, so we can make use of our QObject functionality.

The qmake project file that corresponds to that code is shown below. You would then use qmake to turn this into a Makefile and then compile it with make.

Example 2. QDate unit test project

CONFIG += qtestlib
TEMPLATE = app
TARGET +=
DEPENDPATH += .
INCLUDEPATH += .

# Input
SOURCES += tutorial1.cpp

This is a fairly normal project file, except for the addition of the CONFIG += qtestlib. This adds the right header and library setup to the Makefile.

This will produce an application that can then be run on the command line. The output looks like the following:

Example 3. QDate unit test output

$ ./tutorial1
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
PASS   : testDate::initTestCase()
PASS   : testDate::testValidity()
PASS   : testDate::testMonth()
PASS   : testDate::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********

Looking at the output above, you can see that the output includes the version of the test library and Qt itself, and then the status of each test that is run. In addition to the testValidity and testMonth tests that we defined, there is also a setup routine (initTestCase) and a teardown routine (cleanupTestCase) that can be used to do additional configuration if required.

Failing tests

If we had made an error in either the production code or the unit test code, then the results would show an error. An example is shown below:

Example 4. QDate unit test output showing failure

$ ./tutorial1
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
PASS   : testDate::initTestCase()
PASS   : testDate::testValidity()
FAIL!  : testDate::testMonth() Compared values are not the same
   Actual (date.month()): 4
   Expected (3): 3
    Loc: [tutorial1.cpp(25)]
PASS   : testDate::cleanupTestCase()
Totals: 3 passed, 1 failed, 0 skipped
********* Finished testing of testDate *********

Running selected tests

When the number of test functions increases, and some of the functions take a long time to run, it can be useful to only run a selected function. For example, if you only want to run the testMonth function, then you just specify that on the command line, as shown below:

Example 5. QDate unit test output - selected function

$ ./tutorial1 testValidity
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
PASS   : testDate::initTestCase()
PASS   : testDate::testValidity()
PASS   : testDate::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********

Note that the initTestCase and cleanupTestCase routines are always run, so that any necessary setup and cleanup will still be done.

You can get a list of the available functions by passing the -functions option, as shown below:

Example 6. QDate unit test output - listing functions

$ ./tutorial1 -functions
testValidity()
testMonth()

Verbose output options

You can get more verbose output by using the -v1, -v2 and -vs options. -v1 produces a message on entering each test function. I found this is useful when it looks like a test is hanging. This is shown below:

Example 7. QDate unit test output - verbose output

$ ./tutorial1 -v1
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
INFO   : testDate::initTestCase() entering
PASS   : testDate::initTestCase()
INFO   : testDate::testValidity() entering
PASS   : testDate::testValidity()
INFO   : testDate::testMonth() entering
PASS   : testDate::testMonth()
INFO   : testDate::cleanupTestCase() entering
PASS   : testDate::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********

The -v2 option shows each QVERIFY, QCOMPARE and QTEST, as well as the message on entering each test function. I found this useful for verifying that a particular step is being run. This is shown below:

'Example 8. QDate unit test output - more verbose output

$ ./tutorial1 -v2
********* Start testing of testDate *********
Config: Using QTest library 4.1.0, Qt 4.1.0-snapshot-20051003
INFO   : testDate::initTestCase() entering
PASS   : testDate::initTestCase()
INFO   : testDate::testValidity() entering
INFO   : testDate::testValidity() QVERIFY(date.isValid())
    Loc: [tutorial1.cpp(17)]
PASS   : testDate::testValidity()
INFO   : testDate::testMonth() entering
INFO   : testDate::testMonth() COMPARE()
    Loc: [tutorial1.cpp(25)]
INFO   : testDate::testMonth() COMPARE()
    Loc: [tutorial1.cpp(27)]
PASS   : testDate::testMonth()
INFO   : testDate::cleanupTestCase() entering
PASS   : testDate::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of testDate *********

The -vs option shows each signal that is emitted. In our example, there are no signals, so -vs has no effect. Getting a list of signals is useful for debugging failing tests, especially GUI tests which we will see in the third tutorial.

Output to a file

If you want to output the results of your testing to a file, you can use the -o filename, where you replace filename with the name of the file you want to save output to.

Continued

This tutorial has not been fully migrated yet. Please continue porting from http://developer.kde.org/documentation/tutorials/writingunittests/tute2.html