Kdenlive/Development/Testing: Difference between revisions

From KDE Community Wiki
 
(8 intermediate revisions by 3 users not shown)
Line 2: Line 2:
== Unit tests ==
== Unit tests ==


Kdenlive's unit tests are built upon two excellent tools: [https://github.com/eranpeer/FakeIt FakeIt] and [https://github.com/catchorg/Catch2/tree/master/docs Catch]


https://github.com/eranpeer/FakeIt
To run the tests, you first need to build the project. It's preferable to build in debugging mode  (see [[Kdenlive/Development|the building guide]]). Don't forget to add the flag <code>-DBUILD_TESTING=ON</code> to your CMake invocation.


https://github.com/catchorg/Catch2/tree/master/docs
Once the build is successful, you can simply run <code>./runTests</code> from your build directory to execute them.


== Fuzzing ==


===  Running the tests ===
If you want to try fuzzing, you need to enable it when you build the project. Add <code>-DBUILD_FUZZING=ON</code> to your cmake command line (see [[Kdenlive/Development#Building_Kdenlive|the building guide]]).


Building Kdenlive also builds a test application in /build. To run it:
=== First step: generate a corpus ===
<code>build/runTests/<code>
 
We will build a corpus thanks to the unit tests. From your build directory run:
<pre>
./runTests
mkdir corpus
mv fuzz_case* corpus
</pre>
 
=== Second step: Run ===
 
You will simply run the fuzzer continuously from your build dir, until you find a crash, if that ever happens:
<pre>
./fuzzer/fuzz -detect_leaks=0 corpus
</pre>
 
=== Third step: get the crash! ===
 
If you are (un)lucky, you may uncover a crash through the fuzzer.
LibFuzzer will save the faulty input in the current directory, under a name like "crash-9127ecb8d6f30e400d35c47c45bf406eb605b9bb".
 
The first thing you need to do is to try to minimize the crash. LibFuzzer will attempt to find the minimal input that triggers the crash:
<pre>
./fuzzer/fuzz  -detect_leaks=0 -exact_artifact_path=mycrash.txt -minimize_crash=1 crash-9127ecb8d6f30e400d35c47c45bf406eb605b9bb
</pre>
This will run for a while, if you feel like it's not making any more progress, you can stop it whenever you want. It will store the resulting minimized crash in <code>mycrash.txt</code>
 
If your minimized mycrash.txt happens to be 0 bytes, check that your original crash file is not a false positive with
<pre>
cat crash-9127ecb8d6f30e400d35c47c45bf406eb605b9bb |fuzzer/fuzz_reproduce
</pre>
 
Congratulations! You have now found your first fuzzing bug in Kdenlive. Please [https://invent.kde.org/kde/kdenlive/issues report it here], don't forget to add the <code>mycrash.txt</code> that you found.


== Continuous integration ==
== Continuous integration ==

Latest revision as of 21:47, 15 April 2019

Unit tests

Kdenlive's unit tests are built upon two excellent tools: FakeIt and Catch

To run the tests, you first need to build the project. It's preferable to build in debugging mode (see the building guide). Don't forget to add the flag -DBUILD_TESTING=ON to your CMake invocation.

Once the build is successful, you can simply run ./runTests from your build directory to execute them.

Fuzzing

If you want to try fuzzing, you need to enable it when you build the project. Add -DBUILD_FUZZING=ON to your cmake command line (see the building guide).

First step: generate a corpus

We will build a corpus thanks to the unit tests. From your build directory run:

./runTests
mkdir corpus
mv fuzz_case* corpus

Second step: Run

You will simply run the fuzzer continuously from your build dir, until you find a crash, if that ever happens:

./fuzzer/fuzz -detect_leaks=0 corpus

Third step: get the crash!

If you are (un)lucky, you may uncover a crash through the fuzzer. LibFuzzer will save the faulty input in the current directory, under a name like "crash-9127ecb8d6f30e400d35c47c45bf406eb605b9bb".

The first thing you need to do is to try to minimize the crash. LibFuzzer will attempt to find the minimal input that triggers the crash:

./fuzzer/fuzz  -detect_leaks=0 -exact_artifact_path=mycrash.txt -minimize_crash=1 crash-9127ecb8d6f30e400d35c47c45bf406eb605b9bb

This will run for a while, if you feel like it's not making any more progress, you can stop it whenever you want. It will store the resulting minimized crash in mycrash.txt

If your minimized mycrash.txt happens to be 0 bytes, check that your original crash file is not a false positive with

cat crash-9127ecb8d6f30e400d35c47c45bf406eb605b9bb |fuzzer/fuzz_reproduce

Congratulations! You have now found your first fuzzing bug in Kdenlive. Please report it here, don't forget to add the mycrash.txt that you found.

Continuous integration

https://build.kde.org/

Coverity

https://scan.coverity.com/projects/kdenlive