Difference between revisions of "Get Involved/Quality"

Jump to: navigation, search
(Created page with '__NOTOC__ == Becoming a KDE Tester == Testing|leftIn the early 2000's there was a specific team at KDE which was focused on finding loose ends in KDE appli...')
 
m (A couple of http-to-https upgrades.)
 
(76 intermediate revisions by 9 users not shown)
Line 1: Line 1:
__NOTOC__
+
[[Category:Testing]]
== Becoming a KDE Tester ==
 
  
[[Image:testing.png|Testing|left]]In the early 2000's there was a specific team at KDE which was focused on finding loose ends in KDE applications and tying them together. This was a task of user case studies, writing articles, documentation, creating missing artwork for consistancy, and other miscellanea. Ultimately, this team contributed patches of code and documentation that really rounded out the KDE experience. This team is not active currently, and people that are doing this type of effort are generally working alone and are focused on specific use cases and tasks. If you are interested in an independent project like this, please continue reading for some ideas.
+
[[Image:testing.png|Quality|left]]In the early 2000's there was a specific team at KDE which was focused on finding loose ends in KDE applications and tying them together. This was a task of user case studies, writing articles, documentation, creating missing artwork for consistency, and other miscellanea. Ultimately, this team contributed patches of code and documentation that really rounded out the KDE experience.
  
== Communicating with the team ==
+
Early 2012 this team was revived and now has a [https://mail.kde.org/mailman/listinfo/kde-testing mailing list] as well as a channel called #kde-quality on irc.freenode.net.
  
There is no active KDE quality team, but if you are interested in contributing improvements to KDE, you will be well received at [irc://irc.kde.org/kde-devel <nowiki>#kde-devel</nowiki>] on irc.freenode.net.
+
There are many different domains a Quality team should cover (see a complete list here:  *http://techbase.kde.org/Contribute/Quality_Team), but as a newly starting team we decided to focus our work on testing, mainly also because of the reduced manpower we started with.
  
== Context Help: Whatsthis ==
+
= Testing =
  
Context help is inseparable from the dialogs and widgets, as they are the target of the context help. In fact, in order to write context help, you have to touch programming or programming tools. Indeed, the context help is a property of widgets. In object oriented programming, a property can have different values, and behave differently depending on the value. In Qt/KDE programming, the name of the property is "whatsthis", and its value is the text the context help is going to display.
+
== What exactly does testing mean? ==
  
Fortunately, this task is usually not very difficult, as there are good tools to deal with user interface design, and better, you will use the knowledge acquired here later when dealing with user interface in general. Using the Qt framework (Qt is the base of KDE technology), it is possible to separate code and user interface. You have two basic cases here: the user interface is written with the general code of application (usually .cpp files) or in Qt Designer files (.ui files: it is a XML document). The second case is the best to start with, as it is simpler to work with. If you don't have Qt Designer installed, you can do so by installing the devel package of Qt from your distribution or the Qt Designer package (if your distribution has more fine grained packages).
+
Testing is part of the overall Quality Assurance of software. More information about the exact definition can be found here: http://en.wikipedia.org/wiki/Software_quality_assurance and here: http://en.wikipedia.org/wiki/Software_testing
  
Here you can find a detailed guide for writing whatsthis using Qt Designer and working directly with the source code: [http://bddf.ca/~aseigo/whatsthis_tutorial/ WhatsThis Tutorial], by Aaron J. Seigo.
+
A very interesting read is this: http://www.thebraidytester.com/downloads/YouAreNotDoneYet.pdf
  
== Application Help Or Manuals ==
+
Please also have a look at the [http://community.kde.org/Getinvolved/Quality/Tutorial Tutorial on how to become a KDE Tester]
  
More information is available at [[getinvolved/documentation|Getting Involved: Documentation]]
+
== Initial steps ==
  
== User Interface ==
+
Since this is a new start we need to define the exact goal of this team. There is a [http://community.kde.org/Getinvolved/Quality/Brainstorming Brainstorming] page where ideas are gathered.
  
User interface is a very wide subject, and very subjective too, as something obvious to someone is absurd to others and vice versa. Therefore, don't assume, argue clearly, stating your logical steps. Your main tool discussing it are objective reasoning and good sense.
+
=== Wiki work ===
  
It is easy to perform a quick user interface analysis, but it is hard to convince people to change the interface. A good, convincing analysis can gain much if it incorporates information from the KDE guidelines, competing program and operational system analysis, general design principles found in many books, user testing or individual (anecdotal) feedback. It is a volunteer project, and even if everybody agree with you, someone has to implement it.
+
The basics is of course to establish a useful wiki resource. We currently use https://trello.com/kdetesting to avoid duplicate work. Please ping Anne-Marie (annma) or Myriam (Mamarok) in #kde-quality on irc.freenode.net to be added to the group.
  
The [http://mail.kde.org/mailman/listinfo/kde-usability KDE Usability Mailing List] is very active and a good place for discussing your ideas, and their homepage is at http://techbase.kde.org/Projects/Usability. If you are already an usability expert, please check [http://www.openusability.org/ OpenUsability.org], a project that brings open source developers and usability experts together, and is collaborating closely with KDE.
+
=== Trunk testing ===
  
Some documents guiding documents include the [http://developer.kde.org/documentation/standards/kde/style/basics/index.html  KDE User Interface Guidelines (design standards)] and [http://developer.kde.org/documentation/design/ui/index.html KDE User Interface Guidelines (design principles)].
+
Trunk testing can be done with packages as well as building from source: [[Plasma/InstallingNext]]
  
Some projects for analysis of user interfaces may include: checking that shortcut keys are coherent across KDE applications, making sure that dialogs are directly relevant to the interaction that the user would expect, and finding users of KDE software to see how they perform common workflows.
+
=== Beta testing ===
 +
 
 +
Please see [http://community.kde.org/Getinvolved/Testing/Beta/ the Beta subpage] for more information.
 +
 
 +
== Existing testing infrastructure ==
 +
 
 +
=== Continuous Integration (Jenkins) ===
 +
 
 +
KDE already runs a build server with [http://community.kde.org/Sysadmin/Jenkins Jenkins]: http://build.kde.org/ Please ask the KDE sysadmins if you would like to use it for your project.
 +
Who gets the results? Who fixes them? This tool needs to be really used.
 +
 
 +
=== Unit tests ===
 +
 
 +
Tutorial for unit tests in KDE: https://community.kde.org/Guidelines_and_HOWTOs/UnitTests
 +
 
 +
=== Code (syntax) tests ===
 +
 
 +
A static code analyzing tool is provided by the [http://englishbreakfastnetwork.org/ EnglishBreakfastNetwork].
 +
 
 +
Another static code analyzer is [http://sourceforge.net/apps/mediawiki/cppcheck/index.php?title=Main_Page cppcheck] which can be integrated with Jenkins.
 +
 
 +
The [http://clang-analyzer.llvm.org/ Clang Static Analyzer] is also a useful tool and can be integrated with Jenkins.
 +
 
 +
More information can also be found here: http://techbase.kde.org/Development/Tutorials/Code_Checking
 +
 
 +
Coverity Prevent is another tool, not Open Source but we can get the results from it.
 +
 
 +
Currently KDE is also subscribed to http://scan.coverity.com/, all developers can get an account on  it, the project admins just have to approve them.
 +
 
 +
Other available tools:
 +
 
 +
*[http://www.parasoft.com/static-analysis Prevent] by Parasoft
 +
*[http://www.klocwork.com/products/insight/source-code-analysis/ Klocwork]
 +
 
 +
=== Debugging ===
 +
 
 +
KDE already has an extensive wiki for debugging: http://techbase.kde.org/Development/Tutorials/Debugging
 +
 
 +
== Existing testing tools ==
 +
 
 +
An non-exhaustive and maybe not up-to-date list of testing tools can be found here: http://www.opensourcetesting.org/
 +
See also http://techbase.kde.org/Development/Tools#Quality_Assurance
 +
 
 +
{| class="wikitable" style="text-align: center;
 +
! Name
 +
! Description
 +
|-
 +
| QtTest
 +
| Qt provides a testing module that can be used for unit testing: https://doc.qt.io/qt-5/qttest-index.html There also is a possibility to do basic UI testing.
 +
|-
 +
| Valgrind
 +
| A tool to analyze memory leaks: http://techbase.kde.org/Development/Tools/Valgrind All apps should be ran through Valgrind on a regular basis, part of the Quality Assurance.
 +
|-
 +
| Piglit
 +
| A tool to test OpenGL drivers: https://people.freedesktop.org/~nh/piglit/ might be useful to test parts of KWin and other OpenGL applications.
 +
|-
 +
| Gamma Ray
 +
| A dynamic code analyzer: http://www.kdab.com/kdab-products/gammaray/ It is more a tool for developers to help them track down problems than a QA tool.
 +
|-
 +
| Testopia
 +
| Testopia provides a test case management together with Bugzilla. This is currently evaluated by the KDE sysadmins: http://www.mozilla.org/projects/testopia/ please be patient
 +
|-
 +
| Squish
 +
| Not Open Source software, but there is a free KDE version. Email [email protected] and say what you're doing to get it. Note that the KDE version isn't mentioned on the website. There is generic information: https://www.froglogic.com/
 +
|-
 +
| UI tests
 +
| We will need to evaluate what tool would be the best for KDE. A list can be found here: https://en.wikipedia.org/wiki/List_of_GUI_testing_tools (incomplete) and here: http://www.opensourcetesting.org
 +
|}
 +
 
 +
== ATP Examples ==
 +
 
 +
The following are examples for application testing procedures:
 +
 
 +
[http://websvn.kde.org/trunk/KDE/kdesdk/umbrello/test/ATP.txt?revision=1244714&view=markup Application Test Procedure for Umbrello]
 +
 
 +
 
 +
== Quality Guidelines ==
 +
 
 +
Plasma Applets: http://community.kde.org/Getinvolved/Testing/Plasma_Applet_Quality_Guidelines
 +
 
 +
= Bug handling =
 +
 
 +
== Bug triaging ==
 +
 
 +
An essential part in the testing process is to have a cleaned up bugzilla database in terns of actuality of the bugs. For more information about bug triaging and participating in the KDE Bugsquad please see also the [https://community.kde.org/Bugsquad KDE Bugsquad wiki]
 +
 
 +
== The Extra Mile ==
 +
 
 +
There also is an initiative that aims to help KDE applications and workspaces to identify and fix small bugs and UI issues which get in the way of the user:
 +
 
 +
* [http://community.kde.org/Getinvolved/Extra_Mile Extra Mile Initiative]

Latest revision as of 08:53, 26 October 2019


Quality

In the early 2000's there was a specific team at KDE which was focused on finding loose ends in KDE applications and tying them together. This was a task of user case studies, writing articles, documentation, creating missing artwork for consistency, and other miscellanea. Ultimately, this team contributed patches of code and documentation that really rounded out the KDE experience.

Early 2012 this team was revived and now has a mailing list as well as a channel called #kde-quality on irc.freenode.net.

There are many different domains a Quality team should cover (see a complete list here: *http://techbase.kde.org/Contribute/Quality_Team), but as a newly starting team we decided to focus our work on testing, mainly also because of the reduced manpower we started with.

Testing

What exactly does testing mean?

Testing is part of the overall Quality Assurance of software. More information about the exact definition can be found here: http://en.wikipedia.org/wiki/Software_quality_assurance and here: http://en.wikipedia.org/wiki/Software_testing

A very interesting read is this: http://www.thebraidytester.com/downloads/YouAreNotDoneYet.pdf

Please also have a look at the Tutorial on how to become a KDE Tester

Initial steps

Since this is a new start we need to define the exact goal of this team. There is a Brainstorming page where ideas are gathered.

Wiki work

The basics is of course to establish a useful wiki resource. We currently use https://trello.com/kdetesting to avoid duplicate work. Please ping Anne-Marie (annma) or Myriam (Mamarok) in #kde-quality on irc.freenode.net to be added to the group.

Trunk testing

Trunk testing can be done with packages as well as building from source: Plasma/InstallingNext

Beta testing

Please see the Beta subpage for more information.

Existing testing infrastructure

Continuous Integration (Jenkins)

KDE already runs a build server with Jenkins: http://build.kde.org/ Please ask the KDE sysadmins if you would like to use it for your project. Who gets the results? Who fixes them? This tool needs to be really used.

Unit tests

Tutorial for unit tests in KDE: https://community.kde.org/Guidelines_and_HOWTOs/UnitTests

Code (syntax) tests

A static code analyzing tool is provided by the EnglishBreakfastNetwork.

Another static code analyzer is cppcheck which can be integrated with Jenkins.

The Clang Static Analyzer is also a useful tool and can be integrated with Jenkins.

More information can also be found here: http://techbase.kde.org/Development/Tutorials/Code_Checking

Coverity Prevent is another tool, not Open Source but we can get the results from it.

Currently KDE is also subscribed to http://scan.coverity.com/, all developers can get an account on it, the project admins just have to approve them.

Other available tools:

Debugging

KDE already has an extensive wiki for debugging: http://techbase.kde.org/Development/Tutorials/Debugging

Existing testing tools

An non-exhaustive and maybe not up-to-date list of testing tools can be found here: http://www.opensourcetesting.org/ See also http://techbase.kde.org/Development/Tools#Quality_Assurance

Name Description
QtTest Qt provides a testing module that can be used for unit testing: https://doc.qt.io/qt-5/qttest-index.html There also is a possibility to do basic UI testing.
Valgrind A tool to analyze memory leaks: http://techbase.kde.org/Development/Tools/Valgrind All apps should be ran through Valgrind on a regular basis, part of the Quality Assurance.
Piglit A tool to test OpenGL drivers: https://people.freedesktop.org/~nh/piglit/ might be useful to test parts of KWin and other OpenGL applications.
Gamma Ray A dynamic code analyzer: http://www.kdab.com/kdab-products/gammaray/ It is more a tool for developers to help them track down problems than a QA tool.
Testopia Testopia provides a test case management together with Bugzilla. This is currently evaluated by the KDE sysadmins: http://www.mozilla.org/projects/testopia/ please be patient
Squish Not Open Source software, but there is a free KDE version. Email [email protected] and say what you're doing to get it. Note that the KDE version isn't mentioned on the website. There is generic information: https://www.froglogic.com/
UI tests We will need to evaluate what tool would be the best for KDE. A list can be found here: https://en.wikipedia.org/wiki/List_of_GUI_testing_tools (incomplete) and here: http://www.opensourcetesting.org

ATP Examples

The following are examples for application testing procedures:

Application Test Procedure for Umbrello


Quality Guidelines

Plasma Applets: http://community.kde.org/Getinvolved/Testing/Plasma_Applet_Quality_Guidelines

Bug handling

Bug triaging

An essential part in the testing process is to have a cleaned up bugzilla database in terns of actuality of the bugs. For more information about bug triaging and participating in the KDE Bugsquad please see also the KDE Bugsquad wiki

The Extra Mile

There also is an initiative that aims to help KDE applications and workspaces to identify and fix small bugs and UI issues which get in the way of the user:


This page was last edited on 26 October 2019, at 08:53. Content is available under Creative Commons License SA 4.0 unless otherwise noted.