Phonon/QA: Difference between revisions

From KDE Community Wiki
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Testing ==
== Testing ==
=== Description ===
Functional tests are per application where the application type is denoted in brackets (A = Audio, V = Video, A/V = Audio & Video, G = Game = Audio).
Test cases have unique identifiers suffixing the name of the test. The identifier is composed of the test type (NF = NonFunctional, F = Functional), an app identifier (usually the first letter of the name, e.g. 'A' for Amarok) and a number. The number must be kept unique, so if a test is removed the number must not be reused.
Bug reports of a failed case ought to contain the unique identifier in the subject.


=== Non-functional ===
=== Non-functional ===
Line 11: Line 16:
'''NF-003: Build Demos'''
'''NF-003: Build Demos'''
* Build Phonon demos in equally clear environment.
* Build Phonon demos in equally clear environment.
'''NF-004: Binary Compatibility (Phonon only!)'''
* Git checkout previous version of Phonon
* Build with PHONON_BUILD_DESCRIPTOR
* Install using abi-check-install.sh
* Same for target release branch
* Run abi-compliance-checker on the two versions
* Ensure changes are binary compatible


'''NF-005: Build Consumers (Phonon only!)'''
'''NF-005: Build Consumers (Phonon only!)'''
Line 66: Line 63:
* Try to run Bangarang with libphonon
* Try to run Bangarang with libphonon


==== Gwenview (A/V - Surface Painting/QGV) ====
'''F-G001: Run'''
* Try to run Gwenview linked against libphonon
'''F-G002: Video'''
* Play a video of <720p
** Switch to fullscreen and back
* Play a video of 720p
** Switch to fullscreen and back
* Play a video of 1080p
** Switch to fullscreen and back
'''F-G003: Video Seeking'''
* Seek a video, forward and backwards
'''F-G004: Video Pause & Seek'''
* Pause a video
** Resume the video
* Pause a video and seek
** Resume the video
'''F-G005 Volume'''
* Change volume at various occasions


==== Dragon Player (A/V) ====
==== Dragon Player (A/V) ====
'''F-D002: Run'''
'''F-D002: Run'''
* Try to run Dragon with libphonon
* Try to run Dragon linked against libphonon


'''F-D003: Video'''
'''F-D003: Video'''
* Play a video of <720p
* Play a video of <720p
** Switch to fullscreen
** Switch to fullscreen and back
* Play a video of 720p
* Play a video of 720p
** Switch to fullscreen
** Switch to fullscreen and back
* Play a video of 1080p
* Play a video of 1080p
** Switch to fullscreen
** Switch to fullscreen and back


'''F-D004: Video Control'''
'''F-D004: Video Control'''
* http://streams.videolan.org/samples/Matroska/subtitles/multiple_sub_sample.mkv
* http://streams.videolan.org/samples/Matroska/subtitles/multiple_sub_sample.mkv
* http://streams.videolan.org/samples/Matroska/vorbis-audio-switch.mkv
* Play an MKV with subtitles and audio channels make sure Dragon can control them appropriately
* Play an MKV with subtitles and audio channels make sure Dragon can control them appropriately
* Make sure non-ASCII characters are displayed properly
* Make sure non-ASCII characters are displayed properly
Line 112: Line 133:
* Run the following command repeatedly:
* Run the following command repeatedly:
** kdialog --error "test" &
** kdialog --error "test" &
==== KTuberling (G) ====
'''F-Kt001 Play'''
* Start KTuberling
* Click on an object to hear the name of the object
'''F-Kt002 Stress'''
* Click at different objects quickly for a longer period of time


==== Tomahawk (A) ====
==== Tomahawk (A) ====

Latest revision as of 10:11, 4 November 2013

Testing

Description

Functional tests are per application where the application type is denoted in brackets (A = Audio, V = Video, A/V = Audio & Video, G = Game = Audio). Test cases have unique identifiers suffixing the name of the test. The identifier is composed of the test type (NF = NonFunctional, F = Functional), an app identifier (usually the first letter of the name, e.g. 'A' for Amarok) and a number. The number must be kept unique, so if a test is removed the number must not be reused. Bug reports of a failed case ought to contain the unique identifier in the subject.

Non-functional

NF-001: Build Phonon

  • Build latest Phonon in a clean environment without Phonon in system path.

NF-002: Build Backend(s)

  • Build Phonon & Phonon backend(s) in a clean environment without Phonon in system path.

NF-003: Build Demos

  • Build Phonon demos in equally clear environment.

NF-005: Build Consumers (Phonon only!)

  • Build amarok
  • Build bangarang
  • Build dragon
  • Build kde-runtime
  • Build tomahawk

Functional

Amarok (A)

F-A001: Run

  • Try to run Amarok with libphonon, expect no crashes.

F-A002: Audio

  • Play a .wav audio file, expect audio
    • Adjust volume during playback, expect volume changes
    • Seek to the middle of a track, expect seeking
    • Seek back to the beginning, expect correct audio playback
    • Seek to the end, expect correct audio playback
    • Seek back to the beginning, expect correct audio playback

F-A003: MP3 Codec Failure

  • Remove system mp3 support
  • Restart Amarok and play a .mp3 file, expect failure.
    • For GStreamer, expect a dialog prompting the installation of required codecs. Allow it to install the codecs.
  • Install the codecs
  • Re-try playing the .mp3, expect success.
  • Restart Amarok
  • Play the .mp3, expect success

F-A004: Gapless Playback

  • Queue up at least 5 tracks suitable for testing
  • Start playback, allowing Amarok to play all five tracks. Expect no gaps.
  • Restart playback from beginning
  • Seek within 2 seconds of the end of the first track, immediately seek back to the beginning of the track. Expect correct playback behavior.
  • Skip the current track, expect the second track.
  • Seek within 2 seconds of the end of the track, allow Amarok to continue to the third track without a gap.
  • Allow Amarok to play until the last 2 seconds of the track, seek back to the beginning of the track, and allow Amarok to continue to the fourth track.
  • Allow Amarok to play until the last 2 seconds of the track, skip back to the first track.
  • Allow Amarok to play until the last 2 seconds of the track, skip to the last track.
  • Allow Amarok to finish, expect stopped state.

Bangarang (A/V)

F-B-001: Run

  • Try to run Bangarang with libphonon

Gwenview (A/V - Surface Painting/QGV)

F-G001: Run

  • Try to run Gwenview linked against libphonon

F-G002: Video

  • Play a video of <720p
    • Switch to fullscreen and back
  • Play a video of 720p
    • Switch to fullscreen and back
  • Play a video of 1080p
    • Switch to fullscreen and back

F-G003: Video Seeking

  • Seek a video, forward and backwards

F-G004: Video Pause & Seek

  • Pause a video
    • Resume the video
  • Pause a video and seek
    • Resume the video

F-G005 Volume

  • Change volume at various occasions

Dragon Player (A/V)

F-D002: Run

  • Try to run Dragon linked against libphonon

F-D003: Video

  • Play a video of <720p
    • Switch to fullscreen and back
  • Play a video of 720p
    • Switch to fullscreen and back
  • Play a video of 1080p
    • Switch to fullscreen and back

F-D004: Video Control

F-D005: Video Settings

  • Try changing the video settings (hue, saturation, brightness)

F-D006: Video Asepct

  • Try changing the video aspect ratio

F-D007: Video Seeking

  • Seek a video, forward and backwards

F-D008: Video Pause & Seek

  • Pause a video
    • Resume the video
  • Pause a video and seek
    • Resume the video

F-D009 Volume

  • Change volume at various occasions

KNotify (A)

F-K001 Play

  • Play a sound via the KCM
  • Run the following command:
    • kdialog --error "test"

F-K002 Stress

  • Quickly play a sound via the KCM repeatedly
  • Run the following command repeatedly:
    • kdialog --error "test" &

KTuberling (G)

F-Kt001 Play

  • Start KTuberling
  • Click on an object to hear the name of the object

F-Kt002 Stress

  • Click at different objects quickly for a longer period of time

Tomahawk (A)

F-T002 Run

  • Run tomahawk with new lib

F-T003 Play

  • Play a local track
  • Same for remote track

F-T004 Pause & Seek

  • Seek a local track
  • Pause a local track
    • Resume
  • Pause a local track
    • Seek the track
    • Resume
  • Same for remote track

F-T005 Next & Prev

  • Skip from a playing track to the next one
  • Skip back to the previous track
  • Same for remote track

F-T006 Repeat

  • Repeat single track >2 times
  • Repeat whole playlist (play last track, ensure it starts from beginning)
  • Same for remote track

F-T007 Shuffle

  • Shuffle through songs by skipping forward and backwards
  • Same for remote track

F-T008 Volume

  • Change volume at various occasions

F-T000 Foo