Phonon

Welcome to the Phonon wiki space. Here, we try to document what is being worked on, and our plans for the immediate future. New tasks should go to Phonon/Ideas instead of being added here.

The upcoming Randa 2011 sprint plans can be found at Phonon/Randa2011

A list of envrionment variables is available here

TODO

• Documentation!
  • Phonon/Leaks
• Document the complete phonon state machine, and make sure all backends implement it properly.

Junior Jobs

• Update/restructure the wiki to make it easier to find/plan out tasks: Phonon/Ideas

Done as GSoCs

• High level capture API (needs maturing for Phonon proper)
• Low level PCM I/O

GStreamer

• Ensure streamreader works (technically done, needs thread protection for KIO though)
• Convert to use playbin2. More or less done, needs extensive testing:
  • Play a .ogv: yes
  • Play a .mkv with missing codecs and get prompt for installation: yes
  • Play a .mkv with multiple embedded subtitles: yes
  • Seeking: yes
  • Gapless playback: yes
  • Playing a dvd: yes
  • State machine compliance: yes
  • Playing a movie: yes
  • DVB: todo
  • Streaming: yes
  • musics: yes
• Crossfading (negative transition time). Thread magic pain.

4.6 release plan

Release date: August 29, 2011

• Clean up the code. A lot.
  • No: mediaobject.cpp:m_datasource = GST_ELEMENT(g_object_new(phonon_src_get_type(), NULL));
  • Yes: m_datasource = gst_element_factory_make("phononsrc", NULL); done
• Apparently subtitles sometimes render with HTML in them.
• Do we really need a copy of alsasink2 around? I mean, its 2011... removed
• Fix v4l capture done
• Implement subtitles the right way done
• Investigate opengl bugs
• Investigate if it is possible to eradicate MediaNodeEvent done
• Implement the Plumber class done, with playbin2
  • Responsible for handling all the pipeline creation and linking events
• State machine! done
• Check if we still need to queue all gstreamer signals. If not, mediaobject.cpp can shrink by a few hundred lines of code. done, via pipeline.cpp
• Less debugging cruft, more useful debugging
  • Hundreds of lines of "pipeline state set to X" isn't useful.
  • Lines of "element rsndvdbin state set to X" is more useful.

VLC

• Ensure streamreader works (works with phonon threaded-streaming branch for QIOD, broken for KIO due to KIO not being threadsafe, needs protection)
• Crossfading (negative transition time). Thread magic pain.

Standing Issues

Effects API returning random identifier namers

Amarok finds an Equalizer by looking at the names of the available effects, for xine that would be KEqualizer (which is what Amarok looks for), Gstreamer however uses completely different names...

3 Approaches suggested:

1. Introduce categories - does not resolve original problem, since a consumer still needs a way to differ between devices, introduces additional complexity

1. Fixed Enum - not very dynamic WRT additional magic supported by specific backends. The capabilities would just return enum values and the consumer then issues creation of e.g. Phonon::Effect::Equalizer

1. Standardize on names.

Apps switching on backend switch

This issue is difficult to describe since it has multiple incarnations. The most occurring one is that at backend switch we can only partially destruct the path, leaving it in an inconsistent state from our POV and implicitly depend on the backend to nuke all nodes (only xine does that). In general the whole backend switching business is difficult to support because at every point in time different parts are (or needed) to be responsible for path deconstruction leading to multiple issues.

In an IRC discussion it was concluded that the backend switching is not a terribly important usecase and that we will stop supporting it.

• The KCM's part for switching needs to be moved to an own KCM that is not listed in systemettings
• Switching will only take effect on app restart, the KCM needs to tell the user about this

State mess

States are a mess right now, not very well defined and attached to some very weird signals (like aboutToFinish)... nothing is reliable and it causes crappy implementations (see Amarok EngineController why that is bad). In particular discussion with some of our major API consumers should be conducted to find a solution. Suggested discussion partners: Amarok, Dragon, Bangerang

Graph destruction & Monster nodes

Amarok's enginecontroller has a function called initializePhonon which gets executed when phonon enters error state (in case of missing codec for example). This function deletes all phonon objects held by amarok and reconstructs the whole thing. While this is not a very good practise at all it raises the general concern that Phonon can not administer global graph destruction and instead depends on the backends to handle this (we also see this issue with runtime backend switches). This is a problem as the logic is only implemented in the xine backend and ends up in undefined behaviour land with other backends (mostly resulting in a segfault at some point). Harald suggested the introduction of a Phonon::MonsterNode which then ought to be used as base for the backend's media nodes. Within the ctor of the MonsterNode it can then register with some sort of global node tracker. Upon destruction the MonsterNode unregisters and takes care of path destruction etc. That way it can be ensured globally that the graph is always in a consistent state.

Signals, Threads and Deletes

All signals that get forwarded by Phonon from the backend to the consumer should be queued. This is to "simulate" threading, as signal within the same thread context (i.e. the main app one) would get executed right away ontop of the currently present stack. If a backend emits statechange to error without queuing this will directly end up in a slot call in for example Amarok (note: no stack unwinding happened at this point, in fact it grew), Amarok then does things to it (at the time of writing: reinitialize everything by deleting all Phonon objects), once Amarok is done the stack unwinds to the emission in the backend, and the backend continues excuting statements (if any). At this point the presen stack operates on not allocated memory! To prevent this all signals must be queued, so that a backend can finish whatever it is doing, and only once the stack returned to the mainloop the slot on Amarok's side of things actually gets called. This allows both the backend and the consumer to do whatever they want to do. General note: it should be enforced that backends must not be running in a different thread context or if they do they must ensure that all used contexts are cleaned up properly (i.e. the dtors must only return once all threads are cleaned up).

QIODevice

We should not forget to QA QIODs as those will be used in Qt-only Phonon. It would appear that there are also some problems there. While ongoing kio related fixes should also bring great improvements to the QIOD side of things there are some special things that should be considered. In general it would appear that reading from a KIO slave before it is ready will lock the application as the read will never get data. Similar behaviour is to be expected with QIOD, which should only be read once it emitted its readyRead signal.

Threading and other madness

It should be evaluated where we should/could/need to introduce threading. Some general possibilities:

• All backends at phonon level (or phonon itself for that matter) - this has unique value as any non-threaded operations in Phonon or the backends will lock the application and flood the apps' event loop with Phonon stuff.
• Selective parts of backends
• Abstractstreaming - through the course of the great KIO wars of 2011 it became apparent that kiostreaming support needs a major remake but most of all that threading could make reader implementations much more flexible and doable than having all in the main thread or even mixed with callbacks from a framework thread. Additionally since KIO is not threadsafe putting up an internal design that forces us to protect our own thread would also protect KIO, which is all the more important considering that current or future backend implementations might get calls from a framework thread and direct action taking on those is potentially dangerous.

De-KDEification

Phonon has a vast amount of KDE specific stuff going on that makes it terrible to use Phonon for Qt. This is becoming a major issue as Qt 4.8 will not have a bundled outdated version of Phonon anymore, but instead point interested parties to our tarballs. Out of the top of the writer's head standing issues here are: installation of backends to kde plugin paths && installation of headers to KDE/Phonon only. For source compatibility reasons we need to maintain the KDE header stuff, but at the same time also need to provide convenient ways to keep includes like <Phonon/MediaObject> working.

Master Plan

4.6.0 (git master) - dev name: hyperspeed

Estimated release: some time

Changes

• AbstractMediaStreams used for all URLs except file: (i.e. KIO streaming)
• Stabilize effects API to resolve random-naming issue (see above)
• [DONE] Eliminate runtime backend switching (see above)
  • Change KCM to not call the DBus if and instead show notfication
• Add subtitle support (set file subtitle and what not)
• Mature Capturing API to move out of experimental
• Fix states (see above)
• Possibly MonsterNodes (see above)
• Evaluate threading options (see above - linked to signals)
• [WIP] QML plugin (Audio at least)
• [WIP] New debug system (import from Amarok)
• Figure out how to handle translations (install QTranslator? limited to non-plural translations...)
• Revise VDO: Use same impl for VGO and VDO, allow frame extraction, allow frame-by-frame playback (perhaps introduce VideoPlaybackControl to allow both in VGO?)

4.5.1 (git 4.5) - dev name: spitfire1

Estimated release: Whenever necessary

Changes

• None worth mentioning :P

4.7.0 (not yet in development) - dev name: thunder

Estimated release: fall 2011

Changes

• Investigate usage of automoc and possibly drop it (rather inconvenient dep for Qt-only API consumers) Possibly in the cmake_tng branch

Phonon Five (TODO accumulation stage) - dev name: Hotel

Phonon5 will use Moby Album names for minor and song names for patch releases.

Changes

• Port to GTK+
• Redo AbstractMediaStream
• phononversion split out of phononnamespace

Getting Involved

If you want to get involved with Phonon backend development please subscribe to [email protected]. If you want to contribute, make comments or useful suggestions you can also write to the KDE Multimedia mailinglist. You can join the mailinglist using the web interface or take a look at the archives.

There's also a #phonon channel on freenode where you may find developers.

Release HOWTO

$ git tag -s VERSION
$ git clone kde:releaseme
$ cd releaseme
$ ./phonon-backend-gstreamer.rb --git-branch master -b trunk -v VERSION -p ssh -u tdfischer
$ scp phonon-backend-gstreamer-VERSION.tar.bz2 some-public-server:phonon-backend-gstreamer-VERSION.tar.bz2

Get a number of people to test the tarball to make sure it passes the sanity checks (configures, compiles, installs, plays media). Afterwards, upload to upload.kde.org, poke Sho on IRC with the SHA and MD5 sums and wait ~6 hours. Finally, announce it to the world via blog and mail.

[1]