Phonon: Difference between revisions

From KDE Community Wiki
(→‎Getting Involved: IRC -> libera.chat)
 
(105 intermediate revisions by 10 users not shown)
Line 1: Line 1:
[[Image:Phonon-logo.png]]
[[Image:Phonon-logo.png]]
Welcome to the Phonon wiki space. Here, we try to document what is being worked on, and our plans for the immediate future. New tasks should go to [[Phonon/Ideas]] instead of being added here.
Releases are documented at [[Phonon/Releases]].
Feature matrix is at [[Phonon/FeatureMatrix]].
QA process is described at [[Phonon/QA]].
A list of envrionment variables for debugging is available [[Phonon/Environment_Variables|here]]. More information for debugging can also be found [https://community.kde.org/Guidelines_and_HOWTOs/Debugging/Phonon here]


== TODO ==
== TODO ==
* API to allow applications to implement their own effects (independent from the backend)
* Documentation!
* encoding to files (or any QIODevice)
** [[Phonon/Environment_Variables]]
* Hardware Mixer API
** [[Phonon/Ideas]]
* kded module to handle tasks like:
** [[Phonon/Sources]]
** react on plugged new (never seen before) media device
** [[Phonon/Leaks]]
** manage software volume controls
* Document the complete phonon state machine, and make sure all backends implement it properly.
** implement policy like muting/lowering volume of selected applications/categories depending on special events (e.g. incoming call)


=== Junior Jobs ===
=== Junior Jobs ===
* Exchange K_D with Q_D since they are absolutely the same (except for a stinky whitespace in Q_D ;))
* Update/restructure the wiki to make it easier to find/plan out tasks: [[Phonon/Ideas]]
 
=== GStreamer ===
* Ensure streamreader works (technically done, needs thread protection for KIO though)
* Crossfading (negative transition time). Thread magic pain.
 
==== 4.7 release plan ====
* Fix up the AudioOutputDevice to be responsible for creating the proper sink element, instead of the DeviceManager
** Same with the VideoWidget, etc
* Clean up the DeviceManager a bit
* Make sure PulseAudio works flawlessly
* Remove all kinds of mess that duplicates PulseAudio functionality, and defer it to the PulseSupport class.
* Implment autoaudiosink-esque logic for figuring out the default audio sink.


=== Done as GSoCs ===
=== VLC ===
* High level capture API (needs maturing for Phonon proper)
* Ensure streamreader works (works with phonon threaded-streaming branch for QIOD, broken for KIO due to KIO not being threadsafe, needs protection)
* Low level PCM I/O
* Crossfading (negative transition time). Thread magic pain.


== Standing Issues ==
== Standing Issues ==
Line 29: Line 50:
# Standardize on names.
# Standardize on names.


=== Apps switching on backend switch ===
=== State mess ===
This issue is difficult to describe since it has multiple incarnations. The most occurring one is that at backend switch we can only partially destruct the path, leaving it in an inconsistent state from our POV and implicitly depend on the backend to nuke all nodes (only xine does that). In general the whole backend switching business is difficult to support because at every point in time different parts are (or needed) to be responsible for path deconstruction leading to multiple issues.
States are a mess right now, not very well defined and attached to some very weird signals (like aboutToFinish)... nothing is reliable and it causes crappy implementations (see Amarok EngineController why that is bad). In particular discussion with some of our major API consumers should be conducted to find a solution.
Suggested discussion partners: Amarok, Dragon, Bangerang


In an IRC discussion it was concluded that the backend switching is not a terribly important usecase and that we will stop supporting it.
=== Graph destruction & Monster nodes ===
* The KCM's part for switching needs to be moved to an own KCM that is not listed in systemettings
Amarok's enginecontroller has a function called initializePhonon which gets executed when phonon enters error state (in case of missing codec for example). This function deletes all phonon objects held by amarok and reconstructs the whole thing. While this is not a very good practise at all it raises the general concern that Phonon can not administer global graph destruction and instead depends on the backends to handle this (we also see this issue with runtime backend switches). This is a problem as the logic is only implemented in the xine backend and ends up in undefined behaviour land with other backends (mostly resulting in a segfault at some point). Harald suggested the introduction of a Phonon::MonsterNode which then ought to be used as base for the backend's media nodes. Within the ctor of the MonsterNode it can then register with some sort of global node tracker. Upon destruction the MonsterNode unregisters and takes care of path destruction etc. That way it can be ensured globally that the graph is always in a consistent state.
* Switching will only take effect on app restart, the KCM needs to tell the user about this


=== DVD Menus (how Dragon should not link against xine) ===
=== Signals, Threads and Deletes ===
Currently Phonon does not provide the means for a video player to access the menu of its media (think DVD). This lead to a situation where dragonplayer links against xine to provide (primitive abilities of menu switching). A situation that can easily be prevented. The following implementation was considered: have MediaController get an enum of menus, introduce an interface to allow players to query what menus are available (neverminding if the backend or the medium does not support one). The player can then query what menus are there, since there is a fixed amount of supported menus it then can map the supported ones to (e.g.) QActions and upon action issues MC to instruct the backend to switch the menu. Possible menu types: main, title, root, audio, angle, chapter, subtitle.
All signals that get forwarded by Phonon from the backend to the consumer should be queued. This is to "simulate" threading, as signal within the same thread context (i.e. the main app one) would get executed right away ontop of the currently present stack. If a backend emits statechange to error without queuing this will directly end up in a slot call in for example Amarok (note: no stack unwinding happened at this point, in fact it grew), Amarok then does things to it (at the time of writing: reinitialize everything by deleting all Phonon objects), once Amarok is done the stack unwinds to the emission in the backend, and the backend continues excuting statements (if any). At this point the presen stack operates on not allocated memory!
To prevent this all signals must be queued, so that a backend can finish whatever it is doing, and only once the stack returned to the mainloop the slot on Amarok's side of things actually gets called. This allows both the backend and the consumer to do whatever they want to do.
General note: it should be enforced that backends '''must''' not be running in a different thread context '''or''' if they do they must ensure that all used contexts are cleaned up properly (i.e. the dtors must only return once all threads are cleaned up).


== Release Schedule ==
=== QIODevice ===
=== 4.4.4 (git 4.4) - dev name: iguanos ===
We should not forget to QA QIODs as those will be used in Qt-only Phonon. It would appear that there are also some problems there. While ongoing kio related fixes should also bring great improvements to the QIOD side of things there are some special things that should be considered. In general it would appear that reading from a KIO slave before it is ready will lock the application as the read will never get data. Similar behaviour is to be expected with QIOD, which should only be read once it emitted its readyRead signal.
Estimated release: soonish


==== Changes ====
=== Threading and other madness ===
* New build system magic to facilitate independent backend building
It should be evaluated where we should/could/need to introduce threading. Some general possibilities:
* GStreamer and Xine released independently with build system changes
* All backends at phonon level (or phonon itself for that matter) - this has unique value as any non-threaded operations in Phonon or the backends will lock the application and flood the apps' event loop with Phonon stuff.
* Otherwise '''only''' bug fixes
* Selective parts of backends
* Abstractstreaming - through the course of the great KIO wars of 2011 it became apparent that kiostreaming support needs a major remake but most of all that threading could make reader implementations much more flexible and doable than having all in the main thread or even mixed with callbacks from a framework thread. Additionally since KIO is not threadsafe putting up an internal design that forces us to protect our own thread would also protect KIO, which is all the more important considering that current or future backend implementations might get calls from a framework thread and direct action taking on those is potentially dangerous.


=== 4.5.0 (git master) - dev name: spitfire ===
=== De-KDEification ===
Estimated release: before Randa 2011 dev sprint
Phonon has a vast amount of KDE specific stuff going on that makes it terrible to use Phonon for Qt. This is becoming a major issue as Qt 4.8 will not have a bundled outdated version of Phonon anymore, but instead point interested parties to our tarballs.
 
Out of the top of the writer's head standing issues here are: installation of backends to kde plugin paths && installation of headers to KDE/Phonon only. For source compatibility reasons we need to maintain the KDE header stuff, but at the same time also need to provide convenient ways to keep includes like <Phonon/MediaObject> working.
==== Changes ====
* AbstractMediaStreams used for all URLs except file: (i.e. KIO streaming)
* Stabilize effects API to resolve random-naming issue (see above)
* Eliminate runtime backend switching (see above)
* Add DVD menu support (see above)
* Add subtitle support
* Mature Capturing API to move out of experimental
* Fixed states


== Getting Involved ==
== Getting Involved ==
If you want to get involved with Phonon backend development please subscribe to [https://mail.kde.org/mailman/listinfo/phonon-backends [email protected]]. If you want to contribute, make comments or useful suggestions you can also write to the [mailto:[email protected] KDE Multimedia mailinglist]. You can join the mailinglist using the [https://mail.kde.org/mailman/listinfo/kde-multimedia web interface] or take a look at the [http://lists.kde.org/?l=kde-multimedia archives].
If you want to get involved with Phonon backend development please subscribe to [https://mail.kde.org/mailman/listinfo/phonon-backends [email protected]]. If you want to contribute, make comments or useful suggestions you can also write to the [mailto:[email protected] KDE Multimedia mailinglist]. You can join the mailinglist using the [https://mail.kde.org/mailman/listinfo/kde-multimedia web interface] or take a look at the [http://lists.kde.org/?l=kde-multimedia archives].


There's also a [irc://irc.kde.org/#phonon #phonon] channel on freenode where you may find developers.
There's also a [irc://irc.kde.org/#kde-multimedia #kde-multimedia] channel on Libera Chat where you may find developers.
 
== Release HOWTO ==
 
<pre>
$ git tag -s VERSION
$ git clone kde:releaseme
$ cd releaseme
$ ./phonon-backend-gstreamer.rb --git-branch master -b trunk -v VERSION -p ssh -u tdfischer
$ scp phonon-backend-gstreamer-VERSION.tar.bz2 some-public-server:phonon-backend-gstreamer-VERSION.tar.bz2
</pre>
 
Get a number of people to test the tarball to make sure it passes the sanity checks (configures, compiles, installs, plays media). Afterwards, create a KDE sysadmin report at https://bugs.kde.org/enter_sysadmin_request.cgi. Include the SHA-1 and SHA-256 sums so they can verify. Also note that the destination for the upload should be stable/phonon/phonon-backend-gstreamer/4.5.1 (replace values as needed). Wait ~6 hours. Finally, announce it to the world via blog and mail to [email protected] along with your backend's own mailing announcement lists.
 
[https://bugs.kde.org]

Latest revision as of 21:16, 18 June 2021

Welcome to the Phonon wiki space. Here, we try to document what is being worked on, and our plans for the immediate future. New tasks should go to Phonon/Ideas instead of being added here.

Releases are documented at Phonon/Releases.

Feature matrix is at Phonon/FeatureMatrix.

QA process is described at Phonon/QA.

A list of envrionment variables for debugging is available here. More information for debugging can also be found here

TODO

Junior Jobs

  • Update/restructure the wiki to make it easier to find/plan out tasks: Phonon/Ideas

GStreamer

  • Ensure streamreader works (technically done, needs thread protection for KIO though)
  • Crossfading (negative transition time). Thread magic pain.

4.7 release plan

  • Fix up the AudioOutputDevice to be responsible for creating the proper sink element, instead of the DeviceManager
    • Same with the VideoWidget, etc
  • Clean up the DeviceManager a bit
  • Make sure PulseAudio works flawlessly
  • Remove all kinds of mess that duplicates PulseAudio functionality, and defer it to the PulseSupport class.
  • Implment autoaudiosink-esque logic for figuring out the default audio sink.

VLC

  • Ensure streamreader works (works with phonon threaded-streaming branch for QIOD, broken for KIO due to KIO not being threadsafe, needs protection)
  • Crossfading (negative transition time). Thread magic pain.

Standing Issues

Effects API returning random identifier namers

Amarok finds an Equalizer by looking at the names of the available effects, for xine that would be KEqualizer (which is what Amarok looks for), Gstreamer however uses completely different names...

3 Approaches suggested:

  1. Introduce categories - does not resolve original problem, since a consumer still needs a way to differ between devices, introduces additional complexity
  1. Fixed Enum - not very dynamic WRT additional magic supported by specific backends. The capabilities would just return enum values and the consumer then issues creation of e.g. Phonon::Effect::Equalizer
  1. Standardize on names.

State mess

States are a mess right now, not very well defined and attached to some very weird signals (like aboutToFinish)... nothing is reliable and it causes crappy implementations (see Amarok EngineController why that is bad). In particular discussion with some of our major API consumers should be conducted to find a solution. Suggested discussion partners: Amarok, Dragon, Bangerang

Graph destruction & Monster nodes

Amarok's enginecontroller has a function called initializePhonon which gets executed when phonon enters error state (in case of missing codec for example). This function deletes all phonon objects held by amarok and reconstructs the whole thing. While this is not a very good practise at all it raises the general concern that Phonon can not administer global graph destruction and instead depends on the backends to handle this (we also see this issue with runtime backend switches). This is a problem as the logic is only implemented in the xine backend and ends up in undefined behaviour land with other backends (mostly resulting in a segfault at some point). Harald suggested the introduction of a Phonon::MonsterNode which then ought to be used as base for the backend's media nodes. Within the ctor of the MonsterNode it can then register with some sort of global node tracker. Upon destruction the MonsterNode unregisters and takes care of path destruction etc. That way it can be ensured globally that the graph is always in a consistent state.

Signals, Threads and Deletes

All signals that get forwarded by Phonon from the backend to the consumer should be queued. This is to "simulate" threading, as signal within the same thread context (i.e. the main app one) would get executed right away ontop of the currently present stack. If a backend emits statechange to error without queuing this will directly end up in a slot call in for example Amarok (note: no stack unwinding happened at this point, in fact it grew), Amarok then does things to it (at the time of writing: reinitialize everything by deleting all Phonon objects), once Amarok is done the stack unwinds to the emission in the backend, and the backend continues excuting statements (if any). At this point the presen stack operates on not allocated memory! To prevent this all signals must be queued, so that a backend can finish whatever it is doing, and only once the stack returned to the mainloop the slot on Amarok's side of things actually gets called. This allows both the backend and the consumer to do whatever they want to do. General note: it should be enforced that backends must not be running in a different thread context or if they do they must ensure that all used contexts are cleaned up properly (i.e. the dtors must only return once all threads are cleaned up).

QIODevice

We should not forget to QA QIODs as those will be used in Qt-only Phonon. It would appear that there are also some problems there. While ongoing kio related fixes should also bring great improvements to the QIOD side of things there are some special things that should be considered. In general it would appear that reading from a KIO slave before it is ready will lock the application as the read will never get data. Similar behaviour is to be expected with QIOD, which should only be read once it emitted its readyRead signal.

Threading and other madness

It should be evaluated where we should/could/need to introduce threading. Some general possibilities:

  • All backends at phonon level (or phonon itself for that matter) - this has unique value as any non-threaded operations in Phonon or the backends will lock the application and flood the apps' event loop with Phonon stuff.
  • Selective parts of backends
  • Abstractstreaming - through the course of the great KIO wars of 2011 it became apparent that kiostreaming support needs a major remake but most of all that threading could make reader implementations much more flexible and doable than having all in the main thread or even mixed with callbacks from a framework thread. Additionally since KIO is not threadsafe putting up an internal design that forces us to protect our own thread would also protect KIO, which is all the more important considering that current or future backend implementations might get calls from a framework thread and direct action taking on those is potentially dangerous.

De-KDEification

Phonon has a vast amount of KDE specific stuff going on that makes it terrible to use Phonon for Qt. This is becoming a major issue as Qt 4.8 will not have a bundled outdated version of Phonon anymore, but instead point interested parties to our tarballs. Out of the top of the writer's head standing issues here are: installation of backends to kde plugin paths && installation of headers to KDE/Phonon only. For source compatibility reasons we need to maintain the KDE header stuff, but at the same time also need to provide convenient ways to keep includes like <Phonon/MediaObject> working.

Getting Involved

If you want to get involved with Phonon backend development please subscribe to [email protected]. If you want to contribute, make comments or useful suggestions you can also write to the KDE Multimedia mailinglist. You can join the mailinglist using the web interface or take a look at the archives.

There's also a #kde-multimedia channel on Libera Chat where you may find developers.

Release HOWTO

$ git tag -s VERSION
$ git clone kde:releaseme
$ cd releaseme
$ ./phonon-backend-gstreamer.rb --git-branch master -b trunk -v VERSION -p ssh -u tdfischer
$ scp phonon-backend-gstreamer-VERSION.tar.bz2 some-public-server:phonon-backend-gstreamer-VERSION.tar.bz2

Get a number of people to test the tarball to make sure it passes the sanity checks (configures, compiles, installs, plays media). Afterwards, create a KDE sysadmin report at https://bugs.kde.org/enter_sysadmin_request.cgi. Include the SHA-1 and SHA-256 sums so they can verify. Also note that the destination for the upload should be stable/phonon/phonon-backend-gstreamer/4.5.1 (replace values as needed). Wait ~6 hours. Finally, announce it to the world via blog and mail to [email protected] along with your backend's own mailing announcement lists.

[1]