Phonon: Difference between revisions

From KDE Community Wiki
No edit summary
(→‎Getting Involved: IRC -> libera.chat)
 
(50 intermediate revisions by 8 users not shown)
Line 1: Line 1:
[[Image:Phonon-logo.png]]
[[Image:Phonon-logo.png]]
Welcome to the Phonon wiki space. Here, we try to document what is being worked on, and our plans for the immediate future. New tasks should go to [[Phonon/Ideas]] instead of being added here.
Releases are documented at [[Phonon/Releases]].
Feature matrix is at [[Phonon/FeatureMatrix]].
QA process is described at [[Phonon/QA]].
A list of envrionment variables for debugging is available [[Phonon/Environment_Variables|here]]. More information for debugging can also be found [https://community.kde.org/Guidelines_and_HOWTOs/Debugging/Phonon here]


== TODO ==
== TODO ==
* API to allow applications to implement their own effects (independent from the backend)
* encoding to files (or any QIODevice)
* Hardware Mixer API
* kded module to handle tasks like:
** react on plugged new (never seen before) media device
** manage software volume controls
** implement policy like muting/lowering volume of selected applications/categories depending on special events (e.g. incoming call)
* Let applications request specific audio capture formats (32 bit, 8 bit, little endian, etc)
* Documentation!
* Documentation!
** [[Phonon/Environment_Variables]]
** [[Phonon/Ideas]]
** [[Phonon/Sources]]
** [[Phonon/Leaks]]
* Document the complete phonon state machine, and make sure all backends implement it properly.
* Document the complete phonon state machine, and make sure all backends implement it properly.


=== Junior Jobs ===
=== Junior Jobs ===
* none :(
* Update/restructure the wiki to make it easier to find/plan out tasks: [[Phonon/Ideas]]
 
=== Done as GSoCs ===
* High level capture API (needs maturing for Phonon proper)
* Low level PCM I/O


=== GStreamer ===
=== GStreamer ===
* Ensure streamreader works (technically done, needs thread protection for KIO though)
* Ensure streamreader works (technically done, needs thread protection for KIO though)
* Implemenation of Phonon changes for spitfire
* Crossfading (negative transition time). Thread magic pain.
* Cleanup MO (maybe introduce statemachine? - MO below 2k lines \o/)
** Split it up into some smaller classes
*** PluginInstaller::requirePlugins(QStringList) - Returns false if the listed plugins do not exist. If the plugin install api is enabled, the plugins are installed.
*** A state machine
*** Maybe some kind of PipelineBuilder (Plumber?) class that does all the heavy lifting of building the pipelines for various sources instead of making the mediaobject be a swiss army knife of pipeline construction, message handling, source loading, and more.
* Support for more capture sources, such as ximagesrc


==== 4.6 release plan ====
==== 4.7 release plan ====
Release date: June 29, 2011
* Fix up the AudioOutputDevice to be responsible for creating the proper sink element, instead of the DeviceManager
* Clean up the code. A lot.
** Same with the VideoWidget, etc
** No: mediaobject.cpp:m_datasource = GST_ELEMENT(g_object_new(phonon_src_get_type(), NULL));
* Clean up the DeviceManager a bit
** Yes: m_datasource = gst_element_factory_make("phononsrc", NULL);
* Make sure PulseAudio works flawlessly
* Apparently subtitles sometimes render with HTML in them.
* Remove all kinds of mess that duplicates PulseAudio functionality, and defer it to the PulseSupport class.
* Do we really need a copy of alsasink2 around? I mean, its 2011...
* Implment autoaudiosink-esque logic for figuring out the default audio sink.
* Fix v4l capture
* Implement subtitles the right way
* Investigate opengl bugs
* Investigate if it is possible to eradicate MediaNodeEvent
* Implement the Plumber class
** Responsible for handling all the pipeline creation and linking events
* State machine!
* Check if we still need to queue all gstreamer signals. If not, mediaobject.cpp can shrink by a few hundred lines of code.
* Less debugging cruft, more useful debugging
** Hundreds of lines of "pipeline state set to X" isn't useful.
** Lines of "element rsndvdbin state set to X" is more useful.


=== VLC ===
=== VLC ===
* Ensure streamreader works (works with phonon threaded-streaming branch for QIOD, broken for KIO due to KIO not being threadsafe, needs protection)
* Ensure streamreader works (works with phonon threaded-streaming branch for QIOD, broken for KIO due to KIO not being threadsafe, needs protection)
* Implementation of Phonon changes for spitfire
* Crossfading (negative transition time). Thread magic pain.
* try --no-xlib again, was reverted by markey some time back as it caused problems apparently ... it prevents problems with xthread stuff such as we had with PulseAudio crash on exit


== Standing Issues ==
== Standing Issues ==
Line 65: Line 49:


# Standardize on names.
# Standardize on names.
=== Apps switching on backend switch ===
This issue is difficult to describe since it has multiple incarnations. The most occurring one is that at backend switch we can only partially destruct the path, leaving it in an inconsistent state from our POV and implicitly depend on the backend to nuke all nodes (only xine does that). In general the whole backend switching business is difficult to support because at every point in time different parts are (or needed) to be responsible for path deconstruction leading to multiple issues.
In an IRC discussion it was concluded that the backend switching is not a terribly important usecase and that we will stop supporting it.
* The KCM's part for switching needs to be moved to an own KCM that is not listed in systemettings
* Switching will only take effect on app restart, the KCM needs to tell the user about this
=== DVD Menus (how Dragon should not link against xine) ===
Currently Phonon does not provide the means for a video player to access the menu of its media (think DVD). This lead to a situation where dragonplayer links against xine to provide (primitive abilities of menu switching). A situation that can easily be prevented. The following implementation was considered: have MediaController get an enum of menus, introduce an interface to allow players to query what menus are available (neverminding if the backend or the medium does not support one). The player can then query what menus are there, since there is a fixed amount of supported menus it then can map the supported ones to (e.g.) QActions and upon action issues MC to instruct the backend to switch the menu. Possible menu types: main, title, root, audio, angle, chapter, subtitle.


=== State mess ===
=== State mess ===
Line 96: Line 70:
* Selective parts of backends
* Selective parts of backends
* Abstractstreaming - through the course of the great KIO wars of 2011 it became apparent that kiostreaming support needs a major remake but most of all that threading could make reader implementations much more flexible and doable than having all in the main thread or even mixed with callbacks from a framework thread. Additionally since KIO is not threadsafe putting up an internal design that forces us to protect our own thread would also protect KIO, which is all the more important considering that current or future backend implementations might get calls from a framework thread and direct action taking on those is potentially dangerous.
* Abstractstreaming - through the course of the great KIO wars of 2011 it became apparent that kiostreaming support needs a major remake but most of all that threading could make reader implementations much more flexible and doable than having all in the main thread or even mixed with callbacks from a framework thread. Additionally since KIO is not threadsafe putting up an internal design that forces us to protect our own thread would also protect KIO, which is all the more important considering that current or future backend implementations might get calls from a framework thread and direct action taking on those is potentially dangerous.
=== Testing ===
Our unit tests are in a sorry state, they should be cleaned up, and we should try to increase coverage. Also clean up the fake backend, maybe.


=== De-KDEification ===
=== De-KDEification ===
Phonon has a vast amount of KDE specific stuff going on that makes it terrible to use Phonon for Qt. This is becoming a major issue as Qt 4.8 will not have a bundled outdated version of Phonon anymore, but instead point interested parties to our tarballs.
Phonon has a vast amount of KDE specific stuff going on that makes it terrible to use Phonon for Qt. This is becoming a major issue as Qt 4.8 will not have a bundled outdated version of Phonon anymore, but instead point interested parties to our tarballs.
Out of the top of the writer's head standing issues here are: installation of backends to kde plugin paths && installation of headers to KDE/Phonon only. For source compatibility reasons we need to maintain the KDE header stuff, but at the same time also need to provide convenient ways to keep includes like <Phonon/MediaObject> working.
Out of the top of the writer's head standing issues here are: installation of backends to kde plugin paths && installation of headers to KDE/Phonon only. For source compatibility reasons we need to maintain the KDE header stuff, but at the same time also need to provide convenient ways to keep includes like <Phonon/MediaObject> working.
In addition, a nicely packaged SDK should be made available for folks to download which would install itself alongside the Qt SDK. That will keep everyone happy, as consumers wouldn't need to compile Phonon separately before using it on those platforms.
== Master Plan ==
=== 4.6.0 (git master) - dev name: hyperspeed ===
Estimated release: before KDE SC 4.7
==== Changes ====
* AbstractMediaStreams used for all URLs except file: (i.e. KIO streaming)
* Stabilize effects API to resolve random-naming issue (see above)
* <strike>Eliminate runtime backend switching (see above) </strike>
** Change KCM to not call the DBus if and instead show notfication
* Add subtitle support
* Mature Capturing API to move out of experimental
* Fix states (see above)
* Possibly MonsterNodes (see above)
* Evaluate threading options (see above - linked to signals)
* QML plugin (Audio at least)
* New debug system (import from Amarok)
* Figure out how to handle translations (install QTranslator? limited to non-plural translations...)
=== 4.5.1 (git 4.5) - dev name: spitfire ===
Estimated release: Whenever necessary
==== Changes ====
* None worth mentioning :P
=== 4.7.0 (not yet in development) - dev name: skylined ===
Estimated release: fall 2011
==== Changes ====
* Investigate usage of automoc and possibly drop it (rather inconvenient dep for Qt-only API consumers) '''Possibly in the cmake_tng branch'''
=== Phonon Five (TODO accumulation stage) - dev name: Hotel ===
Phonon5 will use Moby Album names for minor and song names for patch releases.
==== Changes ====
* Port to GTK+
* Redo AbstractMediaStream
* phononversion split out of phononnamespace


== Getting Involved ==
== Getting Involved ==
If you want to get involved with Phonon backend development please subscribe to [https://mail.kde.org/mailman/listinfo/phonon-backends [email protected]]. If you want to contribute, make comments or useful suggestions you can also write to the [mailto:[email protected] KDE Multimedia mailinglist]. You can join the mailinglist using the [https://mail.kde.org/mailman/listinfo/kde-multimedia web interface] or take a look at the [http://lists.kde.org/?l=kde-multimedia archives].
If you want to get involved with Phonon backend development please subscribe to [https://mail.kde.org/mailman/listinfo/phonon-backends [email protected]]. If you want to contribute, make comments or useful suggestions you can also write to the [mailto:[email protected] KDE Multimedia mailinglist]. You can join the mailinglist using the [https://mail.kde.org/mailman/listinfo/kde-multimedia web interface] or take a look at the [http://lists.kde.org/?l=kde-multimedia archives].


There's also a [irc://irc.kde.org/#phonon #phonon] channel on freenode where you may find developers.
There's also a [irc://irc.kde.org/#kde-multimedia #kde-multimedia] channel on Libera Chat where you may find developers.


== Release HOWTO ==
== Release HOWTO ==
Line 159: Line 90:
</pre>
</pre>


Get a number of people to test the tarball to make sure it passes the sanity checks (configures, compiles, installs, plays media). Afterwards, upload to upload.kde.org, poke Sho on IRC with the SHA and MD5 sums and wait ~6 hours. Finally, announce it to the world via blog and mail.
Get a number of people to test the tarball to make sure it passes the sanity checks (configures, compiles, installs, plays media). Afterwards, create a KDE sysadmin report at https://bugs.kde.org/enter_sysadmin_request.cgi. Include the SHA-1 and SHA-256 sums so they can verify. Also note that the destination for the upload should be stable/phonon/phonon-backend-gstreamer/4.5.1 (replace values as needed). Wait ~6 hours. Finally, announce it to the world via blog and mail to [email protected] along with your backend's own mailing announcement lists.


[https://bugs.kde.org]
[https://bugs.kde.org]

Latest revision as of 21:16, 18 June 2021

Welcome to the Phonon wiki space. Here, we try to document what is being worked on, and our plans for the immediate future. New tasks should go to Phonon/Ideas instead of being added here.

Releases are documented at Phonon/Releases.

Feature matrix is at Phonon/FeatureMatrix.

QA process is described at Phonon/QA.

A list of envrionment variables for debugging is available here. More information for debugging can also be found here

TODO

Junior Jobs

  • Update/restructure the wiki to make it easier to find/plan out tasks: Phonon/Ideas

GStreamer

  • Ensure streamreader works (technically done, needs thread protection for KIO though)
  • Crossfading (negative transition time). Thread magic pain.

4.7 release plan

  • Fix up the AudioOutputDevice to be responsible for creating the proper sink element, instead of the DeviceManager
    • Same with the VideoWidget, etc
  • Clean up the DeviceManager a bit
  • Make sure PulseAudio works flawlessly
  • Remove all kinds of mess that duplicates PulseAudio functionality, and defer it to the PulseSupport class.
  • Implment autoaudiosink-esque logic for figuring out the default audio sink.

VLC

  • Ensure streamreader works (works with phonon threaded-streaming branch for QIOD, broken for KIO due to KIO not being threadsafe, needs protection)
  • Crossfading (negative transition time). Thread magic pain.

Standing Issues

Effects API returning random identifier namers

Amarok finds an Equalizer by looking at the names of the available effects, for xine that would be KEqualizer (which is what Amarok looks for), Gstreamer however uses completely different names...

3 Approaches suggested:

  1. Introduce categories - does not resolve original problem, since a consumer still needs a way to differ between devices, introduces additional complexity
  1. Fixed Enum - not very dynamic WRT additional magic supported by specific backends. The capabilities would just return enum values and the consumer then issues creation of e.g. Phonon::Effect::Equalizer
  1. Standardize on names.

State mess

States are a mess right now, not very well defined and attached to some very weird signals (like aboutToFinish)... nothing is reliable and it causes crappy implementations (see Amarok EngineController why that is bad). In particular discussion with some of our major API consumers should be conducted to find a solution. Suggested discussion partners: Amarok, Dragon, Bangerang

Graph destruction & Monster nodes

Amarok's enginecontroller has a function called initializePhonon which gets executed when phonon enters error state (in case of missing codec for example). This function deletes all phonon objects held by amarok and reconstructs the whole thing. While this is not a very good practise at all it raises the general concern that Phonon can not administer global graph destruction and instead depends on the backends to handle this (we also see this issue with runtime backend switches). This is a problem as the logic is only implemented in the xine backend and ends up in undefined behaviour land with other backends (mostly resulting in a segfault at some point). Harald suggested the introduction of a Phonon::MonsterNode which then ought to be used as base for the backend's media nodes. Within the ctor of the MonsterNode it can then register with some sort of global node tracker. Upon destruction the MonsterNode unregisters and takes care of path destruction etc. That way it can be ensured globally that the graph is always in a consistent state.

Signals, Threads and Deletes

All signals that get forwarded by Phonon from the backend to the consumer should be queued. This is to "simulate" threading, as signal within the same thread context (i.e. the main app one) would get executed right away ontop of the currently present stack. If a backend emits statechange to error without queuing this will directly end up in a slot call in for example Amarok (note: no stack unwinding happened at this point, in fact it grew), Amarok then does things to it (at the time of writing: reinitialize everything by deleting all Phonon objects), once Amarok is done the stack unwinds to the emission in the backend, and the backend continues excuting statements (if any). At this point the presen stack operates on not allocated memory! To prevent this all signals must be queued, so that a backend can finish whatever it is doing, and only once the stack returned to the mainloop the slot on Amarok's side of things actually gets called. This allows both the backend and the consumer to do whatever they want to do. General note: it should be enforced that backends must not be running in a different thread context or if they do they must ensure that all used contexts are cleaned up properly (i.e. the dtors must only return once all threads are cleaned up).

QIODevice

We should not forget to QA QIODs as those will be used in Qt-only Phonon. It would appear that there are also some problems there. While ongoing kio related fixes should also bring great improvements to the QIOD side of things there are some special things that should be considered. In general it would appear that reading from a KIO slave before it is ready will lock the application as the read will never get data. Similar behaviour is to be expected with QIOD, which should only be read once it emitted its readyRead signal.

Threading and other madness

It should be evaluated where we should/could/need to introduce threading. Some general possibilities:

  • All backends at phonon level (or phonon itself for that matter) - this has unique value as any non-threaded operations in Phonon or the backends will lock the application and flood the apps' event loop with Phonon stuff.
  • Selective parts of backends
  • Abstractstreaming - through the course of the great KIO wars of 2011 it became apparent that kiostreaming support needs a major remake but most of all that threading could make reader implementations much more flexible and doable than having all in the main thread or even mixed with callbacks from a framework thread. Additionally since KIO is not threadsafe putting up an internal design that forces us to protect our own thread would also protect KIO, which is all the more important considering that current or future backend implementations might get calls from a framework thread and direct action taking on those is potentially dangerous.

De-KDEification

Phonon has a vast amount of KDE specific stuff going on that makes it terrible to use Phonon for Qt. This is becoming a major issue as Qt 4.8 will not have a bundled outdated version of Phonon anymore, but instead point interested parties to our tarballs. Out of the top of the writer's head standing issues here are: installation of backends to kde plugin paths && installation of headers to KDE/Phonon only. For source compatibility reasons we need to maintain the KDE header stuff, but at the same time also need to provide convenient ways to keep includes like <Phonon/MediaObject> working.

Getting Involved

If you want to get involved with Phonon backend development please subscribe to [email protected]. If you want to contribute, make comments or useful suggestions you can also write to the KDE Multimedia mailinglist. You can join the mailinglist using the web interface or take a look at the archives.

There's also a #kde-multimedia channel on Libera Chat where you may find developers.

Release HOWTO

$ git tag -s VERSION
$ git clone kde:releaseme
$ cd releaseme
$ ./phonon-backend-gstreamer.rb --git-branch master -b trunk -v VERSION -p ssh -u tdfischer
$ scp phonon-backend-gstreamer-VERSION.tar.bz2 some-public-server:phonon-backend-gstreamer-VERSION.tar.bz2

Get a number of people to test the tarball to make sure it passes the sanity checks (configures, compiles, installs, plays media). Afterwards, create a KDE sysadmin report at https://bugs.kde.org/enter_sysadmin_request.cgi. Include the SHA-1 and SHA-256 sums so they can verify. Also note that the destination for the upload should be stable/phonon/phonon-backend-gstreamer/4.5.1 (replace values as needed). Wait ~6 hours. Finally, announce it to the world via blog and mail to [email protected] along with your backend's own mailing announcement lists.

[1]