New Decoration API
The current KDecoration API is rather dated, limited and not designed for the use in a composited setup. In order to provide ARGB decorations the PaintRedirector had been introduced which has been identified as a bottleneck. The API has hardly changed with KDE 4 and the only major change was the introduction of Window Tabbing. Unfortunately the API for Window Tabbing entered without a proper API review and without feedback of decoration developers making the API hard to use and impossible to maintain.
Most of the issues with the KDecoration API are not fixable without breaking the ABI promise. In fact most of the API issues are not even fixable in a source compatible way. Introducing a new API would mean that all existing decorations would not be usable any more rendering the only remaining decorations the maintained ones: Oxygen and Aurorae.
Therefore a new API has to be provided alongside the existing KDecoration API without inheriting or using anything of the existing API. In order to properly develop the API we should not install the headers or provide a stable ABI. Only decorations shipped with KWin should be allowed to use it. ABI stability should be guaranteed like it is done for KWin Effects.
API Changes Compared to KDecoration
Signals for Client Changes
All changes of the client should use signals and not virtuals. Using singals makes it easier to extend the API in future releases (e.g. new button types). The existing API is inconsistant as some buttons emit signals instead of using virtuals
All client states should be exported as QProperties. Getters and Setters should be Qt-stylish.
The Client Group API needs a major rework. Especially it is important to notify the client when clients are added or removed. It might be useful to rename everything to WindowTabs as that's how it is named in common speak. No need to have a tech speak.
Remove Border Sizes
Border sizes should be handled completly inside the decoration like it is done in Oxygen and Aurorae.
Remove Most of the Abilities
The only interesting Abilities are the DecorationButtons. All decorations should support ARGB, provide shadow, use blur and so on. In case we do allow clients to define what buttons to render without the user to interact the Abilities are not needed at all.
The concept of a common decoration does hardly make any sense for a modern decoration. Especially the window tabbing addition showed how difficult it is to extend the class in a usingful manner.
Not needed any more. We are composited and have an effect for nice resizing.
No Paint Redirector
The PaintRedirector should be dropped completly. Instead of letting the decoration create the widget, KWin core should create the widget and just pass a QPainter to the decoration. The QPainter could be created directly on the pixmap KWin uses for compositing.
Rendering outside the Decoration
Rendering outside the decoration like shadows should be completly possible. This has to go further than the current possibilities. E.g. button glow on hover has to be supported. Idea: decoration specifies a canvas area and a interaction area. The interaction area is used for mouse events, snapping etc, while the canvas is only rendered. This obsoletes the requirement for specifying border, padding and mask.
Idea to remove the Paint Redirector and rendering out of bounds at once
Since the interaction area is (as on X11) limited by the decoration widget one could keep the very old ::borders(&&&&) call (while the API is less great ;-) to set the widget size as well as setMask() to set the mask. On top of that the decoration should be apply to supply a chain of pixmaps with variable size to be painted as the decoration. This can be implemented either as list or -since the decorated client is rectangular- parameter query ::pixmap(Tile t) accomplished with positioning or even render instructions (the decoration might wish to keep a complete transparent padding around the client or paint on top of the client, what has not been possible so far) The pixmap will then be painted into the scene or the (masked) widget. The decoration might use different "pixmaps" (looks) for the un/composited case. Using a giant pixmap to let the decoration paint on causes memory usage overhead and keeps us the issue of the nvidia CSS driver we bought with the redirector. Also the client can this way keep prefabbed pixmaps for general usage (eg. the buttons or edges but the titlebar - esp. when allowing render instructions like "stretch/tile in this rect") - maybe there should even be support for a distinct "titlelabel" pixmap? --Luebking 20:27, 29 May 2011 (UTC)
Discuorage Button Widget usage
Currenty decorations have usually a layout and some button widgets for the close, min, max buttons what tends to cause visual glitches either with or without compositing / redirecting the paint since we really don't want to use double buffering for the decoration widget now and certainly not when using pixmaps directly. Therefore we'll require an API to ease the usage of pseudowidget buttons, ie. a) similar support for button pixmaps (plus position against the input area hint) b) let the decoration specify a "rule" for the button position ("20px from the left") as well as emitting a signal when the related area has been hoverd/pressed/released/wheeled. The signal should be generic, ie. provide a char parameter to hint the action for the receiving slot. This will not only allow adding button types w/o changing the API but even have the decoration add own button types ("hide menubar" and stuff) by itself. Also the signal must of course carry the button and modifiers. (The signal/slot might be superflous - we could also just pipe a user-defined event into the decoration object) --Luebking 20:27, 29 May 2011 (UTC)
To ease the most important use case it might be an idea to require (OpenGL) Compositing. If Compositing is not enabled KWin won't load the decoration and when compositing gets disabled KWin unloads the decoration and loads a legacy decoration. This would make the life easier for everyone.
In case we require OpenGL Compositing, we could hook the rendering into the normal Scene rendering allowing the decoration to render OpenGL code directly on the screen or maybe provide a QPainter with an OpenGL backend. Fredrik is working on a GL deco.
QML might be interesting for decorations. This could allow to just load a different QML file as the decoration while keeping the interaction in KWin core.
Properly not needed as obsoleted by QML.
Custom Decoration Buttons
Does it make sense to control the decoration buttons globally? Wouldn't it be better to give complete control about the decoration buttons to the decoration and let it decide which to render? (Probably users will consider this as a "GNOME" way of handling things ;-)
GPL instead of BSD
Given that the decoration API will not be made a stable ABI and not be exported, nobody can write decorations outside the KWin source tree obsoleting the need for allowing proprietary decorations. In case the decoration API allows to use QML, we are able to allow proprietary decorations even if the API is GPL. Furthermore it is my personal belief (mgraesslin writing here) that the decoration API is a derived work of KWin core due to exporting the state of the Client object, which requires that the API has to be GPL.
The complete API should have Unit tests and a well defined behavior.
There should be a test-application for Decorations and the API similar to AuroraeDesigner and oxygen-demo.