Frameworks/Akonadi

From KDE Community Wiki

This is a list of things I want to do in kdepimlibs/akonadi for Frameworks 5.

Akonadi repo structure

Merge client libraries and server into one repository. Main advantage is that we will be able to share more code between server and libraries.

Proposed structure:

/autotests
/doc
/src
/src/agentbase (libKF5AkonadiAgentBase.so - AgentBase, ResourceBase and other classes related to agents. Needs a better name)
/src/agentlauncher (utility for loading and running plugin-based agents)
/src/akonadictl (akonadictl utility)
/src/asapcat (asapcat utility)
/src/common (libKF5AkonadiCommon.so - private classes shared between server, core and widgets libraries and other utilities. Does NOT install any (or most of its) public headers)
/src/control (akonadi_control utility)
/src/core (libKF5AkonadiCore.so - non-UI classes, only Jobs, models, Monitor etc.)
/src/core/jobs 
/src/core/models
/src/interfaces (DBus interfaces)
/src/qsqlite (Qt's QSQLITE driver fork - better try to upstream it)
/src/selftest (based on SelfTestDialog class)
/src/server (akonadiserver)
/src/widgets (libKF5AkonadiWidgets.so - UI classes, depends on core)
/src/widgets/views
/src/xml (akonadi/xml library)
/tests/akonaditest (akonaditest utility)
/tests/fake (fake classes to be used for testing)
/tests/searchplugin (akonadi_testsearch_plugin)
/tests/testresource (akonadi_knut_resource)

Client libraries

Code changes

Status Action Comment
IN PROGRESS Fix coding-style (Guy) We have to follow the Frameworks coding style
TO DO Port to Qt 5 Make client libraries compile against Qt 5 <{{{3}}}>
TO DO Replace BOOST_STATIC_ASSERT C++11 has static_assert, so use that <{{{3}}}>

API/ABI changes

Status Class Action Comment Branch Assignee
TODO AgentBase Merge ObserverV2 into Observer There should be "Observer", "BatchObserver" and "TagsObserver" (not inheriting from each other). BatchObserver would enable the same behavior as ObserverV3 currently, TagsObserver would enable tags notifications. frameworks
TODO AgentSearchInterface Remove addSearch() and removeSearch() methods. Search is managed on server now, so we only need search() method to get results. add/removeSearch() are no longer used frameworks
TODO AgentBase Merge ObserversV* into Observer There should be "Observer" and "BatchObserver" (not inheriting from each other). BatchObserver would enable the same behavior as ObserverV3 currently frameworks
TODO ChangeMediator Remove unused methods ChangeMediator is only used by ModifyJobs to invalidate entries in Monitor's EntityCache, but it has bunch of stub methods we don't need, so remove them master
TODO Collection Remove deprecated methods frameworks
TODO CollectionFetchJob Remove deprecated methods frameworks
TODO CollectionFetchScope Remove deprecated methods frameworks
TODO CollectionModel Remove frameworks
TODO CollectionSelectJob Remove? Performs SELECT before FETCH, STORE, APPEND, ... commands. This mimics IMAP behavior, but since we have diverged from IMAP quite a lot by now, we could just change syntax of the commands to include a "COLLECTION" parameter. This could speed up FETCH operation a little master?
TODO CollectionStatisticsModel Remove frameworks
TODO CollectionView Remove deprecated methods frameworks
TODO DBusConnectionPool Remove master?
TODO Entity Make it a subclass of AttributeEntity AttributeEntity was introduced to allow sharing code between Tags and Entities. frameworks
TODO Entity Remove AKONADI_DECLARE_PRIVATE frameworks
TODO Entity Create T* Entity::attribute<T>() and const T* Entity::attribute<T>() const overloads Needed to preserve const-correctness. Right now there's T* Entity::attribute<T>() const, which allows modifying the attribute (and hence the collection) even though the pointer was returned from const method frameworks
TODO EntityTreeViewStateSaver Remove frameworks
TODO IndexPolicyAttribute Remove if Baloo Indexer decides to ignore it frameworks
TODO ItemFetchScope Normalize setter/getter names frameworks
TODO ItemModel Remove frameworks
TODO ItemSearchJob Remove deprecated methods frameworks
TODO ItemSync Make private? Does not seem to be used anywhere frameworks
TODO ItemView Remove deprecated methods frameworks
TODO KDSignalBlocker Remove? frameworks
TODO metatypes.h Remove master?
TODO MimeTypeChecker Maybe remove This class is able to compare mime types, handling aliases, inheritance, etc. IIRC Volker once told me this should be available in Qt 5, so check whether it's there. Might be worth upstreaming, too frameworks
TODO PersistentSearchAttribute Remove deprecated methods frameworks
TODO PluginLoader Remove if Qt 5 plugin system is powerful enough frameworks
TODO ProtocolHelper Share with Server See the Other Plans section below frameworks
TODO RecursiveItemFetchJob Optimize Emit itemsReceived() signal from underlaying fetch jobs master
TODO RenameFavoriteDialog Make private? frameworks
TODO ResourceSettings Remove? frameworks
TODO SearchCreateJob Remove deprecated methods frameworks
TODO SelfTestDialog Make a standalone application SelfTestDialog is the only user of QtSQL in client libraries. By making it a stand-alone app, we can get rid of one dependency from libKF5AkonadiCore.so master
TODO StatisticsProxyModel Remove frameworks
TODO SubscriptionJob Split to CollectionSubscribeJob and CollectionUnsubscribeJob? Keep the naming consistent frameworks
TODO TrashFilterProxyModel Remove? Does not seem to be used anywhere frameworks

Features

Features can mostly be implemented already now in master, but they are something I'd totally love to see in Frameworks.

Status Action Comment
TO DO Use ImapStreamParser in client jobs Will improve performance and memory consumption on client side. Also it's easier to use :) <{{{3}}}>
TO DO (IF POSSIBLE) Share external payload parts helper Concentrate external parts support into one place for easier maintanence and less code duplication <{{{3}}}>
TO DO Direct streaming of external payload parts For parts bigger than 4KB, the server can instead of asking client to provide the data send back a file descriptor that client will use to write the data directly to the file. Server can either verify that sizes match, or rely on client library telling the truth. Memory--, performance++ <{{{3}}}>
TO DO Require explicit call to ItemSync::deliveryDone() Clients should explicitly call ItemSync::deliveryDone() after the sync is over, instead of ItemSync being "clever" and detecting it on it's own. See https://git.reviewboard.kde.org/r/117673/ <{{{3}}}>

Server

Status Action Comment
TO DO Fix coding style We need to follow the Frameworks coding style <{{{3}}}>
TO DO Upstream QSQLITE3 driver I managed to figure out what the actual changes we have in our fork are. They are not implemented optimally, a proper implementation will be more complicated, but generally should ensure better performance. Then we can try to upstream those changes and get rid of the driver for good. <{{{3}}}>
TO DO Remove Nepomuk support We have Baloo now, we don't need Nepomuk anymore <{{{3}}}>
TO DO Remove SELECT command See comment at CollectionSelectJob in the table above <{{{3}}}>
TO DO Remove X-AKLIST and X-AKLSUB commands They are aliases to LIST and LSUB commands from the old days <{{{3}}}>
TO DO Remove EXPUNGE command Not used. We now delete items right away. <{{{3}}}>
TO DO Remove backward-compatibility with older protocol versions Some of the handlers still have several paths for supporting older client libraries. We don't even know whether they still work, so just get rid of them. <{{{3}}}>
TO DO Merge NotificationMessages Merge NotificationMessageV2 and V3 to NotificationMessage <{{{3}}}>
TO DO Remove legacy client support from NotificationManager Don't use the broadcast notify() signal anymore, only support notification subscribers. <{{{3}}}>
TO DO Refactor DB access Try to do as much DB interaction as possible in DataStore, so that it can be easily replaced by a FakeDataStore for testing purposes (see "Testable Handlers" below). <{{{3}}}>
TO DO Document ASAP Properly document all ASAP commands and responses. Ideally should be part of doxygen documentation rather than a plaintext file. <{{{3}}}>
TO DO Remove exception.h Share Akonadi::Exception from client libs. <{{{3}}}>
DONE Optimize CacheCleaner (1.12.0) Right now the CacheCleaner requests all collections every minute and checks their cache policies. Instead we should use some form of scheduling queue.
TO DO Remove LSUB and all subscription related API Subscribtions are obsoleted by Christian's work on LIST filtering <{{{3}}}>
TO DO Remove agentserver It seems that Agent Server was only used on Maemo 5, otherwise it's disabled at build time. Since we don't really support this anymore, let's kill it. <{{{3}}}>
TO DO Fix StorageDebugger DBus iface The sequence number (first argument) in queryExecuted() signal should be qint64, not double. <{{{3}}}>
TO DO Cache and share Collection statistics To reduce the amount of queries we do whenever collection stats change, we should cache those values and only invalidate the cache when they change. <{{{3}}}>

Other Plans

Testable Handlers

Does not really depend on Frameworks, can be done at any time. I want to have it done in Frameworks though. Currently we have no way to test Handlers on the server. This would require having a fake DataStore, fake NotificationCollector, fake AkonadiConnection and somehow fake backend for Entities. However, as a result we should be able to send various commands to the handlers and observe whether correct data are stored in database and correct signals are emitted. Big problem with external payload parts, would require a fake ExternalPartManager too.

Other option is to start a server instance, send it commands and listen to notifications and check what's in the database, but that's slow, requires lot of code and most importantly it's very difficult to debug. The approach outlined above would be much better.

Shared protocol helper

In a perfect world, we would have Handlers in a shared library and the Handler would be able to parse the command into entities and vice versa. But since the server is using different entities than the client libraries (Akonadi::Server::Collection vs. Akonadi::Collection), this probably won't be possible. However we still should try to share code for handling ImapSets, parsing attributes, payload parts, etc. The first step toward this is to have client jobs use ImapStreamParser.