Frameworks/Akonadi

This is a list of things I want to do in kdepimlibs/akonadi for Frameworks 5.

Akonadi repo structure

Merge client libraries and server into one repository. Main advantage is that we will be able to share more code between server and libraries.

Proposed structure:

/autotests
/doc
/src
/src/agentbase (libKF5AkonadiAgentBase.so - AgentBase, ResourceBase and other classes related to agents. Needs a better name)
/src/agentlauncher (utility for loading and running plugin-based agents)
/src/akonadictl (akonadictl utility)
/src/asapcat (asapcat utility)
/src/common (libKF5AkonadiCommon.so - private classes shared between server, core and widgets libraries and other utilities. Does NOT install any (or most of its) public headers)
/src/control (akonadi_control utility)
/src/core (libKF5AkonadiCore.so - non-UI classes, only Jobs, models, Monitor etc.)
/src/core/jobs 
/src/core/models
/src/interfaces (DBus interfaces)
/src/qsqlite (Qt's QSQLITE driver fork - better try to upstream it)
/src/selftest (based on SelfTestDialog class)
/src/server (akonadiserver)
/src/widgets (libKF5AkonadiWidgets.so - UI classes, depends on core)
/src/widgets/views
/src/xml (akonadi/xml library)
/tests/akonaditest (akonaditest utility)
/tests/fake (fake classes to be used for testing)
/tests/searchplugin (akonadi_testsearch_plugin)
/tests/testresource (akonadi_knut_resource)

Client libraries

Code changes

Status	Action	Comment
IN PROGRESS	Fix coding-style (Guy)	We have to follow the Frameworks coding style
TO DO	Port to Qt 5	Make client libraries compile against Qt 5	<{{{3}}}>
TO DO	Replace BOOST_STATIC_ASSERT	C++11 has static_assert, so use that	<{{{3}}}>

API/ABI changes

Status	Class	Action	Comment	Branch	Assignee
TODO	AgentBase	Merge ObserverV2 into Observer	There should be "Observer", "BatchObserver" and "TagsObserver" (not inheriting from each other). BatchObserver would enable the same behavior as ObserverV3 currently, TagsObserver would enable tags notifications.	frameworks
TODO	AgentSearchInterface	Remove addSearch() and removeSearch() methods. Search is managed on server now, so we only need search() method to get results. add/removeSearch() are no longer used		frameworks
TODO	AgentBase	Merge ObserversV* into Observer	There should be "Observer" and "BatchObserver" (not inheriting from each other). BatchObserver would enable the same behavior as ObserverV3 currently	frameworks
TODO	ChangeMediator	Remove unused methods	ChangeMediator is only used by ModifyJobs to invalidate entries in Monitor's EntityCache, but it has bunch of stub methods we don't need, so remove them	master
TODO	Collection	Remove deprecated methods		frameworks
TODO	CollectionFetchJob	Remove deprecated methods		frameworks
TODO	CollectionFetchScope	Remove deprecated methods		frameworks
TODO	CollectionModel	Remove		frameworks
TODO	CollectionSelectJob	Remove?	Performs SELECT before FETCH, STORE, APPEND, ... commands. This mimics IMAP behavior, but since we have diverged from IMAP quite a lot by now, we could just change syntax of the commands to include a "COLLECTION" parameter. This could speed up FETCH operation a little	master?
TODO	CollectionStatisticsModel	Remove		frameworks
TODO	CollectionView	Remove deprecated methods		frameworks
TODO	DBusConnectionPool	Remove		master?
TODO	Entity	Make it a subclass of AttributeEntity	AttributeEntity was introduced to allow sharing code between Tags and Entities.	frameworks
TODO	Entity	Remove AKONADI_DECLARE_PRIVATE		frameworks
TODO	Entity	Create T* Entity::attribute<T>() and const T* Entity::attribute<T>() const overloads	Needed to preserve const-correctness. Right now there's T* Entity::attribute<T>() const, which allows modifying the attribute (and hence the collection) even though the pointer was returned from const method	frameworks
TODO	EntityTreeViewStateSaver	Remove		frameworks
TODO	IndexPolicyAttribute	Remove if Baloo Indexer decides to ignore it		frameworks
TODO	ItemFetchScope	Normalize setter/getter names		frameworks
TODO	ItemModel	Remove		frameworks
TODO	ItemSearchJob	Remove deprecated methods		frameworks
TODO	ItemSync	Make private?	Does not seem to be used anywhere	frameworks
TODO	ItemView	Remove deprecated methods		frameworks
TODO	KDSignalBlocker	Remove?		frameworks
TODO	metatypes.h	Remove		master?
TODO	MimeTypeChecker	Maybe remove	This class is able to compare mime types, handling aliases, inheritance, etc. IIRC Volker once told me this should be available in Qt 5, so check whether it's there. Might be worth upstreaming, too	frameworks
TODO	PersistentSearchAttribute	Remove deprecated methods		frameworks
TODO	PluginLoader	Remove if Qt 5 plugin system is powerful enough		frameworks
TODO	ProtocolHelper	Share with Server	See the Other Plans section below	frameworks
TODO	RecursiveItemFetchJob	Optimize	Emit itemsReceived() signal from underlaying fetch jobs	master
TODO	RenameFavoriteDialog	Make private?		frameworks
TODO	ResourceSettings	Remove?		frameworks
TODO	SearchCreateJob	Remove deprecated methods		frameworks
TODO	SelfTestDialog	Make a standalone application	SelfTestDialog is the only user of QtSQL in client libraries. By making it a stand-alone app, we can get rid of one dependency from libKF5AkonadiCore.so	master
TODO	StatisticsProxyModel	Remove		frameworks
TODO	SubscriptionJob	Split to CollectionSubscribeJob and CollectionUnsubscribeJob?	Keep the naming consistent	frameworks
TODO	TrashFilterProxyModel	Remove?	Does not seem to be used anywhere	frameworks

Features

Features can mostly be implemented already now in master, but they are something I'd totally love to see in Frameworks.

Status	Action	Comment
TO DO	Use ImapStreamParser in client jobs	Will improve performance and memory consumption on client side. Also it's easier to use :)	<{{{3}}}>
TO DO	(IF POSSIBLE) Share external payload parts helper	Concentrate external parts support into one place for easier maintanence and less code duplication	<{{{3}}}>
TO DO	Direct streaming of external payload parts	For parts bigger than 4KB, the server can instead of asking client to provide the data send back a file descriptor that client will use to write the data directly to the file. Server can either verify that sizes match, or rely on client library telling the truth. Memory--, performance++	<{{{3}}}>
TO DO	Require explicit call to ItemSync::deliveryDone()	Clients should explicitly call ItemSync::deliveryDone() after the sync is over, instead of ItemSync being "clever" and detecting it on it's own. See https://git.reviewboard.kde.org/r/117673/	<{{{3}}}>

Server

Status	Action	Comment
TO DO	Fix coding style	We need to follow the Frameworks coding style	<{{{3}}}>
TO DO	Upstream QSQLITE3 driver	I managed to figure out what the actual changes we have in our fork are. They are not implemented optimally, a proper implementation will be more complicated, but generally should ensure better performance. Then we can try to upstream those changes and get rid of the driver for good.	<{{{3}}}>
TO DO	Remove Nepomuk support	We have Baloo now, we don't need Nepomuk anymore	<{{{3}}}>
TO DO	Remove SELECT command	See comment at CollectionSelectJob in the table above	<{{{3}}}>
TO DO	Remove X-AKLIST and X-AKLSUB commands	They are aliases to LIST and LSUB commands from the old days	<{{{3}}}>
TO DO	Remove EXPUNGE command	Not used. We now delete items right away.	<{{{3}}}>
TO DO	Remove backward-compatibility with older protocol versions	Some of the handlers still have several paths for supporting older client libraries. We don't even know whether they still work, so just get rid of them.	<{{{3}}}>
TO DO	Merge NotificationMessages	Merge NotificationMessageV2 and V3 to NotificationMessage	<{{{3}}}>
TO DO	Remove legacy client support from NotificationManager	Don't use the broadcast notify() signal anymore, only support notification subscribers.	<{{{3}}}>
TO DO	Refactor DB access	Try to do as much DB interaction as possible in DataStore, so that it can be easily replaced by a FakeDataStore for testing purposes (see "Testable Handlers" below).	<{{{3}}}>
TO DO	Document ASAP	Properly document all ASAP commands and responses. Ideally should be part of doxygen documentation rather than a plaintext file.	<{{{3}}}>
TO DO	Remove exception.h	Share Akonadi::Exception from client libs.	<{{{3}}}>
DONE	Optimize CacheCleaner (1.12.0)	Right now the CacheCleaner requests all collections every minute and checks their cache policies. Instead we should use some form of scheduling queue.
TO DO	Remove LSUB and all subscription related API	Subscribtions are obsoleted by Christian's work on LIST filtering	<{{{3}}}>
TO DO	Remove agentserver	It seems that Agent Server was only used on Maemo 5, otherwise it's disabled at build time. Since we don't really support this anymore, let's kill it.	<{{{3}}}>
TO DO	Fix StorageDebugger DBus iface	The sequence number (first argument) in queryExecuted() signal should be qint64, not double.	<{{{3}}}>
TO DO	Cache and share Collection statistics	To reduce the amount of queries we do whenever collection stats change, we should cache those values and only invalidate the cache when they change.	<{{{3}}}>

Other Plans

Testable Handlers

Does not really depend on Frameworks, can be done at any time. I want to have it done in Frameworks though. Currently we have no way to test Handlers on the server. This would require having a fake DataStore, fake NotificationCollector, fake AkonadiConnection and somehow fake backend for Entities. However, as a result we should be able to send various commands to the handlers and observe whether correct data are stored in database and correct signals are emitted. Big problem with external payload parts, would require a fake ExternalPartManager too.

Other option is to start a server instance, send it commands and listen to notifications and check what's in the database, but that's slow, requires lot of code and most importantly it's very difficult to debug. The approach outlined above would be much better.

Shared protocol helper

In a perfect world, we would have Handlers in a shared library and the Handler would be able to parse the command into entities and vice versa. But since the server is using different entities than the client libraries (Akonadi::Server::Collection vs. Akonadi::Collection), this probably won't be possible. However we still should try to share code for handling ImapSets, parsing attributes, payload parts, etc. The first step toward this is to have client jobs use ImapStreamParser.