KDE PIM/Meetings/Osnabrueck 5

From KDE Community Wiki

The annual KDE PIM Meeting Osnabrück 5 took place at the Intevation offices in Osnabrück from Friday, January 12th 2007, to Monday, January 15th.


Agenda

Friday

9:00 Arrival

17:00 Kickoff Meeting: Status KDE PIM and Akonadi, Perspectives for KDE 4 and beyond

18:15 Status of Akonadi

20:30 Dinner


Saturday

9:00 Hacking

10:00 Community Relations, Website

14:00 Kolab Enterprise Branch

17:00 Presentation: Akonadi

17:45 Presentation: Mailody

18:15 Presentation: Proko

18:30 KDE4 Status Discussion

20:00 Dinner


Sunday

9:00 Hacking

10:30 Maintainance

11:15 Introduction to Qt Model/View Programming

13:00 Lunch

15:00 Further Introduction to Qt Model/View Programming

19:00 Dinner


Monday

10:00 Round-up meeting: Results and conclusions

11:00 Departure


Meeting Notes

Akonadi Status Before the Meeting

Server

More or less implemented features:

  • storage of items consisting of identifiers, mimetype, parent collection and an arbitrary (binary) data block
  • IMAP-like data access interface supporting retrieval and modifications of both collections and PIM items
  • D-Bus notification signals for changes to items or collections
  • auto-generated low-level database access code
  • uses transactions for all database write operations


Problems and missing features:

  • cache mangement, expiring, filtering
  • sqlite backend can't handle massive concurrent access, MySQL/Embedded looks like a interessting options but doesn't work either (might be a QSQL driver bug)
  • communication with resources and search providers only works because of a huge hack which makes D-Bus calls from threads possible


Client Library

More or less implemented features:

  • classes to represent collections and pim items
  • job classes to retrieve and modify collection and pim items
  • a self-updating collection tree model
  • a self-updating flat pim item model
  • a notification filter


Resources

We have a simple read-only iCal file resource. For more complex resource we probably need some more work on the resource API. Resources also have the same problem with async IPC complexity as search providers (see below).


Search Provider

Currently the biggest missing piece, there are still open questions about the design.

Search providers are supposed to provide type specific functionality, needed for the following features:

  • initial creation of virtual folders (searching)
  • updating existing virtual folders (matching a single item against all existing queries)
  • generate type specific FETCH responses (eg. the mail header listing)


There are a couple of things making this complex:

  • some resources (eg. IMAP, LDAP) provide search capabilities itself but not necessarily for all fields/operators
  • some fields/operators are type specific, some are not; some are handled by the storage itself (eg. mimetype, collection), some by generic external tools (eg. fulltext search with strigi), some by the type specific search providers
  • very complex fields (eg. event recurrence)


Since we probably don't want to write our own query evaluation engine, currently the best option seems to be to use Nepomuk for that. It is supposed to store the meta data of everything on the desktop, so putting PIM data in there is needed anyway. Its query language seems to have enough expressive power for our needs, but the performance has still to be evaluated. The Nepomuk developers are very interesseted in cooperating with us here.

There are also technical problems. First experiments at aKademy showed problems with async IPC complexity, the current solution of delayed D-Bus replies and multi-threading needs either to be simplified or completely hidden behind the search provider API.


Development Tools

There are three applications to access the storage backend:

  • the akonadi command line client, but it is still missing several commands (some of them are available in the libakonadi tests)
  • akonadiconsole contains a folder tree and complete log of network and D-Bus communication
  • KMail via IMAP

Of course there is also netcat localhost 4444 and raw IMAP ;-)


General / Project Management

Given the current progress and our ambitious goals it seems unlikely that we will finish Akonadi and port all applications in time for KDE 4.0 (whenever that will be), especially when considering that Akonadi development also draws badly needed resources from KDE PIM development.

Therefore we might want to concentrate on getting only a subset of Akonadi ready and used for 4.0 (eg. just the storage, no searching), keeping KDE PIM features on the 3.5 level where possible and implement additional capabilities for the following 4.x releases.


Akonadi Results

Server

  • mysql/embedded problem no longer reproducable, backup solution with per-application network-less mysqld exists but is not implemented yet
  • check licensing implications of mysql
  • Simon promised to fix D-Bus calls from threads (which is needed to use strigi/nepomuk as well)


Change notifications / update loops

  • solved and implemented for write operations (very important for resources)
  • not implemented for FETCH, solution is complex: extension of the delivery queue, notification blacklists, session id vs. resource id in notification blacklists


Synchronous server operations for "online" resources

  • needed for FETCH/LIST (really??), implemented expect notification loop prevention
  • two options: do it for all other operations as well vs. do all other operations async
  • a complete sync implementation would be too complex (same problems as with FETCH for every command, resource D-Bus API would be the same as the IMAP command API), consider the online case as a special case of a offline resource instead (ie. a cached resource that syncs immediately)


Error and Status Notification

  • identify jobs via a session identifier and a job identifier
  • send status and error notification via D-Bus signals
  • has similar problems as change notifications, depends on sync/async semantics of write operations


PIM Item Data Structure

  • Use a strictly increasing revision number for optimistic locking
  • Split data into a tree-like structure where appropriate (think IMAP BODYSTRUCTURE)
  • Add extra fields for special FETCH responses (think ENVELOPE)
  • No type-specific code in the backend required for partial data retrieval, all item parsing happens in the resource
  • XML or MIME based format for communication between storage and resources/agents
  • Store bodyparts separately in the database, the raw data optionally in the filesystem (requires extra work for transaction safety)


Search

  • Akonadi does not search itself
  • we feed every data we get into strigi/nepomuk, eg. by writing stand-alone feeder apps (formerly known as search providers) or by writing/extending the corresponding strigi/nepomuk plugin
  • Akonadi should support self-updating virtual folders (if necessary storing the result ids), by sending the query to all involved resources and strigi/nepomuk and merging the results
  • updating virtual folders need still to be discussed


Server-Side Collection Handling

  • switch from path with user visible to unique ids for identifying collections
  • do not store complete path but a pointer to parent collection
  • will completely break the IMAP LIST command -> rename and keep the option for a IMAP-compatible LIST command


Model/View

  • our models contain still lots of stuff which Till told us we shouldn't do, ie. use QModelIndex only for the QAbstractItemView model, solve everything else via data()


General

  • KDE dependencies are allowed for now to safe work, but avoid KDE dependencies where possible
  • next Akonadi meeting in Berlin in April


David's Report

It was good to meet the other developers so as to put faces to the names which I'd previously only encountered on the mailing lists. I now feel more part of the team - about time, since I've been involved in KDE PIM for over 5 years. Getting a better perspective on the issues affecting KDE PIM as a whole (particularly Akonadi) was also very useful.

It was extremely useful technically. The biggest benefit was the session on the Qt 4 model-view framework which gave me enough understanding to go ahead and start converting KAlarm to use it. My initial partial implementation actually displayed some data! And it should enable some additional features to be coded very easily compared to the Qt 3 way of doing things. I've seen the light, halleluiah. :-)

Being able to easily ask questions of other people also contributed to finally getting D-Bus to work in KAlarm.

It also emerged in the meeting that in relation to KDE 4 conversion, KAlarm is in a similar state to most of the other KDE PIM applications. Namely that there is a lot to be done, but it isn't necessary to wait for Akonadi before doing it. It makes me feel a lot more positive about getting on with the conversion, which makes that small part of KDE that much more likely to be ready for the KDE 4 release.


Impressions

Photo of Mailody
Photo of Proko version of Kontact
Mailody Proko
Photo of KDE PIM hackers at work
Photo of meeting conclusions on whiteboard
Hacking Conclusions


Press Coverage


Blogs