Sprints/PIM/2024

Venue

• Etincelle Coworking Carmes - Baobab room
• 1 rue Bouquières – 31000 Toulouse - France
• https://staging.etincelle-coworking.com/toulouse/carmes/salle-baobab (website in French only)

Agenda

Add topics we should discuss:

• Revisit the Tag support in Akonadi resources (current situation is fairly broken, tags from payloads are not usable anywhere)
• How can we get rid of the mimetreeparser fork?
• Windows build of gpgme-qt6
• How to finally retire the Kolab resource
• Status of notes (KJots/KNotes)
• Getting KMime ready for a move to Frameworks, identifying more classes to move to Frameworks.
• Akonadi Sqlite locking issues
• Mobile specific issues:
  • Move resources configurations to QML
  • Sliding sync in Akonadi
• Move libkdepim/src/multiplyingline to frameworks
• E2E testing of resources
• Project Goals

Notes

KJots/KNotes and notes in general

• KJots was deprecated a long time ago and removed from releases, but people kept shipping it
• Only actually working backend was the local maildir backend, which could be importable in e.g. Marknote from that, so we have a data recovery pass.
  • This has to be done before we remove anything.
• The same approach will work for KNotes.

• There's secondary notes integration (Add Note/Create Note in KMail for example), not all of that is using Akonadi note item types though and are actually unrelated and use Akonadi attributes or IMAP annotations.
• Following from that we can also remove the akonadi-notes repository.
• This means also the akonadi-notes resource will go away (see issue below for undeletable resources), same problem already exists for the removed Tomboy resource as well.

• Some of the non-notes notes feature might be obsolete/dysfunctional as well.

Kolab resource retirement

• Akonadi isn't prepared for agent binaries disappearing and agents for that still being configured, up to the point it might not even be possible to delete that anymore. Should be fixed before finally removing the resource.
• Possibly needs a list of dead resources in Akonadi to do this safely.
• Automatic migration is considered technically unfeasible due to no way of getting the magic URLs for DAV.
• Needs 1+ year pre-warning time before final removal.

Mime Tree Parser unification

• For porting to the new version that needs to catch up on features that were removed there, such as the plugins
  • We can't really avoid plugins here, as we otherwise can end up with cyclic dependencies (e.g. via akonadi-calendar)
  • The existing plugins need to be ported, and either be built-in or being plugins
  • HTML rendering is not included

• Can we use the new QML/widget renderer instead of the status quo HTML renderer?
  • That means all the HTML security problems go away, and we can do proper incremental updates
  • We'd lose some of the styling features, but those are mostly unused anyway?
  • Header customization could be done via settings, rather than by custom styles.
  • Adblocker etc depend on webengine, so that could continue to work in html parts of the new renderer (which also uses web engines)

• Can we drop TNEF support? That's no longer in use by Exchange either?

Akonadi DB locking issues

• Problem remains with Sqlite, and is rather due to Akonadi's overuse of (long-living) transactions, not specifically a Sqlite problem
• Sqlite's table-level locking makes this worse though.
• With Sqlite we need to manually do transaction replay on failures.
• Item sync can be optimized to need less long-running transaction, see Dan's client-side item sync branch.
• Client-side transactions need to go anyway, as that makes connections state-less and would remove the need for per-connection threads.

Akonadi for mobile mail

• sliding window sync isn't even the biggest issue, partial downloads is the bigger problem
• sliding window sync also needs a fetch more action when scrolling down
• needs expanded cache expiry to also drop items rather than just content
• resources need to be aware of that, otherwise they download everything
  • only needed for IMAP (DAV has this)
• assuming we have sliding sync, how do we represent this on desktop?
• common infrastructure, or IMAP specific setting?
  • we at least need a common "fetch more" API
• we need collection locking to prevent cache cleaning while user is looking at content
• count-based or date-based? count-based is easier, date-based needs mail content knowledge inside Akonadi core
• also related: remote online IMAP search
  • would need a separate search result collection? virtual collections don't work for this
  • all existing search infrastructure assumes local existence, so we need something new here
• expire policy could also consider unread flags
• collection stats wont work as such, if we need it that would require online IMAP query

--> defer this until we have a more advanced mobile client

• turn agents into core applications, for possibly merging those into one back process (blocked on config and message box error messages)
• refactor the resource UI to be out-of-process QML
  • some resources are ported
  • IMAP is tricky due to having a server connection for feature discovery, that stuff needs to move accordingly
  • EWS also needs to ported
  • agents also have that problem
• how does this relate to account wizard?

• removing message box based error handling
  • tricky for the copy/paste case, notifications are not ideal for this case as we lack context there

EWS

• MS wants to kill this, deprecated in the next 3 years (unless they change plans)
• graph API is supposed to be the replacement, we have no support for that yet
• access to test accounts is tricky, g10code has one at least
• need for EWS support reduced now that we have XOAUTH support for MS IMAP
• short term: fix the current issues, but longer term probably not viable anymore

End to end testing of resources

• CI based integration tests against containerized backends
• Focus on Akonadi server <-> resource interaction
• Could use Python scripting or the Akonadi CLI client for driving the tests
• Harder is probably the service and credential management side
• We can cover Dovecot, Cyrus and Nextcloud via containers, possible some more DAV servers.
• Outlook and GMail would need shared credentials and/or individual test accounts. Those also have tricky login flows with automation counter-measures like captures.
• Separate repository or part of kdepim-runtime? Preference for the latter.

Tag support in Akonadi

• Tags conflict with iCal categories, tags were supposed to be extracted from iCal (or vCard) categories.
• Resources need proper support for tag, which currently is not the case.
• Emails have no concept of tags, there we need external tags.
• In some cases this cannot be translated in resources (e.g. Google Calendar doesn't have support for tags/categories), in some cases persistence via custom properties etc is possible.
• Actions:
  • Needs to be implemented in all resources
  • Needs migration run to extract tags from existing events/contacts that have this in their category fields

Akonadi search

• Dan has a branch with client-side indexing that ensures that all items are immediately searchable the moment they exist.
• See "Make Indexing Great Again" Phabricator ticket and branches.
• Unsolved: re-indexing existing content.
• Needs to be revived and completed, we definitely still want all of that.

Attracting more contributors, external communication

• Bi-monthly blog post: quality declined since we don't have a single person editing them anymore
• Try to get back to this, by having people sign up for a time slot for the year ahead, for editing and managing the entire post.
• Focus on the bigger topics/goals rather than producing a complete change log.
• More blogs and toots are welcome, encouraged and helpful.

• Website: outdated screenshots, as those are not the same as used by appstream, also still mentions KNotes
• Merkuro website
• Zanshin still has a website
• akonadi-project.org should redirect to kontact.org, and has an expired certificate

• Akonadi is still leaking into the UI in a few places by name.

• Make tasks more useful by splitting them in more actionable and more fine-grained sub-tasks.
• Where to put this on Gitlab: pim group or teams/pim? Milestones are shared, group issues are just the aggregation of all project issues in that group, which is not what we need/want.
  • another option might be an empty project in the pim group, which could possibly give us the best of both worlds, but needs checking with Ben whether that's possible

• Update minimum KF versions slightly less quickly, so that working against rolling release KF packages and the Flatpak KDE runtime is possible.
  • we assume about 2 weeks, but check how long Arch, Tumbleweed and maybe Neon need to update KF on average
  • using a newer version behind an ifdef is fine (but not applicable to QML), this is only about the minimal

• Check the general KDE Get Involved documentation for how well that fits getting started on PIM
  • by default recommend building against rolling release distro KF6 rather than a full Plasma kdesrc-build setup, as that's quicker, easier and less risky

• We lack documentation on how to have a separate Akonadi development instance, and how to make sure KMail coonect to the right one
• can we simplify this?
  • separate D-Bus session might fix some problems, but introduces a bunch of new ones (e.g. KWallet)
  • containerization? similar
  • Flatpak? inefficient to build once you touch something in a lower layer, but D-Bus isolation work exactly as we'd need it
  • xdg-dbus-proxy (which is what Flatpak uses)? that's Flatpak D-Bus isolation without all the other isolation -> worth exploring that

• Actively recruit more people to attend the sprint
  • external communication in blogs/mastodon
  • community ML
  • explicitly invite potentially interested individuals

Sprint Schedule

• June 15/16, 2024
• It's encouraged to show up to the venue on the Friday 14 June afternoon
• Kevin will be there after lunch on the Friday to greet the early birds, work can be started when enough people show up

Travel Reimbursement

• File a request on https://reimbursements.kde.org/events/190
• The proposed hotel is around 150€ for two nights.

Attendance

• Kevin Ottens
• Carl Schwan
• Dan Vrátil
• Volker Krause

Hotel

• Plenty of options in the area on all price ranges, living there I didn't get to try them though...
• Proposed hotel is "Hôtel Croix Baragnon": https://www.hotelcroixbaragnon.com (also available through booking.com for those who prefer that)
• It's really really close to the venue, probably the closest hotel actually
• Simple room are 75€ per night, twin rooms are 98€ for those who don't mind sharing space
• They still seem to have availability at time of writing

Arrival and departure times

• Carl arriving on the 13 June at 18h (airport)
• Dan arriving to TLS on Friday at 13:55 (LH1096), leaving on Sunday at 17:50 (LH2221)
• Volker arrives by train Friday 10:38 leaves Sunday 18:16.

Restaurants

To be announced

Report

To be written