Sprints/PIM/2023

Venue

• Artilect Fablab Toulouse
• 10 rue Tripière – 31000 Toulouse - France
• https://artilect.fr/ (website in French only)

Agenda

Add topics we should discuss:

• KF6-based PIM (1h timebox, discussion):
  • blocking issues?
  • release timeline, branching
  • Remaining issues with replacing the Kross-based account wizard

• KF6-based PIM, retirning components (1h timebox discussion):
  • Retiring the Kolab resource in favor of CalDav/CardDav?
  • Retiring the mixed maildir resource in favor or "real" maildir?

• PIM Libraries/Dependencies (10 minutes overview + breakout groups digging one of the topics)
  • KMailTransportAkonadi folding in akonadi-mime?
  • the last KWallet use in KMailTransport - how can we get rid of that?
  • moving the remaining things needed for Kalendar from eventviews and calendarsupport down to akonadi-calendar
  • can we reverse the akonadi-contact -> contacteditor dependency?

• PIM Frameworks in KF6 (10 minutes overview + breakout groups digging one of the topics)
  • KDAV without KIO HTTP?
  • KCalendarCore custom/non-standard properties API

• Windows compatibility (1h timebox, discussion)
  • GnuPG windows toolchain compatibility

• QML compatibility (10 minutes overview + breakout groups digging one of the topics)
  • QML APIs for PIM libraries
  • upstreaming QML API from Kalendar to KF

Notes

QNAM port of KDAV

• we want this given the state of KIO HTTP, the complexity of KIO HTTP deployment and due to not needing anything KIO HTTP provides over QNAM for the most part
• the good news: KIO is not leaking to the outside in the KDAV API, we can switch the implementation whenever we want
• migration plan
  • complete the test coverage using FakeServer scenarios, at least covering the three DAV verbs used here (PROPFIND, PROPPATCH, REPORT), ideally also the HTTP verbs beyond GET. Note that different paths are taken depending on the protocol (CardDav vs CalDav), so we need to consider both for the tests.
  • A single QNAM instance can then be held by the DavManager singleton, API to provide an external one isn't needed initially (and we probably won't want it ever, this opens a can of worms in term of API safety).

KCalendarCore::Calendar async change operations

Use-case: Zanshin using KCalendarCore::Calendar with the calendar plugins instead of Akonadi-specific models.

Option 1:

• Make modifying methods on Calendar return something job-like, and thus make the API fully async.
• That's the right approach on paper...
• ... but very invasive. Likely not realistic for KF6.

Option 2:

• Add additional API for registering an error handler and possibly more change notification signals.
• Can be phased in small steps and possible even in an API-compatible way.
• Far more realistic in terms of effort.

KCalendarCore::Calendar API review

Areas in need of a closer look:

• pseudo-I/O API: isSaving(), save(), close(), reload(), modified state...
  • base implementations are usually non functional dummys
  • semantics make little sense in an Akonadi or Android world
  • isSaving() looks entirely unused, save() and reload() are also unused - https://invent.kde.org/frameworks/kcalendarcore/-/merge_requests/142 + https://invent.kde.org/frameworks/kcalendarcore/-/merge_requests/143
  • isModified() is in active use in KAlarm
  • close() should move into the dtor - https://invent.kde.org/frameworks/kcalendarcore/-/merge_requests/144

• Notebook API
  • we don't even know what that is, not covered in any RFC and apparently existing entirely in memory
  • unused in our code, but possibly something used by Sailfish (https://github.com/sailfishos/mkcal/blob/master/src/notebook.h ??)
  • see https://invent.kde.org/frameworks/kcalendarcore/-/issues/4 - can be deprecated/removed

• Filtering capability
  • Duplicates parts of the read API (raw vs non-raw)
  • Conceptually something that rather belongs in a proxy-calendar on top
  • Direct calls to filtered read API in any of the existing Calendar sub-classes would make a transition to the proxy approach difficult though -> review shows this is not a problem, the cases where it happens are accidental and should be using unfiltered access anyway.
  • Plan:
    • step 1: implement a proxy calendar in KCalendarCore (doable now)
    • step 2: port PIM users to that (doable after branching)
    • step 3: deduplicate the access API in Calendar (needs to be done before KF6 freezing)

• Relations API
  • seems duplicated in Akonadi::CalendarBase, a possible reason for this is lacking change notifications in that base class?
  • setup/removeRelations should likely be protected? currently only used by MemoryCalendar
  • functionality is needed though for hierarchical todos
  • conclusion:
    • relations should not be in Calendar, as there are different ways to model relations
    • find a way to port the single use of the relation API from Calendar, then remove it from Calendar

PIM 6

• aim for a 6-based release with 23.12, assuming a KF6 release in Q3.
• if there is a delay in KF6, we can shift by one Gear release cycle
• QML-based account wizard is not realistic before branching, there is no other big change we would want to get in before branching
• create the kf6 branch "now"
• larger changes happen in kf6 branch, with master being merged into kf6
• move kf6 CI jobs to the kf6 branch and remove them in master, after coordination with Ben

Feedback from Ben:

but from a CI perspective, you branch, remove the Qt 5 jobs, then amend your .kde-ci.yml to point to the @latest-kf6 target and off you go I think
In terms of branching
I'd suggest doing it using -o ci.skip
then setting up a seed job and running that
otherwise the jobs are pretty much guaranteed to explode in your face

External dependencies on PIM modules on the CI (not all of those might have KF6 builds

kde/kdenetwork/kio-gdrive: kde/pim/libkgapi
kde/kdenetwork/kopete: kde/pim/kleopatra
extragear/office/kmymoney: kde/pim/kidentitymanagement
playground/pim/akonadiclient: kde/pim/kmime
playground/pim/kitinerary-workbench: kde/pim/kitinerary
playground/pim/kjots: kde/pim/akonadi
playground/pim/kjots: kde/pim/akonadi-notes
playground/pim/kjots: kde/pim/kmime
playground/pim/kjots: kde/pim/kontactinterface
playground/pim/kjots: kde/pim/kpimtextedit
playground/pim/vakzination: kde/pim/kitinerary
playground/mobile/raven: kde/pim/akonadi
playground/mobile/raven: kde/pim/akonadi-contacts
playground/mobile/raven: kde/pim/akonadi-mime
playground/mobile/raven: kde/pim/kldap
playground/mobile/raven: kde/pim/kmailtransport
playground/mobile/raven: kde/pim/kmime
playground/mobile/raven: kde/pim/libkdepim
playground/mobile/raven: kde/pim/mailcommon
playground/mobile/raven: kde/pim/messagelib
playground/mobile/raven: kde/pim/pimcommon
kde/kdeutils/kgpg: kde/pim/akonadi-contacts
extragear/utils/ktrip: kde/pim/kpublictransport

After review, none of those seem to be a problem, either they have no KF6 build or can move as well.

Kolab resource phase out

• Looks too easy to be true
• We already have infrastructure in place for migrating existing resources
• Should take inspiration from the GoogleResourceMigrator which did a similar job
• Plan
  • Phase 1: Once the new QML wizard is in place stop creating kolab resources and create IMAP + DAV resources instead
  • Phase 2: Create a migrator for the existing setups creating the IMAP and DAV resources and killing the existing kolab resource
    • Warning: make sure we display to the user a warning that something is migrated and it will take a long time
    • Warning: make sure we display a warning to the user that some settings will be lost (like folder/identity mapping etc.)
  • Phase 3: Simplify the IMAP resource again to not have ImapResourceBase
• Problem: who got a Kolab account to test with? this is the limiting factor

Mixed maildir retirement

• we want get rid of this
• step 1: we need to do an mbox to maildir conversion for this, which needs to be atomic without risk of data loss, transparently done by the resource -> find out how the folder structure looks like exactly to see how this could be done
• step 2: replacing mixed maildir resource by a standard maildir resource. The problem with that step is the loss of folder settings and local filter destinations. Especially the latter is problematic, as local filters are likely in use with local folder structures. No solution for this yet.

How does the hierarchy map to the file system?

   mboxfolder (mbox)
   .mboxfolder.directory
       - subfolder (mbox/maildir)

   maildirfolder (maildir)
       - cur
       - new
       - tmp
   .maildirfolder.directory
       <all dirs except . .. cur new tmp>

-> This is also used for the maildir resource.

How does the remoteId of collections look like?

mbox: file name (without path)

maildir: directory name (without path)

Same for mbox and maildir; identical to name of folder

Difference between mbox and maildir

mbox has mRevision; collection has remoteRevision (counted up when compacting folder)

Migrate mbox collection to maildir collection

• Create .mboxfolder.maildir
• Create a hidden Collection (can two collections have the same remoteId?)
• Copy all not-deleted messages from mbox to maildir
  • do it as low-level as possible copying only the raw data?
    • is there item data we want to preserve, e.g. flags?
  • or do it mbox Collection to maildir Collection?
• Rename mboxfolder to .mboxfolder.mbox (or move the Collection?)
  • keep all subfolders in .mboxfolder.directory untouched
• Rename .mboxfolder.maildir to mboxfolder (or move the Collection?)
  • keep all subfolders in .mboxfolder.directory untouched

Alternative (without temporary Collection)

• Lock the Collection (via setEnabled(false)?)
• Create .mboxfolder.maildir
• Copy all not-deleted messages from mbox to maildir (only raw message data)
• Move mboxfolder to mboxfolder.old
• Move .mboxfolder.maildir to mboxfolder
• Recreate the Collection
• Unlock the Collection

Decision

Use the alternative approach but instead of recreating the Collection invalidate the cache. The Resource should then reload the data from disk. This way the id of the Collection doesn't change, so that settings and filters keep working.

Switch of Resource type

After all mbox Collections have been converted to maildir Collections the migration agent could simply change the type of the Resource from mixedmaildir to maildir.

Windows compatibility

• GnuPG's autotools dependency is the main issue here, Kalendar has a Windows build otherwise
• GnuPG's C++ library can be build with CMake, but upstream doesn't accept that, the C stuff is buildable with mingw and/or usable with the existing bindings
• Upstream might accept a CMake buildsystem just for the C++/Qt bindings, maintained by us.
• Craft would then build those bindings only, using prebuilt C libs, as we cannot mix mingw and msvc builds in one tree, but that's something supported already by Craft.

Account Wizard

• Fixed a few warnings and usability in the qml port
• We will drop the list of wizards and prefer focusing on the enter credentials -> find configuration on the internet (or try to guess) -> crypto setup

How to revert the akonadi-contacts -> contact-editor dependency inside of the akonadi-contacts repository

• Move OpenEmailAddressJob and AddEmailAddressJob to contact-editor
• Move StandardContactActionManager back to KAddressBook
• Move plugins to contact-editor
• Move ContactEditorDialog to contact-editor

But also it raises the question of other widget uses in akonadi-contacts so maybe later on...

• Decouple AddContactJob and AddEmailDisplayJob from KMessageBox probably going through an extended ui delegate (like what we do for KIO jobs)
• More generally rename contact-editor into "akonadi-contacts-widgets" and also move to it
  • ContactViewer, GrantleeContactViewer and ContactMetaDataAkonadi
  • AbstractEmailAddressSelectionDialog, EmailAddressSelectionDialog and recipientspicker
  • TextBrowser and WaitOverlay
  • ContactGroupViewer, ContactGroupEditor, ContactGroupEditorDelegate and ContactGroupEditorDialog
  • ContactViewer and ContactViewerDialog
  • EmailAddressRequester, EmailAddressSelectionDialog and EmailAddressSelectionWidget
  • SelectAddressBookDialog

QtKeychain use in KMailTransport

Removing the remaining KWallet::isEnabled call:

• option 1: do a silent fallback to insecure storage in QtKeychain unconditionally if storePassword() is enabled
• option 2: do not support non-keychain password storage at all.
• option 3: keep current behavior, at the cost of significant extra complexity, doing a two-phase QtKeychain job use and checking for NoBackendAvailable/NotImplemented errors
• option 4: implement the counter-part to KWallet::isEnabled() in QtKeychain, requiring only minimal changes to our existing code. Implementation of that is only complicated in the Unix backend, the other platforms have unconditional support.

Going with Option 4: https://github.com/frankosterfeld/qtkeychain/pull/232

KMailTransportAkonadi removal

• classes move to akonadi-mime, move is done as copy/port/delete, can be done before branching if time permits
• the Akonadi KMailTransport plugin moves to kdepim-runtime

Kalendar exposure to CalendarSupport::KCalPrefs

• Remove uses of defaultCalendarId and use Kalendar local config for the feature (no real added value to be in sync with KOrganizer there)
• Remove uses of resourceColorKnown (same thing, not much added value and makes the user wrongly believe it's a two way sync with KOrganizer while it's not, making them think it's broken)
  • Also probably makes sense to store the color information only as an attribute, reduces opportunities for bugs
• TodoModel needs to be reworked a bit
  • Drop the tooltips so there's no need to call enableToolTips anymore (already done)
  • The background color feature should move in a proxy model used only in KOrganizer, this way no need to call todoOverdueColor and todoDueTodayColor anymore
  • Then move TodoModel to akonadi-calendar

New KCalendarCore custom property API

Problem with the current solution:

• No access to property parameters
• Forces X-KDE- prefix or use of awkward "nonKDECustomProperty" API
• No support for property types other than strings

Suggested solution is returning a new Property type instead of a QString, which has access to the content as well as the parameters. There is a backward compatibility challenge with this though:

• a new property only accessible by name becoming available as proper API will then suddenly no longer be accessible by name
• an unsupported property or property parameter type will be reported as string, but change to the real type when support is added

Both scenarios break existing user code.

Conclusion: problem already exists now, we have no good way to avoid that, so accept this can happen.

Sprint Schedule

• April 1/2, 2023
• It's possible to show up to the venue on the Friday afternoon, I will be there to greet the early birds

Travel Reimbursement

• File a request on https://reimbursements.kde.org/events/160
• The proposed hotel is around 130€ for two nights.

Attendance

• Kevin Ottens
• Ingo Klöcker
• Volker Krause
• Laurent Montel
• Carl Schwan

Hotel

• Plenty of options in the area on all price ranges, living there I didn't get to try them though...
• Proposed hotel is "Hôtel de France", seems to be one of those smaller and a bit dated but clean options
• Pricing seems a bit cheaper directly on their website: https://hotel-france-toulouse.com/ (also available through booking.com for those who prefer that)

Arrival and departure times

• Ingo: arriving 5:54 on Friday (by night train), departing 22:18 on Sunday (by night train)
• Laurent: arriving 16:55 on friday (airport), departing 20:20 on sunday (airport)
• Carl: arriving 7:07 on Saturday (by night train), departing 22:18 on Sunday (by night train)
• Volker: arriving 10:2913:24 on Friday by train, departing 17:19 on Sunday by train

Restaurants

• Friday dinner: Le Maharaja - 46, rue Peyrolières - 31000 Toulouse
• Saturday lunch: La Faim des Haricots (take away)
• Saturday dinner: Jalapeños (delivery)
• Sunday lunch: La Faim des Haricots (take away)

Report

Draft of report:

KDE Itinerary compatible event data

Can be copy/pasted into KDE Itinerary:

{
    "@context": "http://schema.org",
    "@type": "EventReservation",
    "reservationFor": {
        "@type": "Event",
        "name": "KDE PIM Sprint 2023",
        "startDate": "2023-04-01T10:00:00+02:00",
        "endDate": "2023-04-02T18:00:00+02:00",
        "location": {
            "@type": "Place",
            "name": "Artilect Fablab Toulouse",
            "address": {
                "@type": "PostalAddress",
                "addressCountry": "FR",
                "addressLocality": "Toulouse",
                "postalCode": "31000",
                "streetAddress": "10 rue Tripière"
            },
            "geo": {
                "@type": "GeoCoordinates",
                "latitude": 43.6017920,
                "longitude": 1.4428500
            }
        }
    }
}