Osnabrück icaldir proposal
icaldir KDE PIM Team Mark Bucciarelli Request for Comments February 8, 2005 CONTENTS I. PROBLEM II. SOLUTION III. DIRECTORY LAYOUT IV. FILE NAMES V. TIME ZONES VI. OPERATIONS VII. CONTACT I.) PROBLEM This document proposes a maildir-like approach for storing iCalendar data . The motivation is to provide a scalable way for client applications to coordinate concurrent updates to iCalendar data. Currently, client programs that access iCalendar data (for example, KDE's KOrganizer , Apple's iCal , Mozilla's Sunbird , the PHP iCalendar ) read and save iCalendar data in one large file. This works fine if there is one program updating the calendar data but is cumbersome if multiple clients are updating the same calendar data. Since all data is stored in a single file, each client must lock the entire file to update one single event. If one client program wants to add an event for January 2006 and another client is editing an old event in March 2005, one must wait for the other one to finish. If an iCalendar file is changed, all client programs displaying data from that file must rescan the entire file to be sure they have the most current data. When editing a file, a program must either load the entire file into memory and keep it there or reread the entire file to find the right spot to edit. For large calendars, either of these approaches requires an inefficient use of system memory. II.) SOLUTION The solution is inspired by and borrows heavily from the maildir format . As maildir broke the one large mbox file into many small files (one for each email), we break the icalendar file into many small files, one for each icalendar event. Client applications then create a lock that is specific to the file they are editing. Any client that creates a lock must touch the file periodically to update it's attributes; in this way, other clients can tell without question when a lock file has become stale. (See the 1997 USENIX paper on the Earthlink Mail System for a great writeup on this approach .) The icaldir is designed to be tolerant of network transience. If a client application begins to edit a calendar entry then but crashes or loses network connectivity before completing the transaction, - there is no data loss and - the lock file unambiguously goes stale and clients are free to update that object. III.) DIRECTORY LAYOUT .../icaldir | +- vcalendar_header | +- vcalendar_footer | +- /cur | +- /tmp III.A.) The vcalendar_header file This stores any iCalendar header information that comes before the iCalendar objects start; for example, BEGIN:VCALENDAR VERSION:2.0 PRODID:-//KDE::pim//icaldir//EN III.B.) The vcalendar_footer file Contains any iCalendar information that comes after the iCalendar objects end; for example, END:VCALENDAR III.C.) The cur directory This directory holds one file for every iCalendar object. The file names are created according to Section IV.) FILE NAMES. III.D.) The tmp directory A temporary working directory. No persistent data is stored here. IV.) FILE NAMES (YYYYMMDD|R)-host-pid-timeslice.type where YYYYMMDD : date event starts R : event is recurring host : hostname of client creating object pid : process id of client program creating object timeslice : exact time on client machine of object creation type : ical object type; for example, journal TODO: review how maildir names files; ie host/pid/time TODO: hmmm, get ical text strings in file/names.h?? ;) TODO: think about sort order that default ls returns V.) TIME ZONES The iCalendar RFC  mentions a time zone registry but does not formally define one. The icaldir format specifies the Olson tz database  as it's time zone registry. In icaldir, VTIMEZONE objects do not need to be identifed, but all TZID codes must be valid codes from the Olson tz database. Per RFC2445, all TZID must be prefixed with the solidus character (aka forward slash); for example: TZID=/America/New_York TODO: Read through Unicode "Common Locale Data Repository" docs. http://www.unicode.org/cldr/ Someone did a big chunk of work analyzing Olson db for the CLDR project: see http://www.unicode.org/cldr/data/docs/design/formatting/time_zone_localization.html TODO: What about Windows? Apparantly, there are registry keys (http://wiki.osafoundation.org/bin/view/Journal/JeffreyHarris20041119): SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Time Zones SYSTEM\\CurrentControlSet\\Control\\TimeZoneInformation The first one is a listing of all timezones that the operating system knows about and the second one is detail information on what the current timezone is set to. TODO: What about OSX? Double check it comes with zoneinfo file. TODO: read: http://www.chronos-st.org/Discovering the Local Time Zone--Why It's a Hard Problem.html ** This is a good doc! ** TODO: What about using "POSIX time zone rule literal"? ref: http://www.chronos-st.org/Installation.html VI.) OPERATIONS VI.A.) Create a New Calendar Object 1. Generate a file name (follow same rules as maildir). 2. Write file to tmp directory using this file name. Use the file name as object's iCalendar UID property. 3. Move file from tmp to cur directory. VI.B.) Update a Calendar Object 1. Create new file in tmp with same filename as event plus a .lock extension. The file has one line, the refresh interval at which you will touch this file. This operation should fail is a file with this name already exists. 2. If lock file creation is successful, go to step 4. 3. The iCalendar object may be in use by another process. Read the refresh interval in the existing lock file and if the file has not been touched for this many seconds, the lock is stale. Overwrite the lock file and goto step 4. Otherwise, either stop or wait refresh interval seconds and go to step 1. 4. Create a temporary file in tmp with file name using file name rules below. Goto step 5. 5. Write the new version of the event to this tmp file. If this process takes longer than the lock refresh you specified in step 1, touch the file before the interval elapses. If you lose network connectivity for more than the refresh interval, go back to step 1. 6. Move file from tmp to cur. 7. Delete the .lock file. VI.C.) Delete a Calendar Object 1. Delete the object's file from the cur directory. Note that since an update transaction is effectively an atomic delete + insert that it is legal for one client process to reinsert an event after another process has deleted it. TODO: Get lock when deleting? VI.D.) Change Calendar Object StartDate Since the start date is in an objects file name, changing the start date requires a rename. This is a delete + insert. VII.) CONTACT Please send any comments on this document to Mark Bucciarelli, c/o the KDE PIM mailing list <a href="mailto:[email protected]">[email protected]</a>. ----------------------------------------------------------------- SECTION TODO'S - mention possibility of multiple ical version within same icaldir (Ingo). Give example of min/max syntax of VERSION tag. - specify that recurrence expections go in same file. From Rienhold's email: If you change only one instance of a recurring event, the RFC-compliant way is to generate an event with the same UID, but a RECURRENCE-ID that indicates which item of the recurring sequence is replaced by that other event. E.g. a recurring event (daily) has: BEGIN:VEVENT UID:KOrganizer-702267947.838 DTSTART:20051018T093000Z DTEND:20051018T160000Z SUMMARY:recurring sequence RRULE;:FREQ=DAILY;COUNT=11;INTERVAL=1 END:VEVENT If you want to change only tomorrow's event (e.g. move it from 9:30 UTC to 12:00 UTC), that would be BEGIN:VEVENT UID:KOrganizer-702267947.838 RECURRENCE-ID:20051019T093000Z DTSTART:20051019T120000Z DTEND:20051019T183000Z SUMMARY:recurring sequence (one moved event) END:VEVENT -----------------------------------------------------------------  http://www.ietf.org/rfc/rfc2445.txt  http://pim.kde.org/korganizer  http://www.apple.com/macosx/features/ical/  http://www.mozilla.org/projects/calendar/sunbird.html  http://phpicalendar.net/  http://cr.yp.to/proto/maildir.html  http://www.usenix.org/publications/library/proceedings/usits97/christenson.html A Highly Scalable Electronic Mail Service Using Open Systems Nick Christenson, Tim Bosserman, and David Beckemeyer EarthLink Network, Inc.  http://www.twinsun.com/tz/tz-link.htm  ACID is an acronym for Atomic : either all changes associated with a transaction take place, or none do. Consistent: the database is transformed from one valid state to another valid state. Isolated : a transaction's results are not visible to other transactions until the transaction is complete. Durable : once committed, the results of a transaction are permanent and survive future system and media failures.
This page was last edited on 26 February 2011, at 22:11. Content is available under Creative Commons License SA 4.0 unless otherwise noted.