Second Akonadi Sprint Re-factors Communication

The second Akonadi sprint this year took place in Berlin, Germany on October 16-18. With a confrontation between the local punks and the law simmering on the streets below, the KDE PIM team and others worked to port applications, add polish to the libraries, and complete PIM software that will raise sparks on the desktop without burning out your CPU.

Status and development

As of KDE 4.3 the Akonadi server is mostly feature complete, allowing the scope of development to expand. The growing KDE PIM team is taking a staggered approach to porting the KDE PIM applications, including KMail, KOrganizer, KAddressBook and of course Kontact, to ensure that each ported application is as robust and stable when included in KDE releases as the legacy versions. The integrity of users' PIM data is of the highest importance. For KDE4.4, KAddressBook and KPilot will become native Akonadi applications. At the same time, KMail and KOrganizer and the other applications are being ported, but will not be released until KDE 4.5, enjoying a long QA soak during the KDE 4.4 cycle.

The KDAB people in Berlin were the PIMsters' hosts again, emphasizing the close links that KDE PIM enjoys with the professional KDE development community. Looking around the two rooms filled with groups of busy hackers bent over laptops, netbooks or explaining APIs by the projector, it is easy to see how this long-established part of KDE development works purposefully to complete an essential part of an integrated PIM system for the Free Desktop.

Akonadi

For anyone new to the KDE 4 architecture, Akonadi improves on what went before by replacing plugins and dedicated code used by each and every application to access mails, contacts and calendars, with a session-wide data cache accessed via a client library. This saves a lot of memory and disk access. A data server approach like this is not new, but previous approaches have been tied to a small set of data types, contacts and calendar events. Akonadi handles the present and will accommodate the future because it is designed to be independent of the types of data it handles. This means it won't have to be intensively re-engineered as new PIM technologies such as Twitter appear. Importantly, Akonadi does not banish users' data to a binary database black hole, but stores them in standard data formats such as Maildir, iCal and vCard or on mail and groupware servers.

The results

Developers at work!

The Dot spoke to the KDE PIM developers at the meeting to see what they were working on.

Volker Krause, the Akonadi team lead, was completely busy helping out others, explaining how things work and conducting a wide-ranging API review to make sure all Akonadi client authors will enjoy the same easy APIs that they are used to from coding KDE and Qt applications.

Thomas McGuire has taken up POP3 mail support in Akonadi, re-factoring the code inherited from KDE 3 and before, and making it maintainable with automated test suites. Having POP3 as just another mail resource in Akonadi allows a mail client like Mailody to get POP3 support for free, where it previously only supported IMAP. Besides POP3 filtering on the server, a general local filter agent contributed by Szymon Stefanek during Google Summer of Code will take care of local filtering of IMAP and OpenChange mails. As the overall KMail architect, we tried to get Thomas to say whether there is a chance that, since an Akonadi KMail will exist around the time of KDE 4.4 without being part of the release, a technical preview might be made available for brave users. However he declined to comment on this.

Tobias Koenig had a lot to do since his application, KAddressBook, will be in the first wave of native Akonadi applications in KDE 4.4. He was spending the meeting working on regressions versus the old implementation, which currently include features such as undo/redo, the Jump Button Bar, and the ability to merge contacts. He is aware of some risks to KDE PIM from adopting Akonadi, since the applications become part of a more complex stack that may be harder to deploy, but is confident that the business stakes held in the project will ensure that issues are discovered and mitigated.

Stephen Kelly is a hacker with fingers in many pies. As the designer of the Entity Tree Model used by several PIM applications to display lists of varying types of items, his advice was constantly in demand at the meeting. He was also working on his Grantlee text templating system, which will be used in KMail in KDE 4.5, and polishing the native Python Akonadi client libraries and the PyGTK applications that he has produced to demonstrate that Akonadi is not just a toy for KDE developers. When asked what he saw coming up in KDE PIM's future, he told the Dot that he was excited by the prospect of javascript-powered Web 2.0 clients using a local Akonadi instance for a powerful offline mode, as well as a new HTML composer for KMail and KJots that would allow multimedia in mails and notes to be properly embedded.

Sebastian Trueg, the face of the Semantic Desktop in the KDE community, came to the meeting to work on the connections between Akonadi's PIM data and the semantic data store in Nepomuk. In a buzzword cocktail, Akonadi apps will use the search services provided by Nepomuk, which indexes Akonadi PIM items as they are added. Since PIM data contains much of users' most important information, it is useful to have good support for accessing this in Nepomuk.

Sebastian told us that the Openlink Virtuoso backend for Nepomuk is important in this regard, because it combines semantic RDF storage with a full-text search index, which were previously provided by separate components. This was decreasing performance, a major concern when there are large mail archives to index. We also asked Sebastian where the dividing line lies between semantic ontologies and existing PIM data formats. On the one hand, vCard, iCal etc are established standards, but now ontologies such as PIMO provide new ways to represent PIM data and Nepomuk can store those. The Dot was curious whether a conflict for PIM data storage is looming. Sebastian answered that a PIM service like Akonadi gives the performance that applications need for getting at lots of data quickly, whereas a general semantic store provides search and cross-referencing between items of varying types. These two are not mutually exclusive and thanks to Nepomuk third party applications won't have to rely on Akonadi or their own iCal or vCard support to be able to take advantage of PIM data.

Akonadi working in KMail

Andras Mantia has been working on the Kolab Proxy Resource. Kolab, a Free Software groupware server, has been well supported for a long time in KDE, and a native Akonadi client implementation for Kolab will benefit from the performance improvements in the IMAP resource. Akonadi-based Kolab clients will not require any changes on the server so admins will be free to deploy a mix of KDE 3, pre-Akonadi KDE 4 and Akonadi-based clients if they wish. During the meeting Andras was porting KMail to Akonadi and from the outdated mimelib to KMime. This re-factoring of existing PIM application components, such as the mail composer, into separate libraries will make writing Akonadi PIM applications be a matter of using these high-level building blocks. The clearly-defined interfaces between components in the upcoming KDE PIM could allow things such as the mail composer to be replaced with specialized versions with enhanced functionality or simplified user interfaces.

Tom Albers is already benefiting from Akonadi with the Mailody email client he leads. He attended the meeting to keep up-to-date with progress, and to work on email expunge support. Tom explained that Mailody has always handled deleting email by first marking mails as deleted, then later expunging these mails from the server, as traditional UNIX mail clients such as Pine do. KMail, on the other hand, deletes mails immediately on the server as Windows users are accustomed to. In addition he was working with Kevin Ottens to fix a bug in the new IMAP resource when used together with a Courier IMAP server. So far, using Akonadi has reduced development time, since so much is done for him by its advanced libraries, such as user identities, mail account setup, and message lists. Tom feels that Mailody will keep its own mail viewer though, which uses separate views to render mail headers and the mail body, preventing phishing attacks from deceiving users by altering headers. He sees Mailody as a mail client for the novice computer user, and could see a role for it as a mail client for netbooks.

Brad Hards has been working on the Exchange support both at the Akonadi resource layer and on the OpenChange library it is built upon. The Exchange protocol is now freely documented, allowing progress without time-consuming reverse engineering, but the sheer scope of the system and its documentation still present a challenge. Contacts, calendaring and mail support are all starting to mature, including advanced features such as filtering and recurrence, for which the Exchange rules have had to be transformed into the open standards that Free PIM software expects. Other features such as RPC over HTTP (aka "Outlook Anywhere"), which uses HTTP as a transport for the Exchange protocol, are also being worked on. For large organizations that use Exchange, Free Software solutions need to support the same server configurations as proprietary clients, without requiring changes to corporate firewalls. Brad traveled to Berlin from Australia to attend the meeting, and said one of his main goals for attending was to meet the European PIM developers face to face and in the same time zone.

Frank Osterfeld had a shorter journey to the meeting as a Berlin resident. He was working on Akregator tasks, including structured search of articles and their full text. He also saw some opportunities to improve the performance of the item lists. Akregator, which will be released as an Akonadi application with KDE 4.5, is now using the Entity Tree Model developed by Stephen Kelly, allowing a lot of custom code to be removed.

Bertjan Broeksema turned his attention towards the mbox file resource. He said that file locking performance is critical for this resource, which combines many mails in one file. By using lock compression, many accesses to the mailbox could be carried out together, reducing disk I/O. For users this simply means less waiting for applications to retrieve data.

The new KAddressbook

Kevin Ottens worked on a standard action manager. Actions are used in KDE to control menu items, toolbar buttons and other controls. Each KDE Akonadi UI component needs to export these actions to the application using it so that they can be included in its menus and toolbars, and since there will be new Akonadi item types in future, it is important to design this for extensibility. Kevin was also looking forward to re-factoring Zanshin, his semi-secret Akonadi To-do tool, to use the Entity Tree Model and support storing tasks in multiple collections. Besides that, he found time to contribute to the API review sessions and validate some IMAP bug-fixes for various ISPs' servers.

Sascha Peilicke was attending his first Akonadi sprint after taking part in GSoC 2009. Besides getting to know the rest of the team, Sascha was working on SyncML support for Akonadi, which had been his GSoC project. He explained that everyone thinks of syncing with mobile phones and other devices when SyncML is mentioned. Of course this is an important goal for him, but he went on to show how it will also be useful for keeping Akonadi instances on different machines in sync, for instance when using disconnected IMAP without first doing a full sync with the IMAP server. He was also working to remove the Akonadi System Settings module, which should not be user visible, by merging it into the akonadiConsole admin tool.

Dmitry Ivanov joined the PIM team after taking part in GSoC with KDE two years ago. His responsibility at the meeting was RSS support and the Nepomuk 'feeder' responsible for passing data from Akonadi to Nepomuk for indexing. The Akonadi Filter agent he is working on will also allow powerful rules so that items from certain RSS feeds may be automatically marked as 'important', for example.

Martin Koller maintains the current implementation of KMail. He was taking the chance to get to know the PIM team and Akonadi, experimenting with virtual folders for searches that are both fast and reliable, two qualities that are absent from the existing virtual folder implementation.

Conclusion

As you can see, the KDE PIM team is active on many fronts, with virtually the entire set of apps being re-factored and redesigned over the course of the next two KDE releases. We look forward to catching up with the team again at their next meeting in January 2010 to see how the final KDE PIM suite will look in KDE 4.4.