Interview With KDE-PIM Hacker Till Adam

Till Adam only started hacking on the KDE mail client, KMail, because he wanted some features implemented from the command-line client Mutt. Now he is one of the main developers of the KDE-PIM project, which KMail is part of. KDE Dot News caught up with Till to talk about e-mail protocols, groupware and implementing them in KDE.

Coming from Laboe, Germany, he is due to visit the NL-PIM meeting scheduled for May 27-29 in Annahoeve. During this meeting, sponsored by NLnet, Trolltech, NLUUG and many more sponsors, he will work with his fellow developers on improving KDE's PIM applications and framework. During his stay in the Netherlands for NL-PIM he has found the time to do a presentation at the NLUUG Spring Conference on using IMAP servers as groupware storage and, together with Kopete developer Will Stephenson, he will give a lecture at HollandOpen.

There is also a Dutch version of this interview.

You've done a lot of work on the IMAP protocol in KMail. What are the main differences between this and POP3 which most people still use to get their email?

Till Adam

POP3 is a protocol for fetching mails from a server to a client. It's then normally stored on the client and removed from the server. Like a real-life mailbox, the server holds the mail until the user fetches it. Whereas with IMAP the mail is stored on the server and clients only retrieve the mails they need when opening them. The client only stores caches not the data itself, it queries the server for everything.

KMail also has support for disconnected IMAP. What's the difference in operation between IMAP and disconnected IMAP?

With disconnected IMAP you have a full copy of the mail stored on the client. You're able to carry it around with you and access it without a network connection to the server.

Could you say that disconnected IMAP is using IMAP as if it were a POP3 server?

No, it's very different. Disconnected IMAP synchronises what is on the server with the client, but the stuff stays on the server. So now both the client and server have identical copies of the data. Then you can change stuff locally without a connection to the server such as deleting mails or moving mails between folders. When you are connected to the server KMail can synchronise again and all your changes are uploaded to the server. All changes on the server that happened in the mean time, such as new mail arriving, are synchronised with the client. After the synchronisation, both client and server are identical again. This is unlike POP3, where you get your mail once, and then delete it from the server.

How did you get interested in the IMAP protocol?

I had been using it for a while since I'm running my own mail/web server at a colo facility with a friend. I wanted to be able to access my mail from several clients and machines - IMAP nicely provides that, and it can even do it in a secure manner with SSL/TLS support. Support for IMAP was a fairly recent addition to KMail at that point and it was not really mature yet. Carsten Burghardt (who still maintains kio_imap4 and works on KMail as well) had done a great job of getting it into shape already but there was still much to be done. KMail was not in a very good shape overall at the time. In particular, it had stability issues. I'm not claiming I cleaned that up all by myself, but I do think I brought some new energy into the team and KMail has nicely improved in the last few years. The attitude towards it, even inside KDE, changed quite a bit. It went from "our crashy mail app" to "KMail, it works well and reliably".

How important is IMAP for an enterprise mail application - and especially for a complete groupware solution?

IMAP is crucial in the enterprise context. IMAP is what is used on corporate desktops. POP is mostly only used for road warriors, if that. It makes sense for enterprises to have the data on the server. It can be centrally backed up, maintained, filtered, etc. So decent IMAP support and performance is rather important.

What difficulties did you have in implementing KMail's IMAP support?

IMAP is not terribly easy to implement because it's a very powerful protocol. In KDE we have this idea of encapsulating protocols in KIO slaves [this makes it possible to access different protocols in a generic way from many different applications - ed]. This works very well for protocols like HTTP where all transactions can be described as "request something by name, wait, get it, do something with it". It's all nicely serialised and maps nicely to file operations: things like open, close, read, write and delete. But the IMAP model is different. You basically just bombard the server with requests, in no particular order. And then you get a bunch of responses back, as quickly as the server can give them to you, again in no particular order. This can be very fast. But to use IMAP as a KIO Slave we had to serialise all operations. So it's: fire a request, wait for the response, do something with it, fire the next request, wait for the response, etc. That makes our IMAP implementation inherently limited, speed wise. Getting it to perform nicely was therefore somewhat of a challenge.

It seems to be very fast in KMail's most recent release, 1.8 included with KDE 3.4.

Well, we (George Staikos, myself and also Carsten) spent quite a bit of time profiling and optimising it leading up to 1.8. Disconnected IMAP is especially a lot faster, but online IMAP is improved as well. I think there's always room for more improvements.

Featurewise, KMail is doing quite alright these days. We have some nice features, such as server side searches, annotation support, ACL support, etc. Some of that came from the Kolab and Aegypten projects.

What do you think of the rise of so-called 'rich' web clients, Google's Gmail comes to mind? Do you think they can ever replace traditional mail clients? Do you see them as competition for KMail?

I think it's partly a different use case. For certain workflows, those interfaces are just fine, even superior to something like KMail in some ways. Much cleaner, less clutter, fewer options, and a nice interface. But Gmail has just a very small fraction of the feature set of KMail, Evolution or other full email clients. So I think there will always be users who just need or want that power and flexibility. Of course they are competition. I personally think we can learn a lot from the web interfaces. For non-powerusers many of our interfaces are way too complicated. We hope to change that, though, with the help of the OpenUsability people, for example at the NL-PIM meeting. Ellen and Jan from OpenUsability will be there to work on that with us. So I'd say Gmail is a challenge, in some ways, but we are not afraid of dying out just yet.

Do you think that a more slick user interface is important for enterprise usage as well? Would you consider having separate 'modes' in KMail for simple and advanced usage?

Yes, enterprise users have different needs, but we could do better interface wise there as well. As for having a 'guru mode', as I like to call it, I don't really know. It would feel like giving up to me to have two user interfaces. Some days I think it's the way to go, but I still have that lingering hope that we can actually make interfaces that are straight forward for casual users but still powerful and discoverable so that people can grow with it and explore things when they need more features. Let's see if the closer cooperation with some excellent usability experts helps us achieve that. It's great to have them involved now.

Has all the activity around Kontact, KDE's PIM solution which includes KMail, help to improve KMail?

Of course. The individual parts of Kontact benefit from each other, and it also brought us some new users: people who want an integrated solution rather than standalone applications for email, calendaring, address book, etc. There are many ways in which the individual applications interact nowadays. Things like KMail using KOrganizer functionality for rendering invitations inside emails.

It's incredible that all these applications work so well together - Kontact really looks like one application. Were there any difficulties in getting the applications that make up Kontact to get along?

Well, there were technical challenges, sure. But since the KDE-PIM developers have a very good team spirit, and get along great, it was just a matter of sitting down and agreeing on how to do things. I think most developers involved with KMail or KOrganizer and the other applications consider themselves KDE-PIM developers these days. The applications are just focus areas if you like.

Has KDE's famously rich development environment, with its existing libraries, infrastructure, documentation etc., helped you in the integration of KMail into Kontact?

Oh yes, majorly. Without our powerfull infrastructure it would not be possible to get the level of functionality and integration with the very limited resources we have, in terms of manpower. It's sometimes astonishingly easy to get things done with the tools we have available. Of course we've also hit limits of the current technologies, but that's good, that means we can improve them. This in turn benefits all of KDE. We are proud of the fact that we've managed to maintain the ability to run all applications standalone without losing any of the integration. You can have KAddressbook and KMail running side by side and they will still interact just as well.

The well-documented code might also help companies to make local customisations to Kontact.

We try to provide extension possibilities, so third parties can do their own Kontact parts, for example. If you want to plug your third party application into Kontact and have it interact with the other parts, you can do that. Or if you want to display calendar info from a custom database or an application exporting its stuff via SOAP, you can do that as well, by writing a calendar resource.

Do you feel that KMail or Kontact is best combined with Kolab as a sister project to KDE?

It's the server project that has worked with the KDE folks the closest and it's probably the one which works best with Kontact. This is because Kolab as a project has people with KDE ties involved. Of the three companies behind Kolab, Intevation, Erfrakon and KDAB, two are run by KDE people, and KDAB employs several KDE and KDE-PIM developers. So naturally Kolab is very nicely integrated.

I also feel that it's important that the newer versions of Kolab allow users of different platforms (Outlook on Windows) to save their data in the same format as KMail & friends, making a slow transition to KDE a lot easier.

Yes, that's an important aspect of the Kolab concept. If we hope to offer people ways out of the Outlook trap a server solution will have to be part of it, and one which allows people to transition step by step. We can of course never catch up with Outlook when it comes to Exchange support.

You mentioned that having some commercial support has helped KMail. Do you think it would be useful for companies to invest in more fulltime KDE hackers?

Of course having people like David Faure hacking fulltime on KDE-PIM for a while is a great thing. The libraries and applications have benefited greatly from that. I do feel that companies would do well to sponsor development of the things they need. My motivation to add enterprise features I don't personally need in my spare time is rather low, which is probably understandable. But if I can do it during paid time, I'll be more than happy to; it's a code base I like to work with, after all. And I do enjoy seeing KMail improve, of course. I've also spent quite a bit of time on KMail and KDE-PIM by virtue of working on Kolab support for KDAB. KDAB will of course gladly implement whatever people need (shameless plug: at reasonable rates, e-mail info@kdab).

Q: What do you feel are the advantages of using a KMail/Kontact/Kolab setup above a MS Outlook/Exchange setup?

Freedom, above all. There are technical benefits as well, of course, but they are maybe less important. While Exchange is more feature-rich in some areas, for very many setups and needs, Kolab is more than sufficient. A lot of development has gone into Exchange of course, and they do get some things right. But for the vast majority of users many of the features of Exchange are useless. Kolab will be a better fit there, I'm sure, and at a price that can't be beat. And that's not even talking about the vastly smaller hardware and infrastructure needs.

In what areas would you like to see improvements to the KDE family?

Exchange's web interface is pretty good, I think. Horde is a web interface that can be used with Kolab and is getting there, but it's not on the same level yet. Of course, Kolab is vastly more platform independent - Exchange's web interface only works fully in Internet Explorer. In general, I think we still have to improve when it comes to user interface polish. Outlook has many little convenient details.

Sounds like there's still something to do at the NL-PIM meeting. We wish Till and all the other people that will visit NL-PIM good luck and a good time in the Netherlands!