OpenSync: Synching on the Free Desktop

"It's the first time I feel good when thinking about syncing," said Cornelius, one of the OpenSync developers. This interview intends to give you some insight into OpenSync, the upcoming free unified synching solution for the free desktop. Arne interviews the developers from OpenSync and KDE PIM, Cornelius Schumacher, Armin Bauer and Tobias Koenig.

Basics

As you are now getting close to version 1.0 of OpenSync, which is expected to become the new synchronisation framework for KDE and other free desktops, we are quite interested in the merits it can provide for KDE users and for developers, as well as for the Open Source Community as a whole. So there's one key-question before I move deeper into the details of OpenSync:

What does OpenSync accomplish, that was not done before?

Cornelius:
First of all it does its job of synchronizing data like addressbooks and calendars between desktop applications and mobile devices like PDAs and cell phones.

But the new thing about OpenSync is that it isn't tied to a particular device or a specific platform. It provides an extensible and modular framework which is easy to adopt for application developers and people implementing support for syncing with mobile devices.

OpenSync is also independent of the desktop platform. It will be the common syncing backend for at least KDE and GNOME and other projects are likely to join. That means that the free desktop will have one common syncing solution. This is something really new.

How do the end-users profit from using synching solutions which interface with OpenSync as their framework?

Cornelius:
First, the users will be able to actually synchronize all their data. By using one common framework there won't be any "missing links", where one application can sync one set of devices and another application a different one. With OpenSync all applications can sync all devices.

Second, the users will get a consistent and common user interface for syncing across all applications and devices. This will be much simpler to use than the current incoherent collection of syncing programs you need if you have more than the very basic needs.

How does OpenSync help developers with coding?

Cornelius:
It's a very flexible and well-designed framework which makes it quite easy for developers to add support for new devices and new types of data. It's also very easy to add support for OpenSync to applications.

The big achievement of OpenSync is that it hides all the gory details of syncing from the developers who work on applications and device support. That makes it possible for the developers to concentrate on their area of expertise without having to care what's going on behind the scenes.

I have written quite a lot of synchronization code in the past. Trust me, it's much better, if someone just takes care of it for you, and that's what OpenSync does.

Tobias:
Another point to mentioned is the python wrapper for OpenSync, so you are not bound to C or C++, but can develop plugins in a high level scripting language.

Why should producers of portable devices get involved with your team?

Cornelius:
OpenSync will be the one common syncing solution for the free desktop. That means there is a single point of contact for device manufacturers who want to add support for their devices. That's much more feasible than addressing all the different applications and solutions we had before. With OpenSync it hopefully will become interesting for manufacturers to officially support Linux for their devices.

Do you also plan to support applications of OpenSync in proprietary systems like OSX and Windows?

Cornelius:
OpenSync is designed to be cross-platform, so it is able to run on other systems like Windows. How well this works is always a question of people actually using and developing for this system. As far as I know there isn't a real Windows community around OpenSync yet. But the technical foundation is there, so if there is somebody interested in working on a unified syncing solution on Windows, everybody is welcome to join the project.

What does your synchronisation framework do for KDE and for KitchenSync in particular?

Cornelius:
OpenSync replaces the KDE-specific synchronization frameworks we had before. Even in KDE we had several separate syncing implementations and with OpenSync we can get replace them with a common framework. We had a more generic syncing solution in KDE under development. This was quite similar from a design point of view to OpenSync, but it never got to the level of maturity we would have needed, because of lack of resources. As OpenSync fills this gap we are happy to be able to remove our old code and now concentrate on our core business.

Who, How and Why?

What was your personal reason for getting involved with OpenSync?

Cornelius:
I wrote a lot of synchronization code in the past, which mainly came from the time where I was maintaining KOrganizer and working on KAddressBook. But this always was driven by necessity and not passion. I wanted to have all my calendar and contact data in one place, but my main objective was to work on the applications and user interfaces handling the data and not on the underlying code synchronizing the data.

So when the OpenSync project was created I was very interested. At GUADEC in Stuttgart I met with Armin, the maintainer of OpenSync, and we talked about integrating OpenSync with KDE. Everything seemed to fit together quite well, so at Linuxtag the same year we had another meeting with some more KDE people. In the end we agreed to go with OpenSync and a couple of weeks later we met again in Nuernberg for three days of hacking and created the KDE frontend for OpenSync. In retrospect it was a very pleasant and straightforward process to get where we are now.

Armin:
My reason to get involved (or better to start) OpenSync was my involvement with its predecessor Multisync. I am working as a system administrator for a small consulting company and so I saw some problems when trying to find a synchronization solution for Linux.

At that point I joined the Multisync project to implement some plugins that I thought would be nice to have. After some time I became the maintainer of the project. But I was unhappy with some technical aspects of the project, especially the tight coupling between the syncing logic and the GUI, its dependencies on GNOME libraries and its lack of flexibility.

Tobias:
Well, I have been a KDE PIM developer for several years now, so there was no way around getting in touch with synchronization and KitchenSync. Although I liked the idea of KitchenSync, I hated the code and the user interface [...]. So when we discussed to switch to OpenSync and reimplementing the user interface, I volunteered immediately.

Can you tell us a bit about your further plans and ideas?

Cornelius:
The next thing will be the 1.0 release of OpenSync. We will release KitchenSync as the frontend in parallel.

Armin:
There are of course a lot of things on my todo and my wishlist for OpenSync. For the near future the most important step is the 1.0 release, of course, where we still have some missing features in OpenSync as well as in the plugins.

One thing I would really like to see is a Thunderbird plugin for OpenSync. I use Thunderbird personally and would really like to keep my contacts up to date with my cellular, but I was not yet able to find the time to implement it.

Tobias:
One thing that would really rock in future versions of OpenSync is an automatic hardware detection mechanism, so when you plugin your Palm or switch on your bluetooth device, OpenSync will create a synchronization group automatically and ask the user to start syncing. To bring OpenSync to the level of The Syncing Solution [tm] we must reduce the necessary configuration to a minimum.

What was the most dire problem you had to face when creating OpenSync and how did you face it?

Cornelius:
Fortunately the problems which I personally would consider to be dire are solved by the implementation of OpenSync which is well hidden from the outside world and [they are] an area I didn't work on ;-)

Armin:
I guess that I am the right person to answer this question then :)

The most complicated part of OpenSync is definitely the format conversion which is responsible for converting the format of one device to the format that another device understands.

There are a lot of subsystems in this format conversion which makes it so complex, like conversion path searching, comparing items, detection of mime types and last but not least the conversion itself. So this was a hard piece of work.

What was the greatest moment for you?

Cornelius:
I think the greatest moment was when, after three days of concentrated hacking, we had a first working version of the KDE frontend for OpenSync. This was at a meeting at the SUSE offices in Nuernberg and we were able to successfully do a small presentation and demo to a group of interested SUSE people.

Armin:
I don't remember a distinct "greatest moment". But what is a really great feeling is to see that a project catches on, that other people get involved, use the code you have written and improve it in ways that you haven't thought of initially.

Tobias:
Hmm, also hacking on OpenSync/KitcheSync is much fun in general, the greatest moment was when the new KitchenSync frontend synced two directories via OpenSync the first time. But it was also cool when we managed to get the IrMC plugin working again after porting it to OpenSync.

As we now know the worst problem you faced and your greatest moment, the only one missing is: what was your weirdest experience while working on OpenSync?

Cornelius:
Not directly related to OpenSync, but pretty weird was meeting a co-worker at the Amsterdam airport when returning from the last OpenSync meeting. I don't know how high the chance is to meet somebody you know on a big random airport not related at all to the places where you or the other person live, but it was quite surprising.

Tobias:
Since my favorite language is C++, I was always confused how people can use plain C for such a project, half the time your are busy with writing code for allocating/freeing memory areas. Nevertheless Armin did a great job and he is always a help for solving strange C problems :)

Devices and Programs

Now I'd like to move on to some more specific questions about current and planned abilities of OpenSync. As first, I've got a personal one: I have an old iPod sitting around here. Can I or will I be able to use a program utilizing OpenSync to synchronize my calendars, contacts and music to it?

Cornelius:
I'm not aware of any iPod support for OpenSync up to now, but if it doesn't exist yet, why not write it? OpenSync makes this easy. This is a chance for everybody with the personal desire to sync one device or another to get involved.

Armin:
I dont think that there is iPod support yet for OpenSync. But it would definitely be possible to use OpenSync for this task. So if someone would like to implement an iPod plugin, I would be glad to help :)

Which other devices do you already support?

Cornelius:
At this time, OpenSync supports Palms, SyncML and IrMC capable devices.

Which programs already implement OpenSync and where can we check back to find new additions?

Cornelius:
On the application side there is support for Evolution [GNOME] and Kontact with KitchenSync [KDE] on the frontend side and the backend side and some more. I expect that further applications will adopt OpenSync once the 1.0 version is released.

Armin:
Besides Kitchensync there already is a command line tool and a port of the multisync GUI. Aside from the GUIs, I would really like to see OpenSync being used in other applications as well. One possibility for example would to integrate OpenSync into Evolution to give users the possibility to synchronize their devices directly from this application. News can generally be found on the OpenSync website.

Technical Dive

So far for questions for users. I think it is time to give the developers something to devour, too. I'll keep this as a short twice-fold technical dive before coming to the takeoff question, even though I'm sure there's information for a double-volume book on technical subleties.

As first dive: how did you integrate OpenSync in KitchenSync, viewed from the coding side?

Cornelius:
OpenSync provides a C interface. We wrapped this with a small C++ library and put KitchenSync on top. Due to the object oriented nature of the OpenSync interfaces this was quite easy.

Recently I also started to write a D-Bus frontend for OpenSync. This also is a nice way to integrate OpenSync which provides a wide variety of options regarding programming languages and system configurations.

And for the second, deeper dive: Can you give us a quick outline of those inner workings of OpenSync, from the developers view, which make OpenSync especially viable for application in several different desktop environments?

Cornelius:
That's really a question for Armin. For those who are interested I would recommend to have a look at the OpenSync website. There is a nice white paper about the internal structure and functionality of OpenSync.

Armin:
OpenSync consists of several parts:

First there is the plugin API which defines what functions a plugin has to implement so that OpenSync can dlopen() it. There are 2 types of plugins:

A sync plugin which can synchronize a certain device or application and which provides functions for the initialization, handling the connection to a device and reading and writing items. Then there is a format plugin which defines a format and how to convert, compare and detect it.

The next part is a set of helper functions which are provided to ease to programming of synchronization plugins. These helper functions include things like handling plugin config files, HashTables which can be used to detect changes in sets of items, functions to detect when a resync of devices is necessary etc.

The syncing logic itself resides in the sync engine, which is a separate part. The sync engine is responsible for deciding when to call the connect function of a plugin, when to read or write from it. The engine also takes care of invoking the format conversion functions so that each plugin gets the items in its required format.

If you want more information and details about the inner workings of OpenSync, you should really visit the opensync.org website or ask its developers.

Takeoff

To add some more spice for those of our readers, whose interest you just managed to spawn (or to skyrocket), please tell us where they can get more information on the OpenSync Framework, how they can best meet and help you and how they can help improving sync-support for KDE by helping OpenSync.

Cornelius:
Again, the OpenSync web site is the right source for information. Regarding the KDE side, the [email protected] mailing list is probably the right address. At the moment the most important help would be everything which gets the OpenSync 1.0 release done.

[And even though] I already said it, it can't be repeated too often: OpenSync will be the one unified syncing solution for the free desktop. Cross-device, cross-platform, cross-desktop.

It's the first time I feel good when thinking about syncing ;-).

Armin:
Regarding OpenSync, the best places to ask would be the opensync mailing lists at sourceforge or the #opensync irc channel on the freenode.net servers.

There are always a lot of things where we could need a helping hand and where we would be really glad to get some help. So everyone who is interested in OpenSync is welcome to join.

Many thanks for your time!

Cornelius:
Thanks for doing the interview. It's always fun to talk about OpenSync, because it's really the right thing.

Armin:
Thank you for taking your time and doing this interview. I really appreciate your help!

Tobias:
Thanks for your work. Publication and marketing is something that is really missing in the open source community. We have nice software but nobody knows ;)

Further Information on OpenSync can be found on the OpenSync Website.

This Interview was done by Arne Babenhauserheide in April 2006 via e-mail and KOffice
on behalf of SpreadKDE.org and the Dot. It is licensed under the
cc-attribution-sharealike-license.

Dot Categories: 

Comments

by Inge Wallin (not verified)

Will OpenSync / KitchenSync give me a nice way to sync my laptop automatically when I put it in a docking station or connect via VPN to my office from, say, my home or a hotel?

by AC (not verified)

Perhaps komparator can do the job?
http://kde-apps.org/content/show.php?content=33545

by M (not verified)

Unison is the cross platform tool for this: http://www.cis.upenn.edu/~bcpierce/unison/

by bsander (not verified)

Is OpenSync capable of syncing over bluetooth? I to finally have my calendar synced over bluetooth with SyncML to my phone :)

by pmarat (not verified)

I have already did this with its predecessor Multisync.

Look at:
http://multisync.sourceforge.net/wiki/index.php?Nokia6600Instructions

There are my experiences with my good old 7650.

by Armin Bauer (not verified)

Yes, OpenSync is already capable of syncing over bluetooth with syncml capable phones (i tested it with my nokia 6680). But there still might be problems, since a lot of phones have bugs in their syncml implementation i have to work around these bugs.

If you want to try and run into problems, just post them to the opensync mailing list and i will try to fix them.

BR, Armin

by Michael Jahn (not verified)

Can you sync with a bluetooth cell phone (SyncML) and kontact? kbluetoothd is set up, but I don't know how to continue from there. I searched a lot (google) but failed the last time I tried. Is there a howto of some kind? A mailing list to ask? Or will this be possible with kontact in the kde 4.0 series? The cell phone is a sony ericsson w800i.

by bsander (not verified)

From what I can gather from the article: Pretty soon (with OpenSync 1.0) a KDE frontend will be released which should plug into Kontact and then you can.

by Michael Jahn (not verified)

Oh, and by the way - very good article!

by Arne Babenhause... (not verified)

Thanks a lot!

I'm glad you like the text (and it feels good to hear; creates the fuzzy warm feeling to have done good work :)).

by Ascay (not verified)

I wonder how they will manage the quirks of different devices/software.

E.g. my desktop PIM supports events that last longer than one day - my Palm Tungsten E2 calender doesn't.

The Palm and the desktop PIM can store several e-mail adresses and phone numbers for one person - my Motorola SLVR V7 cell phone seperates such data in different persons.

Syncing such data generates strange effects. Of course you can seperate a multiple day event into a few single day events but that works only in one direction.

I'm curious how they will engage those problems. My present experiences with other sync software wasn't much encouraging...

I also hope there will be a quite simple way for people to add support for devices themselves without much programming knowledge (Firefox extensions are a good example, they are quite simple to develop).
A sync platform rises and falls with its amount of supported devices. It should be very easy to add support for your cell phone or PDA yourself.

by N.N. (not verified)

I can't access www.opensync.org, what's the problem?

by pmarat (not verified)

I think it is a nameserver problem.

Try 129.187.150.92 or add it as opensync.org in your /etc/hosts.

by Sven (not verified)

This "problem" exists for a year at least. What's wrong with the DNS-entry and should it not be fixed in order to get more folks on that site?

by hup (not verified)

It is very difficult for me to keep track of the different Sync frameworks, projects, libraries etc. It seems every 12 month a new sync project is started.

by Paul Eggleton (not verified)

That's the point of OpenSync. Most of the previous syncing projects have been too specific (cellphones only, Palm only, etc.) or tied to a particular platform (eg. old KitchenSync, MultiSync). As mentioned in the article OpenSync is not tied to a specific platform nor is it limited to one class of devices. Thus it has a greater chance of being successful than previous efforts, because it has a wider audience.

by Cornelius Schumacher (not verified)

OpenSync is meant to solve exactly this problem.

by Gonzalo (not verified)

When can we expect Opensync to be included in a shipping distro? What syncing framework is currently included in Suse 10.1 or the upcoming Kubuntu release?

by AC (not verified)

opensync will be part of kde4

by Cornelius Schumacher (not verified)

Once there is a stable release of OpenSync it's safe to assume that all relevant distros will ship it. We have broad support by the distributions.

by Joergen Ramskov (not verified)

And thanks for all the hard work on getting this to function well! This is on of the few missing pieces of the Linux desktop. I very much look forward to the day where I'll be able to sync my phone with my KDE desktop!

by Carsten Pfeiffer (not verified)

Very interesting interview, looking forward to using OpenSync with my S65 handy.

I just want to mention that plugins are mostly a technical thing to create extensible applications. They shouldn't be exposed to users too much.

So instead of having to choose between a
"KDE Personal Information Management Plugin",
a "File Synchronization Plugin",
a "KIO File Synchronization Plugin" etc,

it would be more sensible to have the options

"Synchronize KDE Addressbook and Events"
"Synchronize Files"
"Synchronize Files over a Network" etc.

or something like that.

by Cornelius Schumacher (not verified)

Good point. I completely agree. We will also try to make configuration automatically as far as possible, so that the user doesn't have to choose plugins etc. if this can be done by device detection and things like that.

by asdasd (not verified)

Will this fix the famous "Korganizer events shift time" bug, thats is still open and is from KDE 3.2.3?
http://bugs.kde.org/show_bug.cgi?id=84229