SEP
21
2011

Nepomuk Stability and Performance

Over at his blog, Sebastian Trüg is raising money for Nepomuk. Short version of this story--please give what you can to an important KDE project and a valuable KDE contributor. Background and details below. [update: The fundraiser is done. Thanks to everyone for your support.]

Sebastian’s fundraising effort has support in the KDE community because of the importance of his work on Nepomuk and his lengthy involvement in the KDE community. In 1998, Sebastian created K3B, the CD and DVD authoring application, and maintained it for several years.

Sebastian's Work With Nepomuk

In 2006, Mandriva brought Sebastian into Nepomuk, a Social Semantic Desktop project funded primarily by the European Union "to develop a comprehensive solution for extending the personal desktop into a collaboration environment which supports both personal information management [as well as] sharing and exchange across social and organizational relations". This three year project included big players like IBM, SAP, Thales, HP, leading research facilities such as the Digital Enterprise Research Institute (DERI) and the German Research Center for Artificial Intelligence (DFKI), and several universities and smaller companies. It received an investment of €17 million. The project was intended to be primary research and to lay the groundwork for commercial exploitation. Mandriva's role was to create a bridge to the open-source community, primarily KDE. As a result, the KDE Semantic Desktop was born. When the research project ended in December 2008, Sebastian—with Mandriva's financial support—continued his work on the semantic desktop and made KDE-Nepomuk what it is today.

KDE and Nepomuk

There are a few known implementations of Nepomuk:

Of the three, the KDE implementation has made the most progress toward practical use...in large part thanks to Sebastian’s work and leadership. The KDE implementation is based on the W3C family of specifications for a metadata data model known as RDF (Resource Description Framework) and other industry standards. Thus it is a practical and extensible use of leading edge desktop technologies. Something like Nepomuk has been anticipated since 1945 in "As We May Think", an essay by Vannevar Bush that also predicted technologies such as the personal computer, the Internet and Wikipedia.

Nepomuk is already integrated with some KDE applications. There are many possibilities for extending its use that require more of an R&D approach. With the small Nepomuk team, this is challenging, but it has substantial potential rewards. It supports KDE’s technological leadership. And as free and open source software, it holds promise for endusers beyond KDE. Please see the comments below from several KDE developers.

Foundation Integrity

Detractors have valid complaints about the system that involves Nepomuk, including Akonadi and Strigi. Drawbacks were considered when the decision was made by KDE development leaders to incorporate semantic technology into the fundamental KDE architecture. Since then, significant progress has been made with more still to come. Comments about this fundraising project have demonstrated people’s strong desire for KDE-Nepomuk to have a firmer foundation. In response, your donations will focus on stability and performance, which add stability to KDE applications that use Nepomuk. More Nepomuk features are planned. But they will wait until the quality improvements to the existing implementation are complete.

Please visit Sebastian’s blog for more detail on the plan, and to add your support for the project. Potential Nepomuk investors or employers interested in Nepomuk development, please contact Sebastian Trüg.

Please donate what you can to keep KDE in the lead with this important project.

KDE Application Developers Support Nepomuk

Sebastian Trüg on Nepomuk

Something like Nepomuk has never been done before on the desktop. This makes the development harder. It is important to know that Nepomuk is not about files, not about files alone. It is about any resource on the computer. This includes files, but also emails, people, events, places, projects, tasks, and so on. Nepomuk shifts the focus from files to the resource, which gains meaning by being connected to other resources.

The goal is for the user to be able to work with information rather than physical representation of it on disk. Instead of browsing your file system or your contact list, you look for relations and filter things by context. All in all, the idea is to organize the information more like your brain does.

Peter Penz (Dolphin maintainer)

Nepomuk makes things possible in Dolphin that would not exist without it: rating, tagging and commenting of files, searching, virtual file hierarchies like 'timeline' and more. The best thing with Nepomuk is that the API is well documented and easy to use for application developers. The support from the Nepomuk core developers has been very good; it would be sad to see this promising framework stagnating.

George Goldberg (Telepathy-KDE)

The decision to build the KDE-Telepathy project on top of Nepomuk has allowed us to build an instant messaging system that can integrate with the desktop and PIM environments. The fact that Nepomuk is such a generic and extensive framework means that any other KDE application can make use of the data from and features of KDE-Telepathy without us having to build and maintain yet another specialized solution for application integration.

Ivan Cukic (Plasma Active)

Nepomuk provides the backbone for better inter-application integration than KDE users have ever known before. The developers need not support every and each application individually. They just need to use one of the standard ontologies and store the data in Nepomuk.

Vishesh Handa (Nepomuk team member)

[When I started using Nepomuk,] I loved the fact that it, in a way, understood what it was storing. It wasn't just like another filesystem or something. We were literally building a system that, in its own way, could comprehend what we do with our machines. I'm not talking about a completely self-aware machine or anything like it. I mean the idea that the computer understands - 'Hey! I'm watching a movie, and that movie has these all actors and the movie is based on this book, which I have over here.'

We've seen a proliferation of apps for every thing imaginable, and that's great, but we need to have a common layer that connects everything together. For example - I'm working on an report for a certain project in collaboration with a friend. I fire up my document editing application, and start typing. Later I can say that this document is for this project, and that's it. I don't need to specify a location, think of a name for the document or anything else. I can also specify that I'm working on this report with this person. Nepomuk lets my project management application know all those things about the project.

I could watch a video and say that part of it is relevant to this project. Each application specializes in its own domain. That's where Nepomuk comes in. Nepomuk is like the underlying intelligence that connects everything together. It's the thing that lets my project management application know how a certain individual looks, their email address or when I last talked to them. Chat logs are not the concern of the project management application. And an address book wouldn't know about the person’s appearance. With Nepomuk, each application can still be specialized, and know about the rest of the world as well.

Christian Mollekopf (Zanshin)

Nepomuk finally gives us the means to organize data (files, notes, movies, whatever...) in sensible structures, liberating us from the limits of filesystem-based structures. And it does so in an application-independent manner, allowing for a whole new level of interoperability. Of course it is also an awesome platform to build applications on. I really believe with Nepomuk we can set new standards in the overall user experience of a computing system.

I'm using it mainly in Zanshin to organize Notes and ToDos as well as linking to all kinds of data. I get features like fulltext searching on all data basically for free, only by using Nepomuk.

Andrew Lake (Bangarang)

Nepomuk realizes a long overdue wish of reducing duplication of information on the desktop. Bangarang takes advantage of that by forgoing its own implementation of yet another media library database and simply using the desktop-wide Nepomuk datastore. Benefits include fast start-up times for a media player with full library support for both audio and video libraries as well as media information that is not locked up in an application specific database. Even better, since Nepomuk is about the relationships between information (not just files and file metadata), it is inspiring new, more organic ways of looking at how media can be handled to the benefit of the user. The best of Nepomuk is yet to come.

Martin Klapetek (Telepathy)

I'm a KDE-Telepathy developer. The aim of our team is to create a deeply integrated IM solution for KDE. Part of this integration effort also involves PIM as we'd like to bring IM features into PIM and conversely, we'd like to have PIM data in IM. This can all be done thanks to Nepomuk, which provides us with unified easy-to-use data structures with other relations as a bonus, making it possible to easily reuse its data throughout the whole system. As an example of what could be one day possible thanks to Nepomuk, I'd say this: Imagine taking photos at some party, downloading them with DigiKam and running face recognition on them. It will automatically tag your friends and match them with other data you have stored about them in your system. Now you come to a really funny picture with your friends which made you laugh so badly, that you want to immediately send it to those friends. With Nepomuk enabled, you could immediately see (right from DigiKam) if those particular friends are online and you could send that picture to them right away. One of your friends is offline? No problem, send an email instead (again right from DigiKam). And because all those data are stored in Nepomuk, any other application can take advantage of them and their relations to create a really great and integrated experience in KDE.

Laura Dragan (SemNotes)

Nepomuk helps to make explicit connections between things that before were connected just implicitly. This sometimes leads to discovering connections we never knew before. But even if it doesn't, it always helps with organizing all the information on the computer. There are so many things that Nepomuk can enable, we can't even think of them all. For the ones we can think of, KDE-Nepomuk is a breeze to work with. In SemNotes, I use it for everything, from storage to tagging and most importantly, for making connections between the notes and other things on the computer.

Nepomuk and the Future

Nepomuk holds great promise for KDE and for computing in general. And creates a dilemma for the small development team—balancing the practical and the possible. For a parallel, imagine the Internet in 1969 when its first 2 nodes were connected, and then consider today's Always-On-Connectivity. The first TCP/IP network in 1983. The announcement of the World Wide Web (or the first Linux message) in 1991. The possibilities were always there, waiting to be applied.

Nepomuk is similar...big picture, opportunity, good enough implementation, fix up, next opportunity, all happening concurrently. But with KDE-Nepomuk, the time frame is compressed, and the same small team is doing the entire process.

As to where KDE-Nepomuk might go, "As We May Think" (referenced above) from July 1945 offers clues...

When data of any sort are placed in storage, they are filed alphabetically or numerically, and information is found (when it is) by tracing it down from subclass to subclass. It can be in only one place, unless duplicates are used; one has to have rules as to which path will locate it, and the rules are cumbersome. Having found one item, moreover, one has to emerge from the system and re-enter on a new path.

The human mind does not work that way. It operates by association. With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts, in accordance with some intricate web of trails carried by the cells of the brain. It has other characteristics, of course; trails that are not frequently followed are prone to fade, items are not fully permanent, memory is transitory. Yet the speed of action, the intricacy of trails, the detail of mental pictures, is awe-inspiring beyond all else in nature.

Man cannot hope fully to duplicate this mental process artificially, but he certainly ought to be able to learn from it. In minor ways he may even improve, for his records have relative permanency. The first idea, however, to be drawn from the analogy concerns selection. Selection by association, rather than indexing, may yet be mechanized. One cannot hope thus to equal the speed and flexibility with which the mind follows an associative trail, but it should be possible to beat the mind decisively in regard to the permanence and clarity of the items resurrected from storage.

Consider a future device for individual use, which is a sort of mechanized private file and library. It needs a name, and, to coin one at random, "memex" will do. A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.

This describes the general sense of Nepomuk.

KDE's free and collaborative environment promotes innovation such as KDE-Nepomuk. Even now, the latest Plasma implementation—Plasma Active for mobile devices—is a practical application of KDE-Nepomuk capabilities. Nepomuk is a possibility for any application that can benefit from associative storage and data selection.

Given that, what does Nepomuk provide for people with diminishing mental capabilities? Education? Research? Libraries? Customer service? New product development? Economic development? Social interactions? Any form of computing?

Just imagine.

That's the future of Nepomuk and why we ask you to support Sebastian. [update: The fundraiser is done. Thanks to everyone for your support.]