If you are a scientist and KDE user, you may already have been impressed by the range and quality of specialist software available for your use. But you may not be aware of the scientific graph plotting applications LabPlot (stable KDE 3 release with a KDE 4 version in progress) and SciDAVis (Qt4). The two projects have announced a collaboration to share common backend code to accelerate their development. We contacted the developers to get their take on their applications, the collaboration and the future of their projects. Read on to find out about the future of free software plotting.
SciDAVis 0.2.3 in action
The core developers we interviewed were extremely helpful in replying to a long list of questions we sent them in great detail and the following is a heavily edited summary of their responses. If you'd like to see exactly what they said then
read the full interview.
LabPlot was started back in 2001 by Stefan Gerlach (a scientist, lecturer and IT administrator in the department of theoretical physics at the University of Konstanz) because he “needed a good plotting program and couldn't find any”. In 2006 he realized it “was time to take the next step [...] to really open up the project” and started working on the KDE 4 LabPlot 2. Stefan has another claim to fame in starting the liborigin project, a library used to import projects from the proprietary Origin program. Alexander Semke (working in theoretical hadron physics at GSI in Darmstadt) joined the LabPlot project in 2008. He'd previously used Grace, but “was never happy and comfortable with this motif-based piece of software”. LabPlot, though not everything he wanted, “had the biggest potential”. So, he joined the project: “I contacted Stefan and offered my help. There were no objections :-)”
Stefan describes LabPlot as a free KDE data analysis and visualization program that “tries to combine most of the features needed for advanced data analysis and high-quality plots under a user friendly GUI”. Alexander wants SciDAVis and LabPlot to be “the plotting application the Linux users thinks of first”.
Tilman Benkert (a former scientist now working as a software developer in Stuttgart) joined QtiPlot development after having to use Origin for his PhD thesis and publications due to a lack of a comparable free software alternative. He and Knut Franke (currently working on his diploma thesis in theoretical physics near Cologne) forked QtiPlot in 2007 to start the SciDAVis (Scientific Data Analysis and Visualisation) project and they were joined by Roger Gadiou, QtiPlot's documentation writer at the time. As Knut puts it, the fork happened because they had “different ideas [than Ion Vasilief, QtiPlot's founder and main developer] about many things, including design goals, management of community resources and the right way to make money from a free software project”. Extensive sharing of code between QtiPlot and SciDAVis in the future is unlikely – Tilman explains that “SciDAVis 0.2.0 already features an almost complete rewrite of the table and matrix code to make (among other things) the undo/redo functionality possible”. However, Knut sees some libraries used in QtiPlot that may have an application in SciDAVis: “namely liborigin2 (an improved version of the liborigin project[...]) and libraries for exporting graphics from Qt to TeX and to EMF”.
Tilman describes SciDAVis as “an interactive cross-platform data analysis and visualization program [...] aimed at high-quality plotting of scientific data” and says it “strives to combine an intuitive, easy-to-use graphical user interface with powerful features”. Knut agrees with this and explains his ultimate vision for SciDAVis as being “simple in contrast to completely script-driven tools like GNUPlot or matplotlib and [...] badly designed GUI tools such as Origin and Grace”. Additionally, SciDAVis is “powerful, because you can write Python scripts for automating tasks”. However, he modestly states that “in practically all regards, we haven't reached these grand goals yet, which is reflected by the pre-1.0 version number. Still, SciDAVis is already usable for many tasks”.
3D graphing in the KDE 3 version of LabPlot
None of the developers get paid for working on the projects, so why do they contribute and make their work available as free software? There's a general sense that it's something they enjoy and that they want to make a contribution to the free software community. As Knut puts it:
“I'm using the application myself, it is rewarding to see your work being useful to other people and it's a chance to give back something to the free software community that has provided me with so many great tools”. Stefan also wants to
“give something back” and Tilman enjoys
“creating something useful, practicing my coding skills and being an active part of the free software community”. For Alexander it's a mixture of scratching his own itch -
“a stable and feature complete plotting software for linux for my own use” - and giving something back. He's been using Linux for about ten years and
“from the very beginning [...] I wanted to be a part of this community, but the time elapsed and nothing happened due to the lack of time” but the
“community can only survive, if a certain amount of developers participate actively in it [...] there always was a feeling of guilt – you can help, you do have the knowledge for it [...] but you don't contribute anything”.
Stefan started LabPlot as free software with the aim of “working together without any limitations”. He also had a desire to “be part of this community that created software with a quality that proprietary applications hardly ever reach”. For SciDAVis, forking from GPL QtiPlot code meant there was never going to be a debate over the licence to use. However, Knut “wouldn't want to change the license”. He acknowledges that they may “have fewer resources than a comparable commercial project, so whether it's the best way of 'getting things done' is probably arguable. It is a great way of getting the right things done, though; because you have no deadlines, marketing requirements or hierarchies that influence decisions. If you want something changed, you just do it”. Equally, “compared with selling an application as a freelance developer, you get more contributions from other people, enabling you to create something larger than what you could possible do on your own; and you get free promotion by being included in Linux distributions, and by being featured on the dot. ;-)”. Tilman summarises: “It's basically because of the four freedoms of the free software definition”.
The developers haven't generally been involved heavily in other projects (except for Stefan's work on liborigin), mainly due to a lack of time. Tilman would like to be involved in other projects “if a day had at least 48 hours. ;-)”
Surface plots in the KDE 3 version of LabPlot
So how did the projects come to work together and what can we expect from the collaboration? When starting SciDAVis Tilman already knew about LabPlot but back then
“there was only version 1.x which is using Qt3. Since SciDAVis used Qt4 right from the start and LabPlot 1.x also didn't have an architecture similar to what we wanted, it wasn't very well suited for a collaboration at first”. The collaboration grew out of the rewrite of LabPlot for KDE 4 and a user suggesting sharing their efforts. As Tilman recalls,
“some time after the LabPlot guys started the KDE 4/Qt4-based rewrite (version 2.x), a SciDAVis user pointed out that we seem to pursue similar goals. So we contacted the labplot-devel mailing list, found out that he was right, and the collaboration started”.
Alexander has high hopes for the collaboration: “The hope is that this [...] will lead to the best plotting software the world has ever seen :-)”. Agreed is that “a big part of the code (primarily placed in the backend) will be used in both projects” and Stefan hopes this may be “made available as libraries some day”. Knut comments that such a library “could also be used for creating other (possibly special-purpose) interfaces and for stand-alone scripts”.
The collaboration has stopped short of merging the projects for a few reasons. There are (presently at least) different approaches to the user interface. Alexander notes that “SciDAVis, being a fork of QtiPlot, provides an Origin like way of doing plotting. LabPlot has a different approach. Both programs have their own user basis. This fact justifies the development of two UIs supporting different workflows”. Knut agrees with this but sees a bigger obstacle in the choice of pure Qt or KDE in the applications: “SciDAVis is expressly cross-platform, and the practical viability of KDE on Windows and Mac OS X remains to be proven. LabPlot on the other hand puts some emphasis on its integration with KDE”. Tilman also believes that “Mac OS and Windows users would be discouraged to use SciDAVis if they have to install KDE as a dependency”. Further unification down the line is not ruled out however – Alexander ponders the possibility that “maybe we'll rethink the current designs and end up with a common interface. In this case we'll still provide a KDE version of the program to better fit into the desktop” while Stefan says “if it turns out that no one is using the KDE frontend anymore I would of course help to support the Qt frontend”. In any case, Alexander argues that “a big part of the code will be used in both projects so concentration of resources on the common goals is already achieved”.
An early preview of a KDE 4 version of LabPlot
The respective choices of Qt and KDE follow similar reasoning. SciDAVis came from a fork of the Qt QtiPlot and so the simplest thing was to keep it Qt. For Stefan
“the main platform for LabPlot always was the free software desktop. KDE has some nice things that are not available in Qt (like standard dialogs, icons, communication between application, application configurations, etc.)”. He's also excited about the possibilities KDE 4 will open up:
“now that KDE 4 is supported on Mac and even on Windows platforms we may be able to have these features available there too”. For Alexander it was simple:
“KDE is my favorite desktop. That's it :-)”. The developers also acknowledge several other projects whose technology they put to good use: the GNU Scientific Library for math, muParser, SIP and PyQt for scripting, QwtPlot3D for 3D plotting, and Qwt for 2D although Tilman says they are
“working on replacing Qwt, as it is focused on widgets rather than export quality and has a couple of limitations due to it's backward compatibility to Qt3”. Stefan hopes for more import/export libraries.
Both projects have seen a lot of rewriting of code they inherited from their predecessors. Stefan began a direct port of LabPlot from KDE 3 to KDE 4 “but soon realized that it would be less work and produces cleaner code to start from scratch”. Alexander traces these major changes back to the collaboration with SciDAVis: “there was a very vivid discussion on labplot-devel about the future plans and it became very soon clear that it is worth rewriting the application completely. The SciDAVis guys brought a lot of code into the LabPlot project [...] so we didn't need to start from scratch.” He wants to move away from the dialog based interface of LabPlot 1.x - “the problems with dialogs are apparent – they cover the important information but they make the user click the 'Ok' or 'Apply' buttons all the time. This problems can be solved by using dock widgets like in Koffice2”. Tilman says that in SciDAVis too there have been a lot of changes from the QtiPlot code - “we have been and will be replacing a lot of code completely to overcome limitations of the Qt3 heritage and the inflexible architecture”.
With all these changes taking place, you might wonder about the current stability of the applications. Knut says “SciDAVis is pretty stable; fixing bugs has been a major focus of our work but the common development code (LabPlot 2.0 alpha) is neither particularly stable nor anywhere near feature-complete”. Tilman also gives SciDAVis a vote of confidence: “the stable SciDAVis branch is maintained separately from the bleeding edge development code and new features are merged only slowly into the stable version to keep it stable and usable at all times”. For LabPlot, Stefan states: “LabPlot 1.6 is stable since 2006” and Alexander also believes it is “quite usable but has some issues with stability and broken layouts at different locations in the UI[...] The alpha version of LabPlot 2 is not usable at the moment”. Nonetheless, the brave can check out the current development code of LabPlot 2 from the project's SVN. Do the developers use their own applications in their daily work? Tilman doesn't need such an application day to day, while Stefan uses GNUPlot and Grace mainly. Knut is using SciDAVis and Alexander uses mostly a combination of Mathematica and Grace.
Using built-in data analysis tools in SciDAVis
LabPlot 1.6 and SciDAVis 0.2 are the releases for getting work done at the moment - but when can we expect to see LabPlot 2 and SciDAVis 0.3 arriving? The consensus was
“when they are ready” although Tilman offered that it depends on when
“the 50 developers come along who help us finish them in no time ;-)”. There is a need for contributors then. Tilman is glad that
“SciDAVis benefits from several contributors who write documentation, prepare binary packages, provide translations, report bugs, and suggest features. Most needed are developers, though”. Stefan's current wishes for LabPlot are
“help regarding the website and the documentation of LabPlot needs to be written someday. Of course developers are always welcome, but also non-coders can send us bug-reports and ideas in various ways (bug tracker, mailing lists, etc.)”. Knut has a catalogue of potential jobs waiting for you:
“bugfixing / feature implementation tasks [...] documentation writing, creating packages for Linux distributions that don't have one (e.g. Ubuntu) [...] translations [...] extending the web pages and/or creating wiki content. Apart from that, if you think you can make the application or the project better/prettier/friendlier in some other way, do let us know”.
Outside LabPLot and SciDAVis, the core developers are all using KDE on Linux, although Tilman also uses Windows for gaming and in his job. Mostly they are on KDE 4, except for Stefan who mainly uses KDE 3. When asked for their favourite KDE applications Kmail/Kontact, Amarok, Krusader, K3B, Konsole, Okular, Dragon Player, Kdevelop, Kate, Kile, Digikam, Konqueror and Kopete all got honorable mentions. Knut admires Kmail as an example of an advanced application with an intuitive interface, something he hopes to replicate in SciDAVis: “it is usable even for novice users, although it has some rather advanced features”. Reflecting on the strengths of KDE, Knut would also “like to mention the nifty Plasma desktop at this point :-)”. Not too much seems to be lacking from their ideal desktop, but Stefan would like to see “a nice application for organizing scientific papers”.
Any final words from the four interviewees? Well, Stefan wanted to “thank all people working on SciDAVis and LabPlot. It is really nice to work with you on this project. Hopefully we can sort all things out soon and will be able to deliver an application that was never available before :-)”. All seemed keen to take advantage of the opportunity for a little promotion. If you produce graphs in your daily work (or just for fun) and find that KOffice and OpenOffice don't quite do it for you and the likes of GNUPlot are a little scary then LabPlot and SciDAVis are both well worth checking out. They'll both give you publication quality plots of your data – so then all you'll need is some publication-ready data.
Comments:
Academia and FOSS - einar - 2009-10-16
Thanks for the article - very insightful and interesting to see that there is a place for KDE in science (being a scientist myself...).
Actually, I think that the academic world would greatly benefit from some FOSS like mindset. Academia should be a place where ideas circulate and flourish, but at least in competitive areas the dreaded "publish or perish" syndrome is IMO causing a lot of trouble.
The matter is undoubtedly complex.
It does - jadrian - 2009-10-17
Academia, for the most part, does have an FOSS like mindset. Publishing is releasing the source. It is out there for everyone to see, use, and modify as they please.
If anything the need for publishing might have some impact on a "release soon release often" policy. But even there, seminars are a constant in research groups and you constantly see people talking about their work, what they have done, what they are doing, where they are stuck, possibilities for future research. There are very few secrets.
The process, maybe. The results - not yet. - jospoortvliet - 2009-10-17
Sure. But as of now, many research papers are still only available to academics or ppl willing to shelve out the heavy subscription rates to academic journals. That's a big issue. Furthermore, many academical people work on something new and amazing, but their efforts stop as soon as they have published their paper. They might build upon Free Software but often don't put in the effort to actively contribute it back to the community. In effect, it often contributes little to nothing unless the research is noted by somebody who then does the work of integrating it in the FOSS stack.
Of course, both issues are being worked on - there are various open research archives & projects; and there has been talk about the requirements on research papers to include the integration of demonstrated work in real-life applications and contributing back to the community. But the latter is still mostly talk and the former is heavily opposed by the publishing firms due to the lucrative business they have to protect.
Thanks - swjarvis - 2009-10-17
Thanks, I'm glad you liked the article.
As far as KDE in science is concerned: I'd struggle without Kile and KBibtex and I've made plenty of use of Kalzium and Marble too.
Access is an issue - swjarvis - 2009-10-17
Yes, it's easy to forget when you're in a large university with subscriptions to most of the major journals how inaccessible a lot of the information is to most people.
Particularly when the research is publicly funded it seems that the results should be available to everyone. There are open access journals but it will take time for them to become mainstream as everyone wants to submit their papers to the highest impact journals and therefore new journals struggle to get high impact factors because the best articles get submitted elsewhere.
Not the case in competitive fields - einar - 2009-10-17
"Academia, for the most part, does have an FOSS like mindset. Publishing is releasing the source. "
In my own experience (bioinformatics), this often not true. In high profile groups, information is basically ketp under very closed doors to make sure you're the first to publish - and hence to get funding. This causes a *blockade* of ideas, rather than circulation (for example, protocols, or methods).
Not to mention the quality itself of some papers... but I digress.
Plus, software (at least in my field), at least for small groups, is rarely released under approved OSI licenses. Often there are web services, with varying quality, whose source will never be available.
FOSS has release early, release often. Academia has, in part, publish or perish.
There needs to be a better balance between secrecy and openness - and some initiatives (google for "open science") are in the good direction.
Publishing - Hans - 2009-10-17
I'm still a student, but my aim is to become a researcher. Although I don't have any experience with publishing, I've heard a little about it from other researchers.
"Publishing is releasing the source"
From what I know, many "traditional" journal publishers require you to transfer the copyright to them. This is a huge difference from FOSS.
I hope this'll change, even if slowly, with initiatives like Open Access.
"If anything the need for publishing might have some impact on a "release soon release often" policy."
http://zs1.smbc-comics.com/comics/20090831.gif
(Shouldn't be taken too seriously. :P)
On a related note, I found it surprising how often people send announcements in .doc or give .ppt (PowerPoint) files (that they've used during a lecture) as part of the course material in the university.
Thanks for the article, more FOSS scientific tools are always appreciated.
There are sites like - vld - 2009-10-17
There are sites like arxiv.org that promote open access to the papers.
On the other hand, integrating the work into FOSS stack is a very difficult issue. Certain kinds of research are extremely difficult to integrate immediately, they are rather targeted to enable future research and could contribute something practical only on a bigger timescale.
Another issue is distinction between inventing the idea and making a usable implementation. The former requires high amount of risk that you idea might simply not work. You have to put huge amount of effort into the proof-of-a-concept implementation knowing that it may simply fail or prove to be useless. That leads to very scratch-like implementations, which is really just a proof-of-a-concept and can't be used in the real life. Such work gives you a fame as an inventor of the idea.
On the other hand, solid implementation of the already-proven idea requires huge amount of design- and coding-related work. This could also be very interesting and also gives you a fame, but this is just too different from risky inventing-testing style of work. Moreover, scientists usually have quite a big pull of ideas to work on after finishing the current one, they just don't have a time for a solid implementation.
FOSS reflects an academic approach - Bobber47 - 2009-10-19
Great article. I have seen & used QtiPlot, but not LabPlot or SciDAVis, so I welcomed this feature and the background of their development.
Regarding the other thread in the comments addressing the mindset of FOSS vs Academia, I've always been perplexed by the comparison. Academia gave rise to the FOSS movement, and the philosophy of FOSS is an expression of the classic Academic approach. University environments are remarkably open, collaboration is popular, and attempts to pursue research via open science are underway. It is true, however, that some aspects of the academic routine deviate from these principles, but only as a practical consequence of the manner in which credit, funding, and jobs are rewarded. When one's livelihood depends on garnering credit (publications), funding (grants) and a job (faculty position?), openness is delayed, but the primary objective is indeed as wide a distribution of the results and use of the knowledge as possible. The issue of journal accessibility is also a practical consequence of publishing businesses, and open access journals are a recent method still gaining traction. Journal impact factors haven't changed yet, and the perception of greater merit still lies with many of the paid journals. I look forward to a more open system of distribution, and WWW methods should certainly facilitate that compared to the traditional print model. But it's difficult to establish an objective metric of contribution or performance in a meritocracy spanning thousands of projects, locations, and workgroups: That metric would determine employment and salary. FOSS is less subject to these monetary issues, and thus more likely to run smoothly along the philosophical approach described. Though I haven't participated in FOSS projects, it would seem likely that personal and political conflicts would still interfere in some way.
