JUL
25
2003

LinuxTag 2003: KSpread Development Roadmap

During the LinuxTag 2003 conference, KOffice hackers Ariya, Norbert and Phillip had a chance to meet and discuss the current state of KSpread as well as the future direction of the project. Plans include an Excel export filter as well as switching to the OpenOffice.org Calc format -- read on for the full KSpread Development Roadmap that resulted from the discussions.

kspread development roadmap

Ariya Hidayat, Norbert Andres, Phillip Müller
July 12, 2003

During LinuxTag 2003, we had a
chance to meet and discuss KSpread developments. This is a
summary of what we have discussed.

what we have done

We are now close to the release of KSpread 1.3 (as part of KOffice
1.3). Beta 3 will be soon available, and release date is expected to be
in mid-September.

Since the latest KSpread 1.2, we have done several important things
such as:

  • styles support
  • performance improvement, as well as reduction in memory usage
  • addition of many built-in functions
  • better support for foreign file formats, including OpenOffice.org
    Calc document
  • WYSIWYG enhancement, with real zoom and high-resolution printing
  • bugfixes and other improvement (templates, conditions,
    drag-and-drop, and many others)

Until now, KSpread has been maintained by Laurent Montel. Core
developers are Norbert Andres, Ariya Hidayat, Phillip Müller, with support
from other active KOffice hackers like David Faure and Lukáš
Tinkl. Documentation is mainly written by Pamela Robert. Although we do not
have an official QA team, Ferdinand Gassauer is always very helpful
in reporting various annoying bugs.

where we are going

Our goals for the next release are:

  • more efficient memory consumption, necessary for working with
    large documents
  • full support for OpenOffice.org Calc document format
  • near 100% compatibility for Excel formula functions
  • bidi support
  • better and smarter handling for calculation dependency
  • allow support for up to 4 milllions rows
  • real-zoom for the chart component

work items

Following are a few things which will be developed or are already
being worked on.

  • Macro recorder
  • New format storage data structure, necessary to avoid storing
    format information in each cell. Cache, tree, and other solutions for
    optimization might be required but have to be researched first.
  • Format parser to speed up painting routine. In short, all
    information for painting a cell can be looked up very fast and thereby
    avoid any painting bottleneck.
  • Formula engine which will replace KoScript. It will be a
    stack-based
    interpreter for faster and efficient expression evalution. All built-in
    function will need to be modified but then we will have the possibility
    for external function definition (say, in Perl). via plug-in. There
    will be also support for localized function names.
  • Source code clean up, to make it easier for new developers to
    jump in and avoid unncesseary clutterness. This will include: adding
    more documentation for core classes,
    redoing user interface parts into .ui files, moving dialog box code
    into dialogs/ sub-directory,  introducing namespace (KSpread::Cell
    instead of KSpreadCell).
  • Revision support
  • Excel export filter.
  • Faster document saving and loading. One possibility is using
    experimental SAX-based document loader (vs the current DOM
    approach) which has the advantage that no intermediate step (i.e
    constructing DOM tree) is performed.
  • Website rework. The current content of KSpread website is already
    too ancient. Some new
    screenshots (like KSpread using colorful Keramik style, or KSpread
    running on HP-UX) will be additional bonus.
  • Maintainership. As agreed, for future release Ariya will
    co-maintain KSpread with Laurent.
  • Switch to OpenOffice.org Calc file format. Somehow this
    should be done in a coordinated manner with other KOffice applications.

how you can help
Since we are relatively busy with other things and KSpread has no
full-time developer, any help will be appreciated. Among others, here
are some things which you might want to have a look:

  • File bug report to bugs.kde.org. Check first that it has not been
    reported already.
  • Test and report problems with filters, especially Excel filter.
  • Write and update KSpread documentation. Make sure to contact Pamela first.
  • Provide regressions tests for many built-in functions.

epilog

KSpread still has a future. We love it, and we will always improve it!

Comments

It's nice to read this. I built version 1.2 and then I tried 1.3 beta. The beta was much faster but for some reason it's been unstable lately, so I built CVS and it's been very frustrating. Save As has taken it all down and I've been renaming autosave files which then save fine. I did do a parallel CVS build and was using qt-copy but I've switched back and I'm rebuilding KDE 3.1.2 and CVS. Maybe things will be more stable. Right now in CVS linking to a cell on another page and doing an "if" to compare the text has it comparing accurately if I manually enter the text but not if I paste it. Text comparisons are also case sensitive too, which I don't think users expect, as well as "==" being the comparison operator while I don't think "=" is an assignment operator. It's okay for me as a programmer, but for joe user...

I really love spreadsheets, and I miss Mesa/2 on OS/2 which I beta tested extensively. There are a few things I find really frustrating on Kspread. I have not found an easy method to create date sequences. I'm trying to recall if I succeeded last time. It's not inuitive, and Kspread seemed intuitive when I first tried it. I also can't figure out how to use cell references to adjust formulas based on the cell or row and I can't seem to find sumproduct to multiply two columns. I know, just like with Quanta, you can only do so much, but for me the problem seems to me that I can't really seem to do work on Kspread without the love/hate aspect. OO Calc has great stuff but it's a dog and dog ugly. I wish Kspread were just a bit better. Also areas it seems to have gone back downhill are in building formulas by point and click. A number of times a click just leaves an error where the operator is there without a cell reference during edits. Clicking across pages seems completely disfunctional using CVS on KDE 3.1x. Also find and replace no longer seems to have the power to look inside cells. I also see no easy way to detect the last cell with data in a range. That would be handy. Other things I miss are the custom features to build combobox lists and scripts... I should say something good. My wife is now a spreadsheet junkie too and other than having to break things down to smaller sheets she has no complaints and does everything under the sun on Kspread. I'm just more demanding. ;-)

While we're at it I have a request. It looks like DCOP may be good enough to enable poking data into cells and such. What would be really sweet is the ability to call a shell command with parameters. Then I could easily use Kommander for custom data entry on little spreadhseet apps. It wouldn't take much to get these two to do great things together. I should probably get involved in advanced testing... I just have my hands so full with Quanta, but I really want Kspread to be great. BTW the coolest thing with Mesa/2 was it's dynamic object copy feature. You could create a formula in cell E3 and drag copy down to E15. Then you could edit the formula with a correction in E3 and E4:E15 would change automatically. It was fun helping to spec a lot of interface features there too...


By Eric Laffoon at Fri, 2003/07/25 - 5:00am

I won't tell you something new, but what about bugs.kde.org wishes so other people can vote?


By AC at Fri, 2003/07/25 - 5:00am

I'm rebuilding everything on my system right now (along with recovering greehouses, getting new product lines together, trying to get a Quanta snapshot out and getting ready to go out of town for the weekend...) As soon as I can confirm these bugs on a clean build I will report and get involved. (I'm just afraid I'll look at the code and have to give up sleeping)


By Eric Laffoon at Fri, 2003/07/25 - 5:00am

VBA?

I would be very good to support VBA, I guess Mono implements VB.NET. And perhaps also advances Linux basic implementatiosn like Hbasic may help: hbasic.sf.net


By Gerd at Fri, 2003/07/25 - 5:00am

You want to implement virus support in kspread??


By renaud at Fri, 2003/07/25 - 5:00am

We will have a JavaScript based scripting language as a plugin (but without many dangerous things, like access to other applications/file system...). Mainly you will be able to do all the things you can do with the GUI.


By Norbert at Fri, 2003/07/25 - 5:00am

Are you using QSA? Will thre be DB access and all Qt features?


By panzi at Fri, 2003/07/25 - 5:00am

My current plugin on the hard disc uses QSA indeed, but I'm thinking about using kjs instead.

You will be able to do everything you can do using the KSpread user interface, so you get the database access KSpread currently has.


By Norbert at Fri, 2003/07/25 - 5:00am

Basic is from hell. There's no doubt about that. If you can initiate a shell process then you can run a Kommander dialog which can speak DCOP. So whatever you have available in DCOP is what it could do. This would be very cool because it's here now (in the quanta module) and it doesn't require any actual scripting for simple tasks like a data entry widget. Have you looked at Kommander? It's based on Qt Designer and the concept of associating text with widgets to assemble strings. It can use scripting in any language that runs in the shell and is DCOP enabled. Our objective is to extend it to where it is very user friendly to non programmers. We will also be adding data widgets soon.

It would be very cool to be able to use visual dialogs for spreadsheet mini applications.


By Eric Laffoon at Fri, 2003/07/25 - 5:00am

It would be nice with a small Kommander tutorial. I see you talking about it a lot, still I don't know exactly what the h*ll it is. ;) A small text to help people to see the light would be good.


By Apollo Creed at Fri, 2003/07/25 - 5:00am

See: http://quanta.sourceforge.net/main2.php?snapfile=snap02
Also check out the quanta/quanta/data/scripts directory in the Quanta CVS HEAD (KDE CVS servers) for examples.

Andras


By Andras Mantia at Sat, 2003/07/26 - 5:00am

There is a tutorial with examples on the Quanta site with our downloads. In a nutshell it's a graphical dialog builder based on Qt Designer. The twist is that it allows you to "associate text" with it's widgets. So a button could gather up text from the widgets in a dialog and assemble a string which it could sent to stdout or via DCOP, as an example of what it can do. It can also script natively in shell scripting (bash) and load and run other scripts like Perl or Python. All of this combines to make it very powerful while being relatively simple compared to language based tools.

I'll attach an example. If you have Quanta 3.1 or greater installed on your system you can run Kommander files with the command "kmdr-executor [path/]filename" or edit them with the command "kmdr-editor [path/]filename"


By Eric Laffoon at Tue, 2003/07/29 - 5:00am

As weird as it sounds, I prefer KOffice a million times to Open Office. KSpread is awesome. It has a few quirks to it, but I'm confident in its future and those of other KOffice components. I only wish there are more developers for it, full time ones especially. However, you guys are doing a wonderful job and I am a great fan and a humble user of our priceless baby, KOffice. :-)

Regards,

Mystilleef


By Mystilleef at Fri, 2003/07/25 - 5:00am

Thanks lot. We're glad that KSpread could be useful for you.
We can't promise, but you can have our words that we always improve it for the future release.


By Ariya at Fri, 2003/07/25 - 5:00am

> I only wish there are more developers for it, full time ones especially.

It makes a big difference. Having Andras full time on Quanta has enabled us to do things on a level we would never have tried otherwise. If I had the resources I'd love to sponsor someone full time on the Kspread team. I'm reluctant to get involved myself because I already spend way too much time on Quanta. However I'm just one guy who does a little web work and owns a small business with no employees. I really don't understand why other business people can't understand it makes sense to shell out a small portion of their IT budget to sponsor development. If you guys need a business type to sit at the table and arm wrestle a potential sponsor let me know. ;-)


By Eric Laffoon at Fri, 2003/07/25 - 5:00am

so its finally catching up with gnumeric?


By printf at Fri, 2003/07/25 - 5:00am

Strange comparison. Gnumeric does presentations?


By Anonymous at Fri, 2003/07/25 - 5:00am

Gnumeric is the gnome-office spreadsheet, not a presentation program.


By Eike Dehling at Fri, 2003/08/08 - 5:00am

Correct, so KOffice has at least a point for presentation.


By Anonymous at Fri, 2003/08/08 - 5:00am

I disagree! That's like saying "choice is pointless". Personally I can't wait for KOffice to mature. I really like KWord with its WYSIWYG-ness, and the other applications are pretty slick too.

If you make another choice - and use Gnumeric, or Scalc, or Excel, well that's great. It makes no difference to me. It's YOUR choice! With open source software, very little is sacrificed for you to have choices. Look at KSpread's support for the Scalc file format for example, possible because both are open source. With closed source software, making a choice would usually lock you in to varying degrees, but with open source, you can make one choice which suits you now (to use the more developed Gnumeric for instance), and switch over later (if you like!) when KSpread or KOffice offers something you would like to take advantage of (like presentations, or integration with other apps, or speed, or stability, or how it looks, or whatever.

Is it really such a hard thing to avoid pointless generalizations such as "koffice is pointless"?


By Paul Dorman at Tue, 2003/07/29 - 5:00am

Development is more fast in Gnumeric ... Why ?


By anonymous ... at Fri, 2003/07/25 - 5:00am

gnumeric's original author has stuck with the project for nearly five years..

kspread's haven't.. through a variety of real life issues that occur with open src projects..


By anon at Fri, 2003/07/25 - 5:00am

KSpread suffers from the existance of GNumeric.

I suggest GTK apps stop to exist.

Yours, Kay

PS: Irony included, guess in which sentence ;-)


By Debian User at Fri, 2003/07/25 - 5:00am

Yes, ok, but what about bug 57871?

This seems to be a pretty drastic change or else it would have already been done. I sure hope they plan on implementing this. Not being able to select multiple, non-contiguous rows / columns means KSpread is still behind Excel from at least the Windows 98 era (6+ years ago!) ...

If this functionality interest you but have no KOffice hacking skills (like myself unfortunately) please at least vote for the bug.


By Jesse at Fri, 2003/07/25 - 5:00am

That's on our target, don't worry. But it's just that first we need to restructure internals of KSpread to become much more memory-efficient. Else, nifty features and other advanced stuff would be memory hungry and therefore less useful.


By Ariya at Fri, 2003/07/25 - 5:00am

Anyone know if people are working on improving the chart support. Currently the types of charts is very limited and have no anti-aliasing.


By Vincent at Fri, 2003/07/25 - 5:00am

Anti-aliasing is a canvas problem I guess. Wait for http://www.cairographics.org/, when this is ready it is likely that someone will write an anti-aliased canvas. Right now, with libart being the only solution, it does not make that much sense...


By AC at Fri, 2003/07/25 - 5:00am

This is, to me, the KILLER feature. I prefer koffice interface by far, but I end up using openoffice (and recommending it to others) when I need to do "office" stuff. The reason is compatibility with the m$ monopolic world.

The sooner koffice switches to openoffice formats and merges whatever they have that openoffice doesn't in their filters, the sooner openoffice will get widely adopted. I for one would like to see both of them coexist :-)


By MandrakeUser at Fri, 2003/07/25 - 5:00am

Yeah, I agree, if you switch to OOo file format so much becomes possible.

using uno, you can setup OOo to be a server, which can accept files from KSpread (in OOo format), and save them in whatever file format you want (that OOo supports). You can avoid reinventing many wheels :)

J.


By Jonathon at Thu, 2003/08/14 - 5:00am

Hmm, R integration


By bert at Mon, 2003/08/04 - 5:00am

Fabolous! Kspread with R-project support will be great!. A lot of users in scientific area will be very happy with this support.


By dean at Thu, 2003/08/14 - 5:00am