AUG
28
2007

Pencils Down for KOffice Summer of Code Students!

With an avalanche of last-minute commits, the KOffice Google Summer of Code students finished yet another great Summer of Code. We had some very exciting projects this year, and most of them were as great a success as last year. Read on for details of the achievements.

This year we had:

  • Emanuele Tamponi, Painterly Features for Krita (Mentor: Bart Coppens)
  • Fredy Yanardi, Text-tool plugins for KOffice apps (Mentor: Tomas Mecir)
  • Marijn Kruisselbrink, Music Notation support for KOffice (Mentor: Boudewijn Rempt)
  • Pierre Ducroquet, Improve OpenDocument Compatibility in KWord (Mentor: Sebastian Sauer)
  • Sven Langkamp, Selection Visualisation (Mentor: Camilla Boemann)

We had one disappointment this year, our student due to work on collaborative editing disappeared, only to surface around mid-term with a code drop — before disappearing again. So no collaborative editing in KWord this year. You cannot win them all.

But the other five students have delivered outstanding work, often going far beyond what their mentors thought possible initially. Let's take a look at what they have done up to now!

Emanuele Tamponi

In last week's Commit Digest we already saw part of what Emanuele has done this summer: mixing colours in a way that looks and feels completely natural, by simulating the way light is reflected of pigment. This is a two-way process: to go from a colour defined in RGB to a colour defined by the properties of pigment and light and back to RGB again. The first step is usually done by using a spectrometer and actualling measuring the colours. Emanuele has created code to simulate that without special equipment, and in a way that is repeatable and makes it possible to experiment.

But that was only part of his work: another part of the project was to make it possible to paint with more than one colour at the time and to be able to make the brush pick up existing colours from the canvas: complex loading and bidirectional paint transfer. The colour mixing really grabbed Emanuele, leading him to create more and more sophisticated code, leaving little time for the complex loading and bidi paint transfer part of his project. However, he managed to check in a preliminary version just in time... and right now he is staying in his sea-side cottage coding, coding and coding. More commits are to follow!

Fredy Yanardi

The new Flake architecture for KOffice 2.0 not only enables us to create applications that integrate all kinds of possible content in specialised documents, there is also infrastructure to make it easy to add interesting functionality to shapes through plugins. Fredy's mission was to implement a number of these plugins, both to demonstrate the flexibility of our architecture and to add useful functionality to KWord. Remember, with KWord 2.0 rewrite we lost a lot of existing functionality because it has actually been a complete rewrite.

By now we have:

  • Bookmarks
  • Change Case
  • Autocorrection
  • Thesaurus with wordnet integration
  • Document statistics

And because he was done before the end of the Summer of Code he added a new popup widget to select colours from palettes and edit new palettes.

Marijn Kruisselbrink

They said it could not be done. But Marijn did it: fully editable music right inside any KOffice document. Essentially a plugin of the same level of complexity as KOffice's justly famous formula editor. The plugin supports multiple voices, measures, adding and removing of notes of every description, loading and saving in ODF documents, import and export to and from the standard MusicXML format. Marijn started his work already before the Summer of Code, when he was able to show notes in KWord when he visited me in April. Interesting bits are the automatic line wrapping, which is more complex in music than in text.

This plugin more than anything else shows off the power KOffice has gained through the Flake architecture. Marijn is very enthusiastic and will continue working on the music flake: connected notes, lyrics, multi-page music and fallback support for OpenOffice and other ODF-compatible applications are all on his to do.

Oh, and as you can see... it is actually possible to seamlessly develop KOffice on OS X, too.

Pierre Ducroquet

Confession time! Right after the KWord rewrite started, KWord did not support ODF at all anymore. Pierre's work therefore included essentially recreating ODF support, together with his mentor, Sebastian Sauer. He was present at the ODF meeting in Berlin, in the KDAB offices, and has gone from strength to strength. Not too mention that he had to hack stuff into KWord that was not even supported by Qt Scribe like the line-through styles. Heck, even OpenOffice doesn't support this part of the standard! Among the other hurdles he had to jump was a test suite that depended on OpenOffice.org default behaviour that contradicted the ODF standard. Great work, and Pierre, too, has said he will keep working on ODF support in KOffice.

Sven Langkamp

Krita never had the famous marching ants selection visualisation invented by Bill Atkinson for MacPaint. We have always shown our selections as masks, following my philosophy that there is not that much difference anyway. Besides, I did not know how to do marching ants. Only marching ants is perhaps a little meager for a whole summer of code project, so Sven took on the whole selection-is-a-bitmap thing.

Last year, Emanuele created a bezier curve framework for Krita that led to magnetic selections and bezier curve selections: however, those curves were mercilessly converted to pixels. Given the hard work Jan Hambrechts and Thorsten Zachmann have put into the flake vector path shape, it seemed silly not to try and re-use that in Krita.

Krita's selections now consist of two components: a masks that allows gradations of selection and editable paths that allow editing of the path shape at all times. The combined selection can be visualised as a mask, as marching ants, both or not at all. Sure, there's still a lot of work to do, and I managed to break some of his work when working on nodes, layers and mask — my bad! — but Sven has advanced Krita 2.0 in a really important way.

A selection that consists of vectors:

And one that consists of a pixel mask:

In Other Words...

Like last year, the KOffice hackers have been very lucky with the students who offered to work on our project. The interaction has been great, with students coming to the ODF hack sprint and Akademy. KWord now has three people regularly working on it instead of just one. That said, there is still plenty of space in KOffice for aspiring hackers who want to cut their teeth on a challenging, visible and fun project. If you feel that there should be a presentation tool more fun than Keynote, a diagram application cooler than Omnigraffle, a chart app that makes statistics blush when they lie — or if you think file formats are really cool, then do not wait for next year. Join us now!

Comments

This is one of the most exciting stories I have seen on the dot in a long time. Congratulations to all, and may fame and glory come to you. :)


By KDE User at Mon, 2007/08/27 - 5:00am

Hej, I fully agree, that was awsome what I just read about the SOC-students.It is impressive, what they have done. Mixing colours (without a big computer cluster) or having music notes in ODF, that is great.
You all did a good job. To the students and all the other KDE programmers out there: "Keep up the good work"


By baumhaus at Tue, 2007/08/28 - 5:00am

Dang, those guys are GOOD! Really impressive accomplishments, ALL of them. (I usually don't use caps, but today I am making an exception.) Now, could Google please hire these guys as KDE hackers full time please?


By Martin at Mon, 2007/08/27 - 5:00am

Damn, aren't those guys a little bit _too_ good? I was thinking about applying next year, but seeing these results, I've realized that it's just far beyond my (currently non-existent) ability. ;)

No seriously; great work, all of you! The results are truly amazing, looking forward to trying it out myself.

Off topic question: Is it possible to run KOffice2 on Windows too? (Seeing that Mac OS X screenshot, I guess the answer is yes?)


By Hans Chen at Tue, 2007/08/28 - 5:00am

I'm sure you can compete: the gsoc is enormous learning experience for all involved. I bet some of our students didn't know they'd be able to get this far. Personally, I've seen proposals for funded research that were less ambitious than some KDE gsoc projects.

Oh, and about windows: Adrian Page has been committing fixes to get KOffice compiled with Visual C++, so I guess the answer is yes.


By Boudewijn at Tue, 2007/08/28 - 5:00am

good job.

About that "sheet music"; can you also *play* the music somehow (e.g. simple piano midi instrument), or is it "read only"? (yes I know Koffice is not a music composer program...)


By Darkelve at Tue, 2007/08/28 - 5:00am

well, you may need an instrument to play, and of course you need to actually _read_ what's written on the sheet. (sorry.. couldn't resist)


By thomas at Tue, 2007/08/28 - 5:00am

That sounds like a job for Rosegarden to me, not KOffice. However, maybe it's possible to interface with Rosegarden for this purpose?


By Andre at Tue, 2007/08/28 - 5:00am

Maybe phonon can play a major advanteg in stuff like that. It can be integrated quite easy into your existing code.
If koffice could actually READ the notes, it shouldn't be such a big problem. I like the idea that koffice can play the song you made up. I hope it will be implemented.


By baumhaus at Tue, 2007/08/28 - 5:00am

The ability to import snippets of music from something like Rosegarden would be useful. Whether that is better implemented as a KOffice import or a Rosegarden export, I'm not sure.

Does it spell-check your musical score, putting red squiggly lines under out of key notes?


By Steven at Wed, 2007/08/29 - 5:00am

"Does it spell-check your musical score, putting red squiggly lines under out of key notes?"

I don't think there is a way for a computer to know whether a score is 'right' or 'wrong'... or were you joking? :|


By Darkelve at Wed, 2007/08/29 - 5:00am

Rosegarden can do MusicXML... I guess that would make interfacing easier?

Basically just a play button and perhaps an instrument chooser would suffice.


By Darkelve at Wed, 2007/08/29 - 5:00am

very nice work ! thanks !

i personally like the color mixing stuff most - i will never use it, but i like the complexity behind the mathematical model...


By jumpingcube at Tue, 2007/08/28 - 5:00am

Is ODF not the default file format for KOffice anymore? How is sheet music compatible with ODF in any way, shape, or form? And what do you mean "he had to hack stuff into KWord" because "KWord did not support ODF at all anymore"?! I thought all open source word processors were agreeing that ODF was THE standard to work with from now on! Aren't you essentially adding "proprietary features" that lock users into KWord, or at least force alternative editors like OpenOffice and AbiWord to play catch-up by implementing sheet music support?

Is ODF is some kind of joke, like OpenXML? I tried once to open ODF documents I made in OpenOffice in KWord, and they didn't render correctly. (Whereas if I saved them in Microsoft format, they would!) On the other hand, if OpenOffice is incorrectly supporting the ODF standard (as asserted in your article), how will that change in future releases of OpenOffice? Will my files suddenly change their appearance when/if OpenOffice begins interpreting them according to the standard? Don't take this the wrong way - I appreciate the length that Sun is going to to make ODF a meaningful specification (unlike Microsoft's CrapXML format), and I appreciate all the work done on OpenOffice and KOffice, but when are they going to produce documents that are interpreted by each other the same way? Or is this going to be like the CSS compatibility wars between IE, the W3C specs, and the other browsers?


By kwilliam at Tue, 2007/08/28 - 5:00am

I think he simply means that the major work in rewriting KWord for Version 2.0 broke a lot of things, including saving in the default file format, ODF. THe have not fixed it so that when 2.0 is released, it will work.

At least I hope that is what he means...


By David at Tue, 2007/08/28 - 5:00am

I think he simply means that the major work in rewriting KWord for Version 2.0 broke a lot of things, including saving in the default file format, ODF. The have now fixed it so that when 2.0 is released, it will work correctly.

At least I hope that is what he means... ODF definitely should be a standard across office suites, especially FLOSS ones.


By David at Tue, 2007/08/28 - 5:00am

I think he simply means that the major work in rewriting KWord for Version 2.0 broke a lot of things, including saving in the default file format, ODF. They have now fixed it so that when 2.0 is released, it will work correctly.

At least I hope that is what he means... ODF definitely should be a standard across office suites, especially FLOSS ones.


By David at Tue, 2007/08/28 - 5:00am

There simply wasn't any file load and save code anymore in KWord at one point in time. KWord didn't support anything at that moment -- which is quite a normal occurrence in the middle of software development. And even in 1.6 there were parts of the ODF unsupported. So when porting of the 1.6 ODF loading and saving code started, bits have been added to support ODF more fully.


By Boudewijn Rempt at Tue, 2007/08/28 - 5:00am

No, ODF is the default file format for KWord.
The problem is just that there wasn't a lot of things working about file loading, and nothing for file saving.
Now, there are problems on both sides : OpenOffice mis-implementing the standard, and KOffice mis-implementing... That's quite an issue, I agree.
Moreover, implementing the whole specification is near of impossible.
I hope both OpenOffice and KOffice reach a near 100% implementation in the future, but that'll take time.
For the sheet music, well... it has to be KOffice specific, there is really no way for it to be compatible with ODF since ODF doesn't specify this.
But you can still build valid documents that way. Here is a part of the saving code :
writer.startElement("music:shape");
writer.addAttribute("xmlns:music", "http://www.koffice.org/music");
Of course, OpenOffice won't be able to interpret this.


By Pinaraf at Tue, 2007/08/28 - 5:00am

Why is it not possible to implement the complete specification? I understand why it is impossible for OOXML (quite too long) but ODF should be short enough to implement every feature (at least in a good time)
And why doesn't OOo and KOffice work a little closer together to implement the same stuff? That would be a little better for the user that he can exchange documents between those two without having problems.


By baumhaus at Tue, 2007/08/28 - 5:00am

I think it's possible to implement all of ODF -- but it'll take some time. And, yes, there is cooperation: KOffice people will go to the OpenOffice week in Barcelona in September. But, well, OO.org is sometimes a little arrogant and assumes that what OO.org does is, by definition ODF, regardless of the actual spec. And we try to be very faithful to the spec, but we're also rather more innovative than OO.org, which means that we're following the spec, but doing things with it that haven't been tried before.

(Plus, we're really short-handed. Don't forget that only Jaroslow, the Kexi maintainer, is paid for his work on KOffice: all the other code is writting by us in our copious spare time. So... if you fancy an easy way into KOffice hacking, come help us out with implementing more and more of the ODF standard. I promise you, it's not hard, and it's a very rewarding thing to do.)


By Boudewijn Rempt at Tue, 2007/08/28 - 5:00am

even thos I would like help, I don't think that I have, what it takes to be a GOOD programmer. I allways wanted to get started in a team but I don't think that I'm any good.
That for I'm really impressed what you are doing. I have alot of faith in KOffice2


By baumhaus at Tue, 2007/08/28 - 5:00am

It's really just a matter of starting... You'll be amazed at what you can do.


By Boudewijn Rempt at Wed, 2007/08/29 - 5:00am

You can get started with quite small work and do relatively easy things. Getting to the stage where you can build KOffice from scratch is a difficult first step but it gets easy from there. The Koffice developers were very good about reviewing the few small patches I sent in and were very constructive about it all.


By Alan at Wed, 2007/08/29 - 5:00am

Well, it's really really hard to implement 100% of any specification. It *is* impossible for OOXML (far too big + some undocumented things), and for ODF it's just hard and needs time.
(BTW, a full ODF implementation requires Java since you're supposed to be able to embed Java applets in an ODF document)


By Pinaraf at Tue, 2007/08/28 - 5:00am

Now this is strange...
Is Java an ISO standard?


By Luciano at Tue, 2007/08/28 - 5:00am

No it's not.


By Cyrille Berger at Wed, 2007/08/29 - 5:00am

Supporting java applets should be extremely easy since way java support is implemented in khtml means applets can be embedded by simply using KJavaAppletWidget.


By Richard Moore at Tue, 2007/08/28 - 5:00am

btw, a 100% implementation would also require OLE (9.3.3), DDE (4.4.3, 6.6.9, 8.10), binary plugins (9.3.5), multiple scripting-languages (or at least all that got used by competitors, 2.5), multiple database-drivers (or at least all that got used by competitors, 6.5, 8.6.2), would need to even know about custom app- (2.4) and meta-data (3.3), macros (6.6.7), Change Tracking (8.10), would need a audio-/soundplayer (9.7.1), etc. and that's just a quick look at the table of content ;)


By Sebastian Sauer at Tue, 2007/08/28 - 5:00am

Kudos to all the SoC folk!


By Eugeniu at Tue, 2007/08/28 - 5:00am

Oh no! Look at the "statistics" screenshot. Kerning is as bad as it always has been. The developers used to promise that this would be fixed with Qt 4; more recently they have been pointing to the fonts. Please, show just one rendered PDF or even a screenshot where the text really looks good and prove me wrong! Because so far, sadly, and I am not trolling here, but speaking from experience, I have never seen acceptable output from KWord: any version or font family.


By Martin at Tue, 2007/08/28 - 5:00am

Forgive my ignorance, but what does the kernel have to do with fonts?


By KDE User at Tue, 2007/08/28 - 5:00am

Well, in Windows, the truetype bytecode interpreter is part of the kernel, but that's not what the original poster meant. He means the way letter shapes (glyphs) are slightly moved about to create a pleasing result. Like a lower-case o being shifted a little bit to fit under a capital V in the the sequence "Vo". That's what's called kerning. Whether kerning works or not is dependent on the font -- if the included kerning tables suck, then the result will suck, and on a couple of other things, including Qt Scribe and Freetype.

Why exactly the kerning is off sometimes, with some fonts when using Freetype and Qt Scribe is a bit of a mystery to me. I've seen great results with KWord 2.0 myself, but so I've seen great result with KWord 1.6. But I haven't investigated myself, being busy with Krita, and I'm sure that Thomas Zander is hot on the trail of the remaining font problems.


By Boudewijn Rempt at Tue, 2007/08/28 - 5:00am

Ok, I investigated a bit. Apparently there's currently a bug in Qt that makes KWord obey the hinting settings. If you set hinting to anything but None (as you should anyway, hinting is evil), your text will be really bad. Another issue is with fonts: some fonts just have extremely bad kerning tables. Look at the attached image to see some very good lines of text in KWord, and some less good ones: Sans Serif and Arial are pretty bad, Georgia fair to middling. Times New Roman, Perpetua, Zapf Chancery, Bembo and Garamond are really pretty good. But then, those are expensive, professional fonts.

Actually, I think Bembo has never looked as good on screen as it does in KWord trunk.


By Boudewijn Rempt at Tue, 2007/08/28 - 5:00am

Thank you for a most awesome explanation and illustration!


By KDE User at Tue, 2007/08/28 - 5:00am

Looks like only the windows fonts look really good (I'm looking at the 'u' in "Tussen"). It looks good in Arial, Garamond, Times New Roman, but not in the the other linux-specific? fonts...

Well if it works, at least with the windows fonts, than it is okay. Although I hope ofcourse those other fonts will get better over time....


By boemer at Tue, 2007/08/28 - 5:00am

Actually, only the first font ("sans serif") is a free font. The rest are mac/windows commercial fonts. Pretty expensive ones, too. And I'm so fond of Bembo...


By Boudewijn at Tue, 2007/08/28 - 5:00am

IMHO:

Garamond: The "r" is too strong -- it stands out in "typographie" and "kerning".

Bembo: Bad kerning in in "Waar" and "Tussen".

Perpetua: Bad kerning in "voorbeeld"


By Niels at Tue, 2007/08/28 - 5:00am

Aww, the character positioning looks really bad for all of the fonts in that screenshot.

The problem is that Qt (being based on the OS's native font system) does only support integral character positions, making it impossible to get correct layout (line lengths etc.) and correct positioning at the same time.

Please see http://antigrain.com/research/font_rasterization/index.html for some awesome results that are possible using FreeType directly, bypassing the OS's font system.

Especially worth noting is this screenshot (yes the font is small, but this even more so proves the author's point).


By christoph at Wed, 2007/08/29 - 5:00am

Given that there's no word processing software yet that works the way the author of that article proposes, it's a bit silly to hold it up as the standard. Compared to the same text in OO.org or Abiword, KOffice 2.0 is doing really well.


By Boudewijn Rempt at Wed, 2007/08/29 - 5:00am

Still, you have to admit that the results look pretty amazing. At least to my rather untrained eye.


By Andre at Wed, 2007/08/29 - 5:00am

> The problem is that Qt (being based on the OS's native font system) does
> only support integral character positions, making it impossible to get
> correct layout (line lengths etc.) and correct positioning at the same time.

Qt's font system uses qreal (doubles) in the API everywhere and maps that to a high-res grid of integers to keep things fast. There is virtually no loss there so the statement above doesn't really apply to the KWord case. Sorry.


By Thomas Zander at Wed, 2007/08/29 - 5:00am

You are right about the API and internal calculations using qreals, but the actual rendering is not done in sub-pixel positioned precision.

Try rendering a couple of letters each moved by a fractional pixel coordinate, and you will see that they all end up looking the same.

Compare that to KPDF's rendering (3.5.7, didn't try 3.91 yet), where the bitmap of a letter is different depending on (sub-pixel) position.

Of course it also has to do with Qt (or FreeType?) trying very hard to cache the bitmaps. In earlier versions of Qt/FreeType, you could bypass caching by scaling with, say, a factor of 1.00001, and get very good results. But in recent versions, the cache is used even for rotated fonts, making the result look really funny.

And seeing the good quality of KPDF and its good speed, I doubt that a Word Processor couldn't get nearly the same results.


By christoph at Thu, 2007/08/30 - 5:00am

Sorry maybe I'm blind but I see no problems at all with kerning in Boudewijn's screenshot


By Vide at Fri, 2007/08/31 - 5:00am

That's not only Qt...
I just upgraded my (free) ATI driver to the latest git to get xrandr 1.2 support, and all the font rendering is now far far better everywhere. For instance in KWord the text underline was in the middle of the text, now its position is good. So be careful with this font thing...


By Pinaraf at Tue, 2007/08/28 - 5:00am

yes, very right. I said to Boud on IRC its likely in the 'lower layers', it ended up on his post above as 'in Qt'. Almost right :)

There are actually various shared libs being used by a large set of font rendering stuff on Linux. This are all rapidly improving since not only KWord, but all of Qt, Pango (for gtk stuff) and IIRC also firefox are sharing that lower level stuff.

KWord is not released yet, and we are all working hard on fixing all the things that need to be fixed to make KOffice have the most beautiful and versatile text layouting and rendering engine for you.

So, for the guy that went slightly off topic on this GSoC story; this is alpha stage software and we are aware of such issues. Working on it! :)


By Thomas Zander at Tue, 2007/08/28 - 5:00am

For KWord 1.6 it's rather simple to explain the difference between fonts. Type-1 fonts are fine while Truetype fonts are broken. So actually, when I use KWord I only use Type 1 fonts (like e.g. Bitstream Charter or Adobe Utopia).

Further, in KWord 1.6 I perceive kerning problems only in printout. DejaVu Sans e.g. looks better on screen in KWord 1.6 than in the screenshot of KWord 2.0. Problems begin only when you print a text or create a PDF.

I really hope that this will be fixed in KWord 2.0. Most fonts are Truetype fonts nowadays. Without a proper kerning of these fonts KWord 2.0 will be unusable for most people.

Another problem is support of OTF fonts (i.e. Type-1 based Open Type fonts). In KWord 1.6 they are rendered correctly on screen, but replaced by other fonts in PDF and printout. BTW, OpenOffice.org doesn't support them either on screen nor in printout or PDF. These fonts, too, become more and more common. So, complete support of OTF fonts is also important.


By Stefan Lücking at Tue, 2007/08/28 - 5:00am

Yep, correct support of TrueType and OpenType is highly desirable, especially for Windows versions of KOffice.


By Richard Van Den Boom at Tue, 2007/08/28 - 5:00am

thank you

ps= great work guys!!!


By [OT] When will ... at Tue, 2007/08/28 - 5:00am

Of KOffice? No idea; we are at around alpha 3 now :)


By Thomas Zander at Wed, 2007/08/29 - 5:00am

Pages