JUL
10
2006

Sebastian Sauer Talks About Scripting with Kross

KOffice 1.5 saw the addition of Kross, a framework to allow for scripting plugins in a number of languages. Krita and Kexi come with a number of plugins with more available for download at KDE-Files.org. To find out more about this intriguing technology and how it came about KDE Dot News interviewed the author Sebastian Sauer. Read on to discover how you can use Kross.


Sebastian Sauer, author of Kross

What is Kross?

Kross is the scripting engine for KOffice and provides a complete framework to embed scripting interpreters transparently into native applications and bridge the static and dynamic worlds together.

KDE does provide a rich set of bindings for different languages. PyQt/PyKDE, QtRuby/Korundum and even developing KDE-related tasks with Java and C# is supported.

If it comes to embedded scripting the choice gets cut down to QSA and KjsEmbed which both provide an embedded JavaScript-interpreter with a very small subset of the functionality that existing modules and 3rd-party supported languages like Python or Ruby are enjoying. There is no CPAN-like collection of 3rd party modules for nearly anything, maintained and improved by large communities and reusable for your own tasks.

The open source world is about choices, about reusing functionality and about flexibility. So, why not increase the number of supported languages which could be embedded into an application? Well, writing the code to embed the Ruby-interpreter into your own application, writing more code to add Python, LUA or Perl support and then write bindings for your applications internals for each single supported language would be the requirement and that's only for one single application because every application provides own bindings, the developers prefer one interpreter over another and at the end we would reinvent the wheel for each single application that likes to embed the interpreter the developer likes most.

What is needed, is a transparent and scripting-interpreter independent way to deal with embedded scripting. That is where Kross comes into the game.

First it is transparent because the application that uses Kross doesn't need to know what scripting-language is used at all. All the application needs to know is, that there exists a library which provides scripting-functionality. Well, in fact the application doesn't need to know that either because Kross can be provided and used within an application-plugin (that's how Krita and Kexi include Kross) or maybe just as a KPart if that's what the application prefers.

All the application needs is for the loaded plugin to pass object instances to it. The plugin and Kross will know how to handle those instances with some - again interpreter-independant - binding-code the application provides. Such bindings are implement the interfaces that should be exposed to Kross and Kross itself will then take care of the scripting-backend.

In fact even the core of Kross doesn't know which interpreter finally evaluates the scripting code or how the evaluation is done. All this is handled by interpreter-plugins loaded dynamically at runtime on demand. That means, we have a strict isolation of the interpreter-dependant code and the application itself does not depend on any interpreter! Even better, the interpreter-related code could be packaged separately and only installed if it's really needed/wanted by the enduser. That leaves us enough room to embed for example Java, Mono, LUA or Prolog if someday someone thinks that his application needs it we could introduce it without any prior dependency. Even better, all applications that use Kross could from then on use the new interpreter too and all that without one single line of touched code in the application or its bindings and even without any recompile.

Where is Kross in use today?

Currently it is in use within KOffice. To be more exact, Kexi and Krita as shipped with KOffice 1.5 provide full Python and Ruby scripting. KSpread supports scripting with the hopefully soon arising KOffice 2.0 and support for other KOffice-applications like KWord is planned.

What languages does Kross support?

Python and Ruby are fully integrated and support for Kjs/KjsEmbed and Java are planned. Both of the currently supported interpreters implement the bridge with just a small handful of files (about 5 per interpreter if I remember correctly :-). The main work is done within the Kross-core the application and the interpreters are using to talk with each other. Well, in theory it shouldn't be that difficult to even let one interpreter talk with another interpreter that way.

It's even better. We don't reinvented the wheel rather we provide access to already existing technologies. So, you are even able to use the whole power of PyQt, PyKDE, QtRuby and Korundum in your native KDE-application. You are able to build complex GUI's that way, include them at runtime dynamically as part of your application, connect signals and slots, trade object-instances, manipulate your KApp, access DCOP, QWidget's or whatever you want to archive. For example in Kexi we decided that we don't want to depend on PyQt to display a simple GUI for a small ~100 line "Export Database-table or -query to XHTML File" python-script. So, we wrote an additional ~100 lines of python-script to fallback to TkInter if PyQt is not installed and guess what; it works great. In theory even PyGtk should work (I never tried it), which means we are able to access Gtk at runtime without depending on it.

Could you provide an example of usage?

As an example Kexi already ships with a bunch of scripts that extend the functionality of the application. We also provide a "Get New Scripts" functionality to install or update script-packages over the internet.

At KDE-Files.org we offer a complete webserver for Kexi that runs within its own thread in the main native application and we offer the CopyCenter which uses some of the great external modules Python provides to be able to merge data from/with a Kexi database to e.g. a QtSQL-database or an Excel-compatible CSV-file. The scripts which were written recently outline what is already possible today.

How can a developer add Kross support to their program?

There was a great tutorial written by Cyrille Berger. For some weeks we have managed to hide all the Kross-internals by introducing some template-magic. Let's look at a simple example (see also the trunk/koffice/kspread/scripting code for a more detailed example):

// We like to wrap that class and publish the myFactoryMethod-method to scripting languages.
class MyNativeClass
{
 public:
 MyNativeClass* myFactoryMethod(const QString& name) {...}
}

// So, here is the wrapper class
class MyWrapperClass : Kross::Api::Class
{
 public:
 MyWrapperClass(MyNativeClass* instance)
 {
  // just publish the method above and say who should handle
  // the arguments by defining them as template-arguments.
  this->addFunction1
  ("myFactoryMethodName", instance, &MyNativeClass::myFactoryMethod);
 }
}

that is it already... we are now able to call the factory-method from within scripting code and will get an instance of MyWrapperClass as result. All kind of types could be wrapped that way dynamically and we are even able to delegate method calls dynamic that way :)

What is the advantage of Kross over DBUS, KJSEmbed and Kommander?

Well, DBUS is a great piece of work, but its main area is IPC/RPC (Inter-process and remote-process) while KJSEmbed and Kross are about embedding scripting direct into an application as part of it. That way scripts are running in the address-space of an application and are able to manipulate the application.

KJSEmbed is heavily bound to KJS (KDE JavaScript) and does not allow you to integrate any other scripting language. Users need to use Javascript or don't use scripting at all. It is not possible to use additional existing solutions like PyQt, QtRuby, Java or LUA.

Kommander is a damn great tool and allows easy integration with a designer built GUIs and uses DCOP (now replaced with DBUS in KDE 4). It comes very close to what I would expect a flexible scripting solution should offer. Maybe someday we could integrate Kross into it as well and extend the great user-interface it has with a flexible and fast transparent in-process scripting solution.

Why are Kross scripts better than writing plugins directly in the program?

That is a difficult question because I guess there is no correct answer beside "it's up to the user if he prefers to use C++ or e.g. python".

What security concerns are there with Kross scripts?

That is one of the biggest problem we have currently. Python does not deal with insecure, untrusted scripts and does not provide any sandbox-like execution-environment. While Ruby supports security-levels and while I integrated the great Zope RestrictedPython environment into Kross, I would not trust them to be as safe as a Java sandbox could be. We could try to limit the damage by using signed scripts as outlined on the Kexi wiki Scripting page, but that doesn't look like the most powerful and secure solution. So, currently I don't see another solution than using Java for untrusted scripts or extending KJSEmbed4 to be able to deal with insecure scripts.

Will KOffice get macro scripting ability?

That is another task we are working on. There is a team from within my university which last year worked on integration of macros for Kexi. At this point I would like to thank Hr. Christoph Knabe from the TFH-Berlin a lot for his great support and for allowing us to work on the macro-front. The current state is that it's very alpha and it's not really ready yet to be used widely. So, maybe next year we could talk more detailed about this :)

Will Kross be used outside KOffice?

I hope so, but I guess each developer has to decide on his own if it makes sense and if their users would prefer a language-independent scripting solution. At the end its all about having the choice rather then being limited.

How did you come up with the idea for Kross?

Let me fix the "why this guy had this idea and why he chose that ugly K-name" impression some may have got. When time I joined Kexi and started to look for a scripting-solution we could integrate I found the Kexi wiki-page which looked quite difference then. In fact someone - Cedric Pasteur or Lucijan Busch? - had the idea to integrate an interpreter-independent solution and wrote about it at the wiki and the working-title was "Kross". So, what I did was to absolutely agree with what I read there and just started to implement it :-)

How hard was Kross to implement?

It is hard and it will stay hard. Mostly because I like to do it as good as possible. Everytime other developers ask me if the thing is ready to be integrated I answered "Not really, because my personal feeling is that it's still very alpha and the API may change a lot". Finally there was one developer who didn't agreed with my usual statement and just started to use it and that was absolutely the correct decision cause since that time the thing went from a moving developer-target into a very stable usable solution. So, a lot of thanks to Cyrille Berger!

Is there something like Kross on the MS Windows platform? Is Kross' scope similar to Visual Basic for Applications (VBA)?

VBA is also just one single scripting backend. Kross was designed with the concept in mind to provide an abstraction layers to interpreters and hide the differences to the application that likes to use scripting. So, it follows more the .net idea without the large codebase, patents or a vendor-lock. It's all about reusing already existing technologies and providing access to them without introducing any dependencies.

Is Kross basically finished, or is development and feature expansion still in full swing?

Well, as I said before I stopped saying that it's a moving target cause that may scare potential users away. But there is still a lot to do. For example we like want integrate KJS/KJSEmbed and Java into it. Another goal is to improve the speed by using C++ templates internally everywhere and to have fully transparently DBUS-integration. Also I hope to find some day some more time to look at how we could connect with the great Kommander.

Where can we find out more?

http://kross.dipe.org

If there are any questions, you are also able to reach me direct via mail-at-dipe.org or on #koffice on Freenode IRC. My no-longer secret nickname there is "dipesh" :)

Comments

One area where Kross can really help is automatic spreadsheets and generating graphs based on those spreadsheets. Everyone likes graphs since they make presentations look real spiffy and the presenters look real smart ;)

Jokes aside, lots of diverse fields such as accounting, marketing, biology, engineering deal with huge amounts of numerical data that is stored in spreadsheets. The people that analyze this data are most often *not* programmers so an easy scripting framework would make their lives a whole lot easier. The real advantage would be bringing KDE into the offices of all these professionals.

While M$ may not excel at many things, it has in my opinion done a good job with Visual Basic for Applications which brings together their whole office suite and makes it easily programmable. I suggest KOffice and Kross devs take a look at VB for Apps as a successful model.


By AC at Mon, 2006/07/10 - 5:00am

Kross support will be added to kspread in the upcoming 1.6 release of KOffice (thanks to Isaac), currently it's limited to accessing the data in the spreadsheet. And our plan is to let our user tell us if they need anything else. In that area too, we follow the general KOffice policy, lets users tell us what they need, and do not try to implements every possible features ;)

So if anyone feels limited by what we offer, then just tell us :)


By Cyrille Berger at Mon, 2006/07/10 - 5:00am

With Korundum, could a Kross script actually embed Qt Widgets into an existing application? Or can you only mutate existing QObjects?


By Ian Monroe at Mon, 2006/07/10 - 5:00am

Hi Ian,

yes, that is possible. You are able to use QtRuby and PyQt to embed widgets, manipulate your native app, etc. The WebServer-script written for Kexi as example ( http://www.kde-files.org/content/show.php?content=36246 ) e.g. does connect with Kexi's aboutToQuit()-signal to shutdown the WebServer-threads if Kexi quits (else the running threads would block Kexi from quitting).


By Sebastian Sauer at Tue, 2006/07/11 - 5:00am

I'm very interested in using Kross in Amarok 2.0. This seems like exactly the technology we want for implementing a plugin system, to replace or complement our existing scripting facilities.

My question: How can you implement callbacks with Kross, that is, invoking methods from the script when certain events occur in the application?


By Mark Kretschmann at Mon, 2006/07/10 - 5:00am

The answer is short : we can't yet :) but that something we want to do.


By Cyrille Berger at Mon, 2006/07/10 - 5:00am

I hate to sound like a simpleton,
but isn't this what signals/slots is supposed to do?

Wouldn't it be possible to make the event trigger a signal in the backend and then all of the connected slots in the script get called? I got the impression that QtRuby could do this... (but I am not an expert)

Cheers,
Ben


By Ben Schleimer at Mon, 2006/07/10 - 5:00am

It would be possible to make the event trigger a signal in the backend and call the slots in the script, but it might get bit tricky.

For instance, here's what I'm doing in Korundum/QtRuby to know when a KPresenter presentation has finished:
factory = KDE::LibLoader.self().factory("libkpresenterpart")
@kpresenter = factory.create(w, nil, "KParts::ReadOnlyPart")
@kpresenter.openURL(KDE::URL.new(file))
connect( @kpresenter.widget().children[0], SIGNAL('presentationFinished()'), self, SIGNAL('itemFinished()') )

Look at that: @kpresenter.widget().children[0]. Thanks to Qt introspection I can get to the presentationFinished() signal, but it's quite contrived. It works flawlessly, though.

If the plugin just displays contents and needs no interactivity, another possibility is to use the winId trick: the backend would be an empty QWidget and we then tell the plugin you developed with Kross/QtRuby/PyQt/whatever to display its contents in the window with the winId of the aforementioned QWidget. It works but you get no events, therefore is a very limited approach.


By Pau Garcia i Quiles at Tue, 2006/07/11 - 5:00am

I was told that Novell develops a VBA interpreter?


By pentamax at Mon, 2006/07/10 - 5:00am

Well, tell us more :o)


By rinse at Mon, 2006/07/10 - 5:00am

Nobody said Novells oferring is open source or free. Same for SUN's offering.


By anon at Tue, 2006/07/11 - 5:00am

Once again, you have to note that we don't aim to be fully compatible with Microsoft Office, and therefore we have no intention to have an API identical to VBA. So, supporting Basic is not planned, it's really far to be a good language even for beginners.


By Cyrille Berger at Wed, 2006/07/12 - 5:00am

Will it be possible to host Kross in a python/ruby, so that developers who prefer other languages can also write plugins for a pykde or korundum app? And... if so, would there be any benefit with Kross for a pykde plugin to a pykde app, such as plugin management or cleanness?


By Lee at Tue, 2006/07/11 - 5:00am

Interesting question (that's why I needed some days to think of it). Well, nothing is impossible, but currently it's not planned. A benefit could be maybe to be able to use other scripting-languages from your favorite scripting-language through Kross. Plugin-management is something that is already very easy in both; python and ruby. So, I would not see there any benefit. Same goes for cleanness (or at least I fail to see the advantage here to e.g. use ruby=>kross=>ruby rather then ruby=>ruby direct).


By Sebastian Sauer at Wed, 2006/07/12 - 5:00am

Kross looks like a very slick and useful kde tool.

However, I'm concerned that it will expose too much functionality to the random user and cause easy exploits (ala activeX/VBA). Are there plans for a sandbox api for the embedding programs to use?

Thanks,
Ben


By Ben Schleimer at Mon, 2006/07/17 - 5:00am