There has been some recent buzz around KDE's Get Hot New Stuff framework. As the first in a series looking into KDE technologies, KDE Dot News interviewed author Josef Spillner to find out what all this "stuff" was about... read on for the interview. You may also be interested in recent blog entries about KNewStuff: Kate, desktop backgrounds, Quanta, KNewStuffSecure, its user interface design and the HotStuff server setup.
Josef Spillner
Please introduce yourself and your role in KDE.
Hi, I'm Josef Spillner, a computer sciences student from Dresden in Germany.
About 6 years ago I saw KDE for the first time, and some months afterwards
the first application written by myself appeared on my desktop. It was a simple
KTMainWindow which controlled a voxel-space flight simulation on top of kdesktop.
Needless to say it was horribly slow. But I've been writing KDE games ever since.
What is Get Hot New Stuff and KNewStuff?
The GHNS concept describes a way to let users share their digital creations.
For example, user A is using a spreadsheet application and modifies a template
which comes with it. This template can then be uploaded to a server, and
eventually be downloaded by user B by checking the contents of the
"Get Hot New Stuff" download dialogue. In the context of companies, documents
can be distributed to all employees, and in the context of the internet,
a community sharing framework is built on top of all this.
The KNewStuff library is the KDE implementation for checking which file providers
support the requested data type, and which files are available, including their
version, popularity and preview information. Files can be up- and downloaded,
digitally signed, uncompressed on the fly and more.
Where are they in use today?
Inside KDE CVS, we have Quanta+, KOrganizer, several Edutainment apps and the
desktop configuration as our patient and proud users.
Outside of KDE CVS, a bunch of games of the GGZ Gaming Zone project uses
KNewStuff to keep levels and themes current, and the move of the library to
kdelibs should encourage more projects to follow this example.
Can you give us a brief technical overview of how GHNS works?
Each application can decide for itself whether it wants upload or download or
both of them, and which file providers should be used for these tasks. The
providers can be configured on the server side so a move or scheduled outage won't
harm any users.
A download task would check for all the available files for each provider,
and compare their versions with the locally installed ones, which appears as
green (already installed) or yellow (installed but can be updated) signs to
the user.
After the installation, a post-install script might be run, which can of course
include a DCOP call so the application can be notified. There are ready-to-run
KNewStuff classes available as part of kdelibs (KNewStuffGeneric and KNewStuffSecure),
but for more complex tasks it is possible to subclass KNewStuff.
Where did you come up with the idea for GHNS?
I didn't :-)
To be fair, I first implemented level sharing in 2001 or so, but the direct
ancestors to the current library were the "Hot New Stuff" download in KOrganizer and
the KDEShare library in kdenonbeta, both of which were inspired by Torsten Rahn
of kde-artist fame, at LinuxTag 2002.
Most of the integration work was done at the KDE Kastle conference one year later.
Where would you like to see GHNS used?
One candidate would be KOffice. Another one suggested already is KDevelop with its
templates, which can easily get out of date on the target systems.
And games. All KDE games should be extensible and be flexible enough to handle
data added at runtime.
Has there been any interest from other desktops to implement GHNS?
Sporadically yes, but I'm not yet aware of any actual implementation. It would really
increase the acceptance and the usefulness - think about an artist whipping up a nice
design in Gimp, uploading it, and voilà the users getting it onto the desktops.
There is however a (highly configurable) SDL implementation written in Python,
and a custom in-game download dialogue patch written in C. Using a decent XML library
makes it easy to add other GUI frontends, but this has yet to be done.
Have you considered hosting on freedesktop.org?
Yes I have. The legitimation of freedesktop.org standards always comes from previous
usage, and with KNewStuff we have a well-working example. So I'd see this as one of
the goals of the near future. The usage in GNOME/Gtk+ will likely depend on such a move.
Do you have any plans for KDE 4?
Sure, the README.knewstuff file is full of them, as is the patches/ directory in
KStuff CVS. First, there are always small nitpicks, like the ability to configure this
or configure that, without breaking the ABI. Second, there should be tighter integration
with the KDE privacy framework. Finally, the management of installed data could be
eased, without however converting KNewStuff into a full package installer.
But there are also others hacking on the library, so more features are to be expected.
Nice name, how long did you spend thinking that up?
The name was also adopted by me, but at least I defended it rigorously :-)
Seriously, there were discussions about how it could affect users negatively, but no
one came up with a better idea, and we managed to hide the name from the users who
do not want to see it.
A nice technical solution for a non-technical problem.
References
KNewStuff API documentation
KNewStuff and
KNewStuffSecure tutorials
GHNS backend reference implementation ("Hotstuff")
"KDE: Conquer your networks", talk given at the 5th Fórum Internacional Software Livre 2004
"The Dynamic Desktop", talk given at the KDE Contributors Conference 2003 "Kastle"
Comments:
Versioning support? - Richard Moore - 2005-03-12
Does KHNS support versioning of the resources? eg. Could I upload a new revision of a resource and have it offer to fetch the update?
Rich.
Re: Versioning support? - Aaron J. Seigo - 2005-03-12
yes, version is one of the supported tags in the XML. it's still up to your app to figure out what "upgrading" means though. as Josef mentioned this aspect of HotNewStuff will likely get extended ...
Re: Versioning support? - Andre Somers - 2005-03-13
This could be a very cool technology to keep some map resources up to date in KFLog. Does the framework supply a method to check for updates to installed items? If it does, we can issue an allert when there are updated maps to be downloaded, so our user allways work with the most recent maps, which is important when dealing with flight planning. I think it could be usefull in other scenarios too.
Re: Versioning support? - Aaron J. Seigo - 2005-03-13
not within the framework itself, no. you could do this by comparing the version numbers within your app itself. this is one of the areas that i think GHNS will get improvements around in coming releases, however.
the more it's used, the better - Aaron J. Seigo - 2005-03-12
this is one of those technologies that can be a HUGE boon to usability. why? well, take wallpapers for instance. it allows a person to install new images from the network by just browsing through a list. this means they don't have to know:
o where in the filesystem these wallpapers belong
o where on the Internet to go to download them
o how to complete all these steps in the right order
suddenly getting new wallpapers is a matter of, well, picking new images. not learning how to use a computer. this is fundamental and exciting.
in KDE4, kicker will be using HotNewStuff for clock themes (looks like a new version of styleclock will be part of kicker in kde4!). i'm considering how to safely allow scripted applets and other buttons to be offered by this mechanism as well. this will likely mean using the GPG security features to validate them. however it works, this will allow the kicker hackers to offer a lot more applets and neat fringe features without having to ship them ALL with kicker while keeping them within a few clicks for users.
oh, and Kopete just got support for HotNewStuff to download emoticon themes. it's already in CVS! chat window styles are coming, too.
the possibilities are immense here. and the more places from which it is available for people to use it from, the more they will look for and use it. let's see how many KDE apps, both in KDE's CVS as well as on kde-apps.org, we can get working with HotNewStuff!
Re: the more it's used, the better - Tormak - 2005-03-12
Will this be integrated into kde-look.org so that uploaded backgrounds, themes, etc. can then be downloaded over this framework?
Re: the more it's used, the better - Aaron J. Seigo - 2005-03-12
it already is integrated with kde-look.org! that's where the wallpapers in 3.4 come from. cool huh?
Re: the more it's used, the better - Tormak - 2005-03-13
Sweet!
Re: the more it's used, the better - Brandybuck - 2005-03-14
There are thousands of wallpapers at kde-look. Clicking on them one by one to see their previews sounds like a major usability problem. Trying to sort by them popularity has its own problems, as kde-look artificially degrades the popularity of older content. While integrating kde-look into the control center sounds cool, I'm worried about the major problems that will surface because of it.
Re: the more it's used, the better - Christian Loose - 2005-03-14
> There are thousands of wallpapers at kde-look. Clicking on them one by one to see their previews sounds like a major usability problem.
You will definitely not be able to browse through all wallpapers at kde-look with GHNS. It will just show the most popular and most recent entries.
Re: the more it's used, the better - Aaron J. Seigo - 2005-03-15
there certainly will be new challenges ahead for us as we explore new concepts and ideas. the most interesting ones will be the ones we can't even foresee at this point. this is where it gets fun IMHO: no longer are we simply trying to catch up with the "state of the art", we are now extending it..
as for the issues of degrading popularity of older content, i is natural, normal and nothing to be concerned about. when it comes to esthetics, be it clothing or music, tastes and popularity ebb and flow like the tide.
however, "most downloaded" is something that older content often has an advantage in. the most downloaded wallpaper is over a year old, the second most downloaded wallpaper is even older.
Re: the more it's used, the better - David Johnson - 2005-03-15
"however, "most downloaded" is something that older content often has an advantage in"
Expect, however, for a strange behavior at kde-look. For some reason all content is slowly degraded in popularity over time. Whether this is due to "trolls" voting down content or a deliberate algorithm (mark something down if no one has voted it up in a certain amount of time), is something I cannot determine. But the behavior is there because I have seen it. Why should one of my wallpapers with already low popularity that hasn't been updated in over a year still get a regular degradation in popularity? With some of my more popular works, I notice that everytime I update them they get a bump in popularity, only to see it degrade over the next months until my next update. Popularity is definitely time related.
If the content happens to get enough popularity to remain on the main "most popular" page then it will most likely remain there. But if it gets bumped to page two or three it will slowly start to degrade.
Re: the more it's used, the better - kundor - 2005-04-16
and like the parent said...it still has the advantage in "most downloaded."
Re: the more it's used, the better - Claire - 2005-03-15
Yep, this was on the top of my wishlist, and I will not only use it myself, but also encourage all my friends to use it. Concerning the searching/browsing, I think it's worth considering a KDE-wide search framework (including the KDE help center and desktop search I read about a while ago) that gives users the opportunity to search
- through all KDE related resources (kde.org, kde-look.org, kde-app.org, kde-wiki, kde.country, kde-forum.org, etc.)
- with criteria to narrow down the search (offline/online, application name, dev docs, user docs, file-type, subject, etc.)
- via a webfrontend on kde.org (e.g. with checkboxes for the criteria, links and description, etc.)
- using a kio-slave/webshortcut for this framework, so any kde-application can acces it and instead of typing the webshortcut "gg:", it would be for example "kiss:" as in KDE's Integrated Search Service ;).
- on your local computer and the web easily (hardly notice a difference) AND securely
Are there any other plans/projects for this kind of KDE desktop/community/websearch integration?
Re: the more it's used, the better - David - 2005-03-13
Another problem people have - sharing wallpapers between each other. Trust me, it's true!
I have to say, this sounds exceptionally cool because it looks as if it could be so damn useful.
Re: the more it's used, the better - Garreth Price - 2005-03-14
KOffice integration would be nice as well. I've been drooling over iWork's Pages and Keynotes templates and have thought about trying my hand at creating some for KOffice.
this sounds sweet - ca - 2005-03-12
It does! But why is it only announced here and not on Slashdot, OSNews, Newsforge, LinuxToday? Or are those only GNOME-centric sites?
This sounds like a great new and innovative idea from KDE.
Re: this sounds sweet - Ian Monroe - 2005-03-12
Well, this isn't an announcement, just an interview. But feel free to submit it to those places. ;)
Re: this sounds sweet - ca - 2005-03-13
So will KDE make an announcement?
Re: this sounds sweet - Aaron J. Seigo - 2005-03-13
stories on theDot usually gets picked up by LinuxToday.com and Linux Weekly News (lwn.net), but they only end up Slashdot and other such sites when someone, often an excited user, submits it to them. feel free to do that =)
personally i think GetHotNewStuff is cool enough to shout it out from the rooftops. that's what got this whole ball rolling in the first place: developers thought this was really cool and started talking about it amongst themselves, now it's percolated out to theDot ... where next?
generally stories about KDE related tech are spread by individual users, developers and (sub-)project rather than centrally from the KDE project itself. the notable exceptions are conferences and release announcements. so... please, help spread the word! =)
Re: this sounds sweet - Aaron J. Seigo - 2005-03-13
indeed! spread the word! =)
Repository? - Gary Cramblitt (aka PhantomsDad) - 2005-03-12
It would be a nice enhancement to use KHotStuff to allow users to upload new filter configurations for KTTSD. However, I don't have a server that can serve as a repository. Is there a public repository that can be used?
Re: Repository? - Sam Weber - 2005-03-13
Looks like kstuff.org provides one :)
http://kstuff.org/hotstuff/
-Sam
Re: Repository? - ca - 2005-03-13
How well is this going to support millions of KDE users?
Re: Repository? - Aaron J. Seigo - 2005-03-13
i don't know about khotstuff.org, but the machine that runs kde-look.org and kde-apps.org is a very modest server and they serve a TON of content. 23 million pages wutg 27,000 registered users and 1.8TB of traffice in a month. so as long as there's the bandwidth, it doesn't seem to take much these days to serve some serious amounts of content.
http://kde-look.org/news/index.php?id=159
Re: Repository? - Navindra Umanee - 2005-03-13
Um. So according to Frank, the KDE-Look.org machine is under a constant load of over 30? I honestly find it hard to believe that a machine under that kind of load would respond the way this one is doing.
Re: Repository? - Macavity - 2005-03-13
Hehehe.. The server capabilities of Linux keeps amazing me :-) HTTP acceleration is the coolest invention since sliced bread!
~Macavit
--
Of all the things I've lost, I miss my mind the most.
Re: Repository? - David - 2005-03-13
It depends what kind of help the machine is getting, and how it is optimised - opimisation, such a loaded word. You can get quite a lot out of a modest machine if you're clever.
Re: Repository? - Luke Chatburn - 2005-03-13
Classic rule of network serving:
"With a good OS and server daemon, you'll almost always run out of bandwidth before the server can't cope anymore."
Re: Repository? - David - 2005-03-13
There's nothing like trying to find out ;).
GHNS essential to KDE-Edu - annma - 2005-03-13
The Get Hot New Stuff library is essential in Educational Software. The kdeedu module was getting really big with data so we moved the data in the corresponding kde-i18n modules mainly. But some people like to play KHangMan for example in another language and they don't want to install the whole i18n module for that language. So GHNS allows them in 2 mouse clicks to install the language words they want to add and they are immediately able to play. So far KLettres, KHangMan and the glowing KStars use it. When the upload system will be installed, contributors will be able to share data.
Thanks to Josef and all developers who improved GHNS, thanks to all the amazing people who contribute data and spend a lot of time on this task!
Errors - Evan "JabberWokky" E. - 2005-03-14
"Error parsing providers list".
Hurm. Not exactly what I was hoping for. It seems to not be loading the list from some website, spitting the above and then displaying a sparse (empty) dialog. No help, no... anything.
I'd file a bug report, but I think I'll wait for 3.4 and try again. ;)
I think this is just a tease for people who have to have stable desktops and can't play with the latest and greatest.
I would kill for... - ac - 2005-03-14
I would kill for SuperKaramba support.
This app *screams* for this kind of functionality! :)
Re: I would kill for... - David - 2005-03-14
Integrated KDE Karamba support. Now that would be interesting.
Security problem... - U - 2005-03-14
And who is going to control the content of this feature?
since any user will be abble to upload files, who will be in charge of not let bad files like porn, virus, etc. to be uploaded?
That's the same misstake MS made with ActiveX and look at Windows now, is full of virus and trojans, so, and explanation would be nice.
Re: Security problem... - Richard Dale - 2005-03-14
Have you heard of KNewStuffSecure?
Re: Security problem... - U - 2005-03-14
Link please.
Re: Security problem... - U - 2005-03-14
Nevermind, I just read it.
Only mentions a digital signature, but that doesn't answer my question, who is going to control the files flow?
That digital signature left to much to desire.
Re: Security problem... - Morty - 2005-03-14
The control flow can to be decided on a case to case basis for the different applications. But as for virus and trojans, you have to look at what kind of content we are talking about. It's all about application data not code, it be language files for edu programs, xml data/templates for the likes of Quanta or pictures for you desktop. Providing security for content like this are manageable and it has to be in place anyways, with or without KNewStuff. As for porn, there are already some moderation on KDE-look.org. And you get preview in the downloader for desktop backgrounds, so I don't think you will download anything you don't want:-)
On the other hand I'd guess there are no real problems in setting up a server for downloading x-rated backgrounds either. As for everything on the net, the type and degree of controll on the content have to be set by whoever manage the server. As for what too download, this is up to the user as always.
Re: Security problem... - U - 2005-03-14
Thx. for the reply.
But with your "left all to the end user" wont solve the problem, because we know some users are ignorant of what they are downloading, most of then just click the button "Install" w/o think to much in the results.
Now, if this happens, we would enter to the Windows category, of anty-spayware, anty-virus, etc.
I personally think the security for this kind of project is not mature enought.
Re: Security problem... - Christian Loose - 2005-03-14
> I personally think the security for this kind of project is not mature enought.
I'm sure the developers are eager to hear your proposal how to improve the security.
Re: Security problem... - U - 2005-03-14
With more control over the uploaded files of course.
If no person supervise the files uploaded you can be sure there will be not desired material on KNewStuff.
The philosophy of anyone can upload files anytime of anykind just let secutity holes everywhere.
Re: Security problem... - Morty - 2005-03-14
>The philosophy of anyone can upload files anytime of anykind
This is not the philosophy, it's up to the developer of the application and server where the "stuff" are hosted to decide the policy. The developer decides this when implementing it in the applications. Ranging from free for all, to content only signed and uploaded by the developer.
Re: Security problem... - Christian Loose - 2005-03-14
> If no person supervise the files uploaded you can be sure there will be not desired material on KNewStuff.
Okay, so you're volunteering to check all uploads?
Did you really inform yourself how KNewStuffSecure works?
(http://www.kdedevelopers.org/node/view/908)
(http://www.kdedevelopers.org/node/view/913)
For example Quanta already does use a review queue for uploads from untrusted users.
Re: Security problem... - U - 2005-03-14
>>>Okay, so you're volunteering to check all uploads?
The troll answer this topic doesn't need.
Re: Security problem... - Christian Loose - 2005-03-14
>>>Okay, so you're volunteering to check all uploads?
> The troll answer this topic doesn't need.
No it's not a troll answer.
You said:
"With more control over the uploaded files of course. If no person supervise the files uploaded you can be sure there will be not desired material on KNewStuff."
So you're saying all upload should be checked. The question for me is who should spend his/her free time doing this?
Re: Security problem... - Morty - 2005-03-14
>But with your "left all to the end user"
If you read the whole reply you see this was in the second part of the reply, most specific related to available content against the users moral values and personal beliefs. I can even give a more detailed example. If a end users set the download to point to "x-rated wallpapers site". Selects a wallpaper titled nudemale, and the preview shows a picture of a nude male, and clicks install. And then are so ignorant to get surprised when the nude picture ends up as background, there is nothing technology can do one way or the other.
As for virus and spyware they are all some forms of applications, while this project are targeted towards data for applications. If there are ways to infect via the data, there already are grave security holes in the applications not related to newstuff. And it would be just as easy to infect the ignorant users by mailing them the malicious data files.
Re: Security problem... - U - 2005-03-14
Than you mister, you cleared up my doubs.
Re: Security problem... - Romain - 2005-03-15
And what about use it to add some KParts for Konqueror ?
That would be wonderful. See all those "little improvements" thread on kde-look that just add a toolbar/navbar/whatever on Konqueror...
How many time before we see that ? And before we get some virus on those little KParts ?
I don't mean we shouldn't use this app, but clearly Mister U. has a point. This stuff seems exactly what MS did with ActiveX. And everyone agrees that ActiveX must be removed. All the technology are already here in KDE to get the same thing. The only difference with MS is that we will never have a KHotNewStuff server with public upload access for Konqueror plugins. Or I hope so.
Re: Security problem... - Morty - 2005-03-15
>And what about use it to add some KParts for Konqueror ?
Then you first have to solve the problem of cross platform binaries. Or your distribution has to implement and adapt functionality, in the applications and on their server. Then you have to trust your distributor, which you already do by running their distro.
Trying to compare KHotNewStuff against ActivX are so flawed it border on stupid. If you want to compare it to something, think of iTunes with upload capability.
So, it's OK to upload a PNG exploit?! - Hans - 2005-03-14
So I am allowed to upload an PNG wallpaper that exploits a recent libpng vulnerability?
There is no "secure content" unless it is 7bit standard ASCII.
libpng and libtiff have more often security vulnerabilities than khtml and konqueror. Those mp3 player libs and video libs normaly contain more buffer-overflows then there are mpeg4 codecs.
So I think there is a problem to be solved.
Re: So, it's OK to upload a PNG exploit?! - ac - 2005-03-14
> So I am allowed to upload an PNG wallpaper that exploits a recent libpng vulnerability?
And what prevents you to send this PNG wallpaper per email or to upload it to deviantart.com or ...
Re: So, it's OK to upload a PNG exploit?! - Hans - 2005-03-15
Kamail warns me. With Konqueror you have to actively got to a webpage, select an image and download it.
The controlcenter only says: "Get new wallpapers". Have I overlooked the Popup which says that any wallpaper might be an jpeglib/libtiff/libpng-exploit?
Re: So, it's OK to upload a PNG exploit?! - Morty - 2005-03-15
>With Konqueror you have to actively got to a webpage,
> select an image and download it.
Why do you think you have to download it? With such an exploit don't you think it's more than eunuch to let Konqueror display the image since it's using the same library to put the picture on screen as the desktop backround does. Downloading the image to local disk does not make any difference. As an example, I think it would be much more effective to attach a PNG like that to a posting on the dot, than trough some wallpaper. KHotNewStuff does not magically create new vulnerabilities.
Re: So, it's OK to upload a PNG exploit?! - Morty - 2005-03-14
>So I am allowed to upload an PNG wallpaper that exploits a recent libpng vulnerability?
As you today are allowed to mail the same wallpaper to lots of people and upload it to all kinds of wallpaper sites, or you may use the exploit in a PNG placed on a website.
Mr. - John F wiley - 2005-03-15
I'm a little dissapointed in the creations of some of the KDE programmers.
What I've seen of KOffice just re-creates the wheel. sorry, that's my feeling.
Personally, when I'm creating a document I like to work "on the fly"--that's change the tools (possibly toolbars????--a challange for a games programmer maybe). The acient CanonCat Computer supported this--wordprocessing tools, then you could instantly--"on-the-fly" switch to spreadsheet tools (with calculated fields, but not grids), to draw (now picture)tools, to database tools. You could have the equivelaent of multiple windows open if you wanted to, but you did not have to in order to create a document. I would like to define the page size (and portr. or landscape) before I start though.
Is any team bright enough to create this? Or are we stuck with programmers just re-creating the same old thing???
Definitely a step in the right directions. - somekool - 2005-03-19
awesome guys...
can't wait to see this running and get everything updated through it.