KDE Commit-Digest for 2nd December 2007

In this week's KDE Commit-Digest: The beginnings of screen hotplug detection in Plasma, KRunner gets history support. Fifteen Pieces puzzle becomes the first Plasma applet in the game category. A block of bugfixing in KDevelop, with various other developments in areas such as a threaded debugger. Support for inequality constraints in Step, continued progress in the port of KEduca to KDE 4. Work on printing in okular. Work on Solid-based network management through NetworkManager. Various work towards Amarok 2. Milestones reached in the BitTorrent plugin for KGet. Subsystem rewrites (SSL, SFTP) in KFTP. OpenDocument format loading and saving work in KChart. Colour work in Krita, with Krita becoming one of the first applications to be able to paint in HDR. New Oxygen-themed sound effects, Oxygen icons are optimised for small sizes. New colour schemes added for KDE 4.0. Ruby language bindings based on the Smoke2 framework. Experiments in KBugBuster and on a Plasma "applet designer" application. Read the rest of the Digest here.

Dot Categories: 

Comments

There seems to be a duplication of functionality between KGet and KUIServer: they both list and control file transfers. I can see users getting confused by the two GUIs.

Here's an example: every transfer listed in KGet also appears in KUIServer, but not every transfer in KUIServer is listed in KGet (because KGet doesn't deal with uploading). Very confusing. How about merging all functionality of KGet into KUIServer?

There was also a plan to move towards a centralised "Factory" (working title, maybe?) applet, which maintained/controlled/showed background tasks for all apps in one central place in the systray. It would be nice to see KGet, KUIServer's, and KTorrent's UIs disappear completely, in favour of this.

No, the plan was to show tasks in KUIserver 'too'.not to replace them.means that you will have tasks both in the application ui and in the kuiserver.
(I think so)

by Lukas Appelhans (not verified)

KUIServer is very basic for each Transfer...
The other Applications(KGet, KTorrent...) do the more specialised parts, like Multithreaded-Downloads...

by J Klassen (not verified)

The beginnings of screen hotplug detection in Plasma...
So no more restarting X when you plug in a second monitor? Sounds sweet!

by André (not verified)

I was just browsing through the Solid API pages, and I noticed that screens seem to be missing there. Wouldn't it make sense to make Solid capable of detecting screens and notifying applications about their (dis-)appearance? Or does that work through the GenericInterface of Solid?

by ervin (not verified)

Basically it wouldn't be very useful, you need much more information on top of screen management like the new geometry of the desktop and how it spans on the various screens.

That why IMO for this case it's better to rely on X (at least for now).

by Thomas (not verified)

But it's quite inconsistant! Of course X is the better choice, but why not inform the system about a new screen. at least some applets like the hardware notifier would give feedback leading to making it a more consistant experience for the user as well!

by Aaron J. Seigo (not verified)

> So no more restarting X when you plug in a second monitor?

with new drivers and the latest xrandr, yes. there's a bug in QDestkopWidget, xrandr or the driver atm; however that i haven't had a chance to track down yet 9haven't even started looking, just made a note for myself that i need to delve into it later), but plasma itself is all ready: when a second monitor appears, a containment is indeed made for it. the geometry is not always right though due to the aformentioned bug.

btw, this got implemented solely because a person came onto irc a few evenings ready to report problems and test patches. that's the kind of stuff that really, really helps move these kinds of features forward =)

Wouldn't it be better to focus on improving the FTP and SFTP KIO slaves than the KFTPGrabber client? The KIO slaves are much more convenient as they can be used from the regular file manager (Dolphin) or from Open/Save As dialogs. It's cumbersome to open up a separate application just to deal with FTP and SFTP transfers (KFTPGrabber), another application to deal to with Windows/SMB (Smb4K), and another to deal with local files (Dolphin).

The FTP and SFTP KIO slaves are installed by default in KDE and improvements in the KIO slaves would help every single KDE app. Very few people even bother to install KFTPGrabber anyway.

by Peter Thompson (not verified)

I noticed that, too. Somehow the trend in KDE4 seems to lead away from the interacting modules as it was in KDE3, and back to the paradigm "one app for one purpose".

Dolphin insteasd of Konqueror, different apps instead of just different I/O-slaves - I don't like that.

Yes, I know, all the modular technology is still there, but it gets more and more hidden (for the so-called "poweruser").

by Sebastian Sauer (not verified)

KFTPGrabber also exists for KDE3, see http://extragear.kde.org

> different apps instead of just different I/O-slaves

why "instead of"? Isn't it more "additional to"?

by Will Stephenson (not verified)

KFTPGrabber is an extragear application, therefore it's by definition an add-on rather than a piece of core technology. And KDE is a very diverse free software project, where people work on what they want to. Just because there is a very specific app mentioned in Commit Digest, don't conclude that KDE is moving away from being the most well-integrated Free Software Desktop.

Put another way: If you were going to start hacking on KDE tomorrow, would you be more inclined to start by improving these kioslaves at the core of KDE, or to develop a standalone app in playground? Would doing that change the nature of KDE?

by Thomas Bergheim (not verified)

I actually sometimes prefer a single app - the problem is that KIO slaves does not seem to close when they should - for instance using krusader as an example, the FTP connections always seem to linger long after I have closed them (and indeed they do, because a lot of times I get shut out becauseee I have logged in X amount of times when the max limit is X (and I am really only trying to log in with my first user).

Another plus is you can easily ajust things like bandwidth control, activity times etc from a specialized program.

> Another plus is you can easily ajust things like bandwidth control, activity times etc from a specialized program.
Which would definitly be kool to have integrated into the kio-slaves and kuiserver dialogs.

I dont see how the two are exclusive...

On one hand the core dev can iron out the kinks of the various KIOs (there arent much... if any?) but at some point they must remain generic enough to be used kde-wide (their whole point)

However some people *do* prefer very simple app that serves one well-defined purpose. This app will use the KIO slave, obviously. So the fact that this app exists / is actively developed does not make the KIO any worse.

In fact, if it comes to push the boundary of the underlying libs, those will be adjusted to be better -- everyone win.

This is said very often: one dev working one "KFTPGrabber" does not mean -1 dev working on the ftp:// KIO-slave

> This app will use the KIO slave, obviously.

Actually, it doesn't. The commit says that KFTPGrabber is "Now based on libssh2". Libssh2 != SFTP KIO slave.

...and that is indeed a shame. It would be much better if the KFTPGrabber author forced himself to use the KIO slave, improving it as necessary, adding features perhaps (such as FTP parameters) to support the needs of the app. Then everyone would benefit from this work.

There certainly seem to be bugs in the kio_imap support.

I agree. One i'm missing at kio-slaves, is something like a control-device for actually used slaves. I mean something where i can see what app use which slave, and where i can terminate slaves. fish:/ for example seems every time endlesly open.

Hell YES. Great idea! I wonder if it's anything simple enough to be tackled by someone without (much) C++/KDE experience?

by Thomas Bergheim (not verified)

+1

This would be really great, and it would more easily solve my "multiple-lingering-logins" ftp problem I described above, for instance..

Well, the protocol abstraction system KFTP uses is quite different from KIO slaves (just look the 'src/engine' subdirectory in svn repo). Not only (speaking for the FTP part) that it is state based instead of using blocking socket commands, it also includes things like speed limiting, keepalives etc.

As far as SFTP support goes, kio_sftp actually talks to openssh via stdin and that I don't like at all. Using a library like 'libssh2' that implements the underlying protocol seems like the proper way (yes I know, when kio_sftp was designed there was no such library available).

Ofcourse there are similarities between the two systems and one could try to port it, the code is readily available ;-)

by John Tapsell (not verified)

Are you working on this ftp program?

Please, please improve kio_sftp instead. Using libssh2 sounds like a great idea - please port your changes across and use the kioslave instead if possible.

Can't speed limiting etc be done in kio_sftp? I mean, can it be implemented, or does the api simply not allow that?

by Kevin Krammer (not verified)

Probably using job meta data, like passing user agent and referrer to kio_http

Can these be altered while a job is in progress? Changing upload speed mid-upload is a common use-case for FTP transfers.

by Kevin Krammer (not verified)

Good question, but I don't know.

Probably not (yet)

Yes I am the author of KFTPgrabber. There are the following problems with KIO slaves:

1. KIO slaves should provide a standard way to do speed limiting across multiple connections (KFTP uses the token bucket algorithm to allocate bytes to individual connections) in some logical group. That way you can limit the speed of all transfers inside an application in a fair way and modify the limits at any time.

2. The FTP part cannot be easily ported because (as said before) KFTP uses an async state-based engine (think FSM) with command chaining that requires an active QEventLoop. But as I currently see, all KIO slaves are made around blocking socket operations.

3. KFTPgrabber's FTP engine supports things like FXP (site-to-site) transfers where two separate connections have to synchronize their states to be able to transfer a file. I don't know how this could be done with KIO slaves.

So currently changing the core architecture in KFTP is a no go. I do want to improve kio_* in the future, but at the moment I barely have the time to work on KFTP.

> So currently changing the core architecture in KFTP is a no go.

The KIO architecture will have to be revamped soon because Trolltech is building a new networking stack:

http://labs.trolltech.com/blogs/2007/11/24/one-more-piece-falling-into-p...

There's never been a better time to improve KIO so that it is more flexible and so that it can do speed limiting and everything else KFTP is capable of.

"The KIO architecture will have to be revamped soon because Trolltech is building a new networking stack"

Huh? How does this impact KDE at all?

by jospoortvliet (not verified)

Because KDE will reuse as much as possible from TT's networking stack?

I don't see how it would, considering that KIO binary interface is part of the platform ABI. It might change some backends, of course.

> I don't see how it would, considering that KIO binary interface
> is part of the platform ABI.

The ABI will change for the next KDE version release, KDE5. Or KIO can take the "Plasma approach" and not guarantee binary compatibility in the KDE4 release.

Binary compatibility is overrated anyway -- it goes against the "release early, release often" methodology that best suits open source software.

KDE5 is years away. And KIO -will- guarantee BC and SC.

And you could argue about BC being overrated, but SC certainly isn't, and BC helps with that. I've fixed half a dozen of critical bugs that were caused by tiny semantics changes. There is absolutely no way that changing the internals of a heavily used and mature subsystem won't cause a substantial number of regressions. That includes app compatibility, and also server compatibility --- kio_http contains tons of workarounds for broken stuff that's out there.

Replacing working code with something new and shiny just because it's new and shiny is simply an invitation for problems.

At any rate, we're sure as heck are not obligated to throw out our code just because Qt is duplicating kdelibs functionality yet again.

> heavily used and mature subsystem

If KIO was that great, KFTP wouldn't need to resort to using more feature-rich and updated libraries like libssh2. The very existence of KFTP and SMB4K reveals that KIO lacks required features, pushing many contributors to develop/improve peripheral technologies that don't benefit KDE as a whole. And even if they want to bring KIO up to speed, they're confronted with a "you have to wait 5 years for KDE5" argument. This policy only leads to stagnation in KIO and fragmentation in the development effort.

Look at GNOME, they're dumping their old gnome-VFS layer for GVFS/GIO while in the stable 2.X release. KDE should be able to do that too.

> we're sure as heck are not obligated to throw out our code just
> because Qt is duplicating kdelibs functionality

Duplicated functionality means wasting users' RAM, but I won't argue who's to blame ;)

by Aaron J. Seigo (not verified)

> If KIO was that great, KFTP wouldn't need to resort to using
> more feature-rich and updated libraries like libssh2

if you read the author's comment, you'll see how this statement should've been "if kio_ftp was that great". it's not about kio at all.

> Look at GNOME, they're dumping their old gnome-VFS layer
> for GVFS/GIO while in the stable 2.X release.

which is made easier by the fact that so few apps use it. there are development benefits to a low-integration environment.

> KDE should be able to do that too.

it's slightly more difficult when it is used so pervasively. we could certainly offer a second stack and apps could slowly move over to it, but we would still have to keep kio around until kde5. it's about respecting your developers and users (since making developer's lives harder means taking features and bug fixes away from users)

so yes, it's possible. but for what gain and at what price? kio is actually pretty decent and did see improvements for kde4.

> it's not about kio at all.

Sure it's about the KIO architecture. Let's take Jernej's comments one by one:
http://dot.kde.org/1196818371/1196822000/1196849333/1196855480/1196871644/

1) Speed limiting -- is there any API to allow a KIO client to control the speed of a download/upload in _any_ protocol? Nope, so that a KIO-wide deficiency, not just kio_sftp.

2) Can _any_ KIO slave be written around an async state-based engine (ie. "mounting")? Not according to Jernej, so that KIO-wide as well. GVFS/GIO has made great strides in this area compared to its predecessor, btw. And I agree that it's needed in order to avoid the penalty of re-authentication with every access to the server.

3) Site to site transfer without downloading locally -- will probably have to create a new job class to cover this. So again, KIO-wide deficiency.

Here are my own wish list for KIO:

4) Permissions from KIO::StatJob are worthless because they represent users/groups on the remote system, not the local one. Consequently KFileItem::isWritable() returns a bogus result for remote files.

5) Can't start/use a KIO job from a non-main thread. It's excruciatingly cumbersome to have to shuttle all the requests to the main thread and the results back to the worker thread.

by Kevin Krammer (not verified)

> Can _any_ KIO slave be written around an async state-based engine

Sure, why not?

Blocking on a socket is sometimes convenient if one doesn't have to do anything else, but it can also be done non-blocking using event notification (e.g. select())

Also, KIOs still can't be chained (like opening zip inside tar inside arj, etc),
which is something mc for instance can do for ages.

by Richard Moore (not verified)

We used to support that and removed it. It was rarely used and leads to a number of bugs and ambiguities in things like URL parsing.

> leads to a number of bugs and ambiguities in things like URL parsing.

If introducing abmiguities in in URL is the only problem with chaining KIO slaves, then this can be easily fixed through repeatedly "mounting" each component. Say you have a file essay.txt saved in bundle.tar compressed as surprise.bz2 saved on sftp://school.edu/home/username.

First you'd mount sftp://school.edu/home/username in ~/network/school. Then you'd mount ~/network/school/surprise.bz2 in ~/archives/surprise. Then you'd mount ~/archives/surprise/bundle.tar in ~/archives/bundle and your file will be accessible as ~/archives/bundle/essay.txt.

There'n no URL ambiguity in ~/archives/bundle/essay.txt.

> And even if they want to bring KIO up to speed, they're confronted with a
> "you have to wait 5 years for KDE5" argument. This policy only leads
> to stagnation in KIO and fragmentation in the development effort.

Of cource, you can always make an kio2 and use it beside kio, until it become kio with kde5.

> Of cource, you can always make an kio2 and use it beside kio
> until it become kio with kde5.

Yup, that's a plausible idea.

I know that sounds radical, but i think for network transparent file-management (http is a different story) KIO should simply be deprecated in favor of a thin QT/C++ "QGio" binding for GIO/GVFS. To help migration, KIO netaccess and jobs could wrap this binding.

There is no sane reason to maintain multiple incompatible network transparency layers - Just a waste of time and a usability desaster. GIO/GVFS is nice and lean - and there is no doubt it's design is a step forward. Competition between KDE and Gnome is good, but at the file-system layer it's unhealthy.

If i had time i would start coding "QGio" and the KDE/GIO bridge today...

Two big reasons why KDE shouldn't depend on GIO/GVFS:

1) Linking to two frameworks that do the same basic functionality (QtCore and Glib) means wasting RAM.

2) If a KDE developer wants to fix a bug in file-transfer, he would have to code in C and learn Glib programming. Dealbreaker right here.

Having said this, GIO/GVFS has a lot great ideas that can be incorporated into a new version of KIO.

1) Loading two VFS libraries with all their daemon/slave processes, which will happen as soon a GTK app runs on KDE (quite often, to be realistic) means wasting even more RAM. The whole point of sharing code and libraries is saving RAM. Pure GNOME or KDE desktops may perhaps exist in the dreams of desktop developers - but nowhere else...

2) KDE developers learning C? So what? Being "multilingual" is not a big deal for software developers. It's just normal. I - for instance have to code in Java, PHP, C#, C, C++,... to make a living.

+3) What about usability? Isn't file-management consistency more important than loving a certain programming language or not? Or having another tiny library in memory - who cares...

The whole situation of an estimated third of the applications linking one VFS layer, another third a completely different one, and the rest without access to network resources at all is a big joke. Honestly - i think it's quite pathetic. You will never be able to explain this chaos to anyone who isn't a computer guru.

Monopolizing parts of their library infrastructure has been an unfortunate consequence of the "desktop wars". Using GIO/GVFS in KDE/Qt is a chance to fix this "bug"...

by Kevin Krammer (not verified)

I think the misunderstanding is that the application would somehow link against GIO/GVFS.

Quite likely the architecture of this new VFS is quite similar to KIO, e.g. the application is talking to some other process which in turn is handling the "remote" file system.

Thus the application does not get any additional dependencies, e.g. just like KIO slaves can always be used without KIO Jobs which is basically just the KDE API to file access.

Depends how you design it. I think the best and leanest solution would be linking the GIO client in-process. Because QT4 already links GLib for the main loop by default (i think), that wouldn't be a problem. And it's the best way to avoid redundant code. You only need a thin C++ wrapper layer and you get all the threading stuff for asynchronous local file access, file/directory monitoring, drive/volume management for free.

I my opinion the "lingua franca" for cross desktop libraries should be GLib+C. That's a bit revolutionary, but it would make a lot of things easier. You don't have to code client side libraries twice, especially when they are asynchronous (because GLib provides the main loop). Desktops can move forward instead of wasting time duplicating and maintaining code which practically does the same thing.

But - of course - there are other ways to bring GIO/GVFS to KDE. The easiest way would be to link the GIO client inside a generic KIO slave which handles multiple protocols (file:, smb:, sftp:, ftp:,...). With this you hardly have to change the current system and you can still have slaves for the protocols which GIO/GVFS doesn't provide (http,...). But it's a bit more bloat regarding deamon processes.

Alexander Larsson said that it would also be possible to talk to GVFS mount daemons directly via DBUS. That's the third option, but IMHO a waste of time, duplicating client code.

If KDE wants to move in the GIO/GVFS direction i would start with the second option (the generic multiprotocol GIO/GVFS IO-slave). It would also work for KDE3, thus KDE3 and KDE4 apps can work on the same network shares.

Anyway, i think KDE developers who are interested should join the GIO/GVFS project by subscribing [email protected] as soon as possible. It would be cool for desktop file-system experts to share their knowledge (which hardly happened in the past i think). And also, because GIO/GVFS is not final and API/design changes are still possible.

by Kevin Krammer (not verified)

> Depends how you design it.

Right. I couldn't find anything about GIO architecture, so I assumed that some kind of daemon would handle the IO on behalf of the applications and they would connect to it.

> I think the best and leanest solution would be linking the GIO client in-process

Well, if all the functionality is implemented by threads or select() then you more or less have to link to the library. If it is some kind of daemon it is usually better to implement the protocol natively as this avoids multiple conversions, doesn't depend on any symbol/behavior assumptions and helps find errors in the protocol.

> Because QT4 already links GLib for the main loop by default (i think)

I think this isn't default on even Unix, but even more unlikely on any other platform.

> You only need a thin C++ wrapper layer

Really depends on the library. For example it would make no sense to wrap libhal or the networmanager client library if you can use actual API (D-Bus) directly.
Of course if a library is the functionality, then wrapping makes sense, unless source and target framework differ too much in concepts or one would convert data a lot.

> I my opinion the "lingua franca" for cross desktop libraries should be GLib+C.

True. If I remember correctly there have been numerous proposals to split GLib into the data handling part and the object model part, so that the data handling part can be used without interfering with other frameworks' object models. So far there has been no move into this direction, usually people who want the convenient C function take the data handling part of GLib and include it with their libraries.

> The easiest way would be to link the GIO client inside a generic KIO slave

Right, would especially help to see how each architecture's concepts can be mapped to the other

> and you can still have slaves for the protocols which GIO/GVFS doesn't provide (http,...)

And https, webdav, ftp (according to http://live.gnome.org/GioToDo)

Since googling for GIO on site:gnome.org renders a lot of blogs, minimal API docs but no architectural or design information, it would be interesting to at least get some information about how the integration with the respective desktop's authentification system is handled, how connection state data like cookies are shared between MIME inspector process (e.g. application launcher) and MIME handler (e.g. preferred application) and how user interaction (e.g. asking about overwrite, skip, etc) is performed.

by Kevin Krammer (not verified)

> So far there has been no move into this direction...

Correcting myself:
seems to be a only a packaging issue, i.e. there is a separate libgobject but it is part of a package called libglib. So it is no longer a dependency issue, I stand corrected