The Road to KDE 4: Job Progress Reimagined

Have you ever had your taskbar filled with 10 applications all
doing something that involved waiting for a task to finish?
Document Printing Progress, a K3b CD burning dialogue, Audio Encoding
via KAudioCreator, File Transfers in Konqueror, Kopete, KTorrent,
checking email in KMail... The new Jobs support in KDE 4 will unify
the display of progress for these tasks, making it easy to see and manage what is happening on your system. Read on for details.

Picture it as a cross between the Firefox download manager and
the KDE printer queue, except that there is no real restriction on
what type of jobs can be monitored. The way it works is that each
KDE 4 app that has a progress dialog adds a flag for
something called an Observer. Then, a separate application can observe any running Jobs, displaying progress and even adding
certain actions (like "Cancel Download") which can be submitted back
to the application that actually has the progress dialog. So the
applications like K3b, which already have very good progress
reporting, will not lose their existing dialogs, but rather additionally permit this new applet to observe its progress so that all the progress bars can be pulled into a convenient place.

What started as a
mocked up KDE 4 Improvement
via KDE-Look.org has turned into a
full-fledged KDE 4 integration project, thanks to Rafael Fernandez Lopez.
And there's been a lot of progress to the point where applications
are already being adapted to the new infrastructure. Last
Tuesday's "Binary Incompatible Changes" day saw much of the changes
officially committed to the KDE 4 repository.

Below is the original mock up, done by KDE user and KDE-look.org
contributor
kiras
, used with permission. Click to see the full-sized
mockup.

Please keep in mind that the above is a mockup, and does not
necessarily reflect the ultimate look-and-feel goals of KDE 4,
Plasma or Konqueror.

Currently, it is being prototyped as a standard system tray applet
(similar to the printer queue in KDE 3.5.5) which would allow
interoperability with GNOME's tray implementation as well. However, at this point only KDE applications can be observed, so monitoring download
progress from Firefox for instance, is not currently supported. That is not to
say it cannot be made to happen in the future since progress is observed using the standard D-Bus interprocess communication architecture. There are intentions to collaborate with the GNOME project's Mathusalem team, a
project of similar scope as this one.

Here is a screenshot of the current appearance of the monitoring
application as it would appear when clicking on the tray applet. As
you can see, it's already looking very useful.

As you can see, the Kopete buttons are mostly just placeholders
at the moment, and only exist for testing purposes. However, when
you click on one of those buttons, it actually sends a signal back to Kopete,
and Kopete itself pops up that smaller dialog you see.

The Konqueror download progress bars you see being monitored represent
an actual file download in progress. They continue even after Konqueror
is closed. Useful action buttons like "Abort Download" are in the
works.

If you'd like to get involved in KDE 4 development, adding
support for the new KJobs progress monitoring is a fairly easy entry
point to KDE programming. It takes only a few lines of code to
adapt an application to display progress, and a few more lines to
make the action buttons useful.

This new progress monitoring technology will be able to be
integrated into Konqueror (like in the mockup), desktop applets, and
anything else that uses D-Bus. I can even imagine a small web-app
that lets you monitor progress remotely...

Rafael's goal after the initial implementation is completed is to
add persistence, such that when a job is complete, it would
optionally stay listed until closed by the user. He is also looking
for feedback on this tool and its implementation for future improvements.

Look forward to more feature articles showcasing more great
technologies for KDE 4.

A quick note on methodology: I make sure to use the KDE defaults for all of my screenshots, even if it's ugly, since then you can get a better sense of progress as KDE 4 evolves and grows from week to week. As a rule, all of the
features I've demonstrated so far are publicly available in SVN, and
anyone can reproduce my results. In today's article, I had to
uncomment a single line of code to enable this in-development
feature, which is an exception to my normal rules. Additionally,
the Kopete progress support is not yet in the official KDE SVN repository, but Rafael uses it to test features.

Comments

by Carlos (not verified)

Rafael, the job you've done is great, but don't you think there's way too much space used for each task? Imagine 10 tasks at a time and you'd have a huge scrolling bar.

by Troy Unrau (not verified)

The application you see is an initial prototype useful more to prove the backend work is functional. When the plasmoid data engine is done, the applets should be able to format this information however they see fit.

by Shamaz (not verified)

Thank you for your work Rafael !
About I/O job queuing, you may find some ideas here :
http://sourceforge.net/projects/supercopier/
I DO know this is a Windows application, but it has interesting features : transfer speed control, transfer resuming (and pausing) and the possibility to change queue order.
This kind of feature already exists in many http or ftp client. This is really something I miss for simple disk to disk transfer :(.

Good luck !

by Max (not verified)

Nice to see KDE is doing this via D-Bus, so even Gnome programs con benefit in the future.
I've made two mockups for Gnome but perhaps you are even interested in it:
This one shows the progress directly in the Icons:
http://hagemaenner.de/stuff/gnome/ActiveIcons.png
And this one in the Taskbar:
http://hagemaenner.de/stuff/gnome/gTask.png

by Thiago Macieira (not verified)

KDE is now doing almost everything over D-Bus.

Remember that since last May, D-Bus is the official IPC/RPC mechanism for KDE 4.

One thing I want to do in the near future (whenever I find the time) is to port KIO's own protocol to D-Bus. This would expose the ioslaves to D-Bus, if necessary. That's one of the two remaining non-D-Bus IPC mechanisms that I can remember right now (the other one being the communication between klauncher and kdeinit, but we won't change that).

Now, I'd like to point out that this is not an unified VFS mechanism that many people are clamoring for. But it may be a start, if it proves useful, and it may pave the way to a real implementation.

by Sebastian Kügler (not verified)

It could be done by adding an option "Start when [other_job] is finished", where [other_job] is a dropdown with all active jobs, for example. Jobs that let you pause them can have this feature enabled then.

Just an idea, anyway. Thanks for the great article and this cool new feature which will reduce clutter significantly!

by Jens Rutschmann (not verified)

Or by selecting multiple entries in the list, then Rightclick->"Queue Us" or something like that.
Anyway I think this is a really cool feature !

by Martin (not verified)

Will we also see the lower part of the screenshot implemented, where Konqueror implements its own observer to show download progress in-place?

by superstoned (not verified)

will depend on konqi, i guess... won't be very hard with the whole infrastructure in place, tough.

by Kevin Colyer (not verified)

This is good news - I like this idea very much. It answers that question why do I have so many progress dialogs floating all over my desktop. I often need to know how a print job is progressing, how a file transfer is going. Having it all in one place will be so much clearer for me and new users. Ever seen a newbie look for info on a print job on any OS let alone KDE?

Will it be easy to make hooks for CLI apps? I guess with Dbus it could be. I would love a hook to rsync so I can see how a my home brewed backup is going!

by Thiago Macieira (not verified)

If rsync -- or any tool -- outputs a percentage, it should be quite simple to write a bash script reads from a pipe, creates an entry in the uiserver and updates it. An example tool I can think of right now is cmake & KDE's own build.

How cool would it be to see the KDE build percentage progress in the uiserver? :-)

If you need to implement actions like "pause", "cancel", etc., you will find that bash+qdbus won't suffice. Scripting languages like QtScript, Ruby or Python will come in handy then.

by me (not verified)

It would be great if you could pause a job, reboot and continue it later (session management

This might not be the uiserver's responsibility, but when copying a lot of files, is it possible to FIRST ask for confirmations for all to-be-copied files (do you really want to overwrite etc.) and THEN start copying? That way, I can leave the computer and be sure its done when I come back!

The copying-two-files-at-the-same-time-becomes-too-slow-problem is even worse when you're reading them from a cd/dvd!

by boemer (not verified)

Great Idee, one of the things that most irritate me, having a directory moved under windows and for every thumb.db, it starts to ask again, if I'm sure... I know it is windows... But watching this progress for KDE 4 makes me feel better every time....

by Boris (not verified)

"is it possible to FIRST ask for confirmations for all to-be-copied files (do you really want to overwrite etc.) and THEN start copying?"

Better yet, it would be great if the system popped up a conflict resolution window where you could check which files you want to overwrite and which you don't want to overwrite. Then it would be very simple to select all, and perhaps uncheck a couple.

by Anon_Poster (not verified)

I really like that idea. Perhaps a two-pane view

file1.jpg 2/1/07 102K <-> file1.jpg 12/27/06 99K
document.ods 2/1/07 73K <-> document.ods 12/29/06 99K

where if you hover over the filerow you get a thumbnail comparison of those documents. With option to view details (which opens up in appropriate app).

thumbnail \/
__________________________
| _______ _______ |
| | | | | |
| | PIC 1 | | PIC 2 | |
| |_______| |_______| |
| View det View det |
|__________________________|

Really ugly above I know but?

by Stromek (not verified)

I cannot wait for the beta/RC and release. It is so wonderful, it is going to be so biiiiig improvement to desktop experience.

I wish all the people to use KDE desktop!

by wish (not verified)

My wish is a peacefull world where everybody can do what he/she wants to (accept harming somebody AND something else).

Why should all people use KDE? There is no meaning full reasing for such a wish. Wasn't that something good about all the Linux/BSD/Unix/KDE/GNOME/Fluxbox/... thingy? Use what best fits to you.

by cirehawk (not verified)

Yes, freedom of choice is one of the great things about open source. But I see his/her wish that everyone use KDE as an expression of excitement about the upcoming KDE4 (and KDE in general). There is nothing wrong with that. I didn't take it as trying to mandate that everyone use KDE.

by Diederik van de... (not verified)

When I see examples like today at PlanetKDE (http://www.ereslibre.es/?p=11), I get a bit annoyed to see such generic name as "Observer" being used for this. How about a name like JobObserver or KJob::Observer?

by Thiago Macieira (not verified)

It's KIO::Observer right now.

The name may change if it moves out of libkio into libkdecore.

by Diederik van de... (not verified)

Thanks!

by ra1n (not verified)

I like this, it's not a new concept, you can see this in beos tracker and mac os finder, but limited to file transfers, now a good point would be interoperability, like you said there are already some talks with gnome folks, but I think that a further step would be some kind of freedesktop job queue with common API that can be implemented in every DE, so if I run kde and want to run gftp for example I could see gftp's jobs run in kde tasks.

by superstoned (not verified)

read the article: it'll be d-bus based, and they are already working on standardizing the d-bus interface.

by ra1n (not verified)

well it's not too obvious from the article, there are talks with gnome folks and the use of d-bus, but nothing it's decided yet, especially on the other side of the desktop ;-)

by oldschool (not verified)

It seems this is a great use of multi threading - being able to cancel a long operation. Most Qt3 and KDE3 apps, even if they have a cancel button your are at the mercy of the event loop before if and when it decides to accept the cancel callback.

I think this will go along way in making KDE 4 more user friendly. May I suggest that it be designed to not only work with DBus for interprocess communications but internal to the app itself. The goal being to make it simpler for programmers to do multi threading for simple thins like a "cancel" operation

Way to go Rafael

by Thiago Macieira (not verified)

I hope to make it possible to use D-Bus to do that, so the integration is seamless.

No promises though.

by testerus (not verified)

How is this supposed to work with multiple desktops? Is it going to be like the taskbar: "Show progress information from all desktops", "Group similar jobs", etc.?

by Tom Corbin (not verified)

Is that an existing look or something planned or just blue sky?

by Troy Unrau (not verified)

Mostly just a mockup - kiras has some ideas of how he would like KDE 4 to evolve, which is good as some of the ideas are sound. Other ones are less likely to be implemented from his mockups. You see the little configure button on the window titlebar - well, his idea would be that when you click that button, the window flips over using OpenGL and displays the options on the backside. Not likely to happen for a variety of technical and usability reasons, but! the mockup sure looks sweet :)

Konq at the moment looks pretty much like KDE 3's konq, except with improved HTML/Javascript rendering :)

by kwilliam (not verified)

I loved the mockup screenshot when I first saw it on KDE-look.org. I am totally excited to see that this awesome screenshot transforming into actual code! It's also bolstered my confidence that Konqueror will include improvements from some of the KDE4 mockups on KDE-look.org.

This progress applet will be so cool! A huge THANKS to all the KDE4 developers out there!

by Alex (not verified)

Um, I'd like to point out that some apps, like KTorrent may have multiple tasks waiting to finish... Are we going to have drop-down lists of subtasks for each app? and display an 'overall progress' bar at the top of the list when it's collapsed? Just an idea for making the app useful for one more potential user.

by riddle (not verified)

That should be fairly easy to implement...

You're thinking like this, right?
--------------------------------
-Konqueror
file1 copy: [========= ]
file2 copy: [============= ]
+K4B
+KTorrent
--------------------------------
(Ugly, I know)

by riddle (not verified)

Oops, I just checked out the latest beta. It already supports that.

Smile ;)

by David (not verified)

that the main panel buttons should reflect this. Whereas the windows taskbar buttons flash blue when the window wants attention, we could turn those buttons into miniature progress bars that would scroll their text.

by great (not verified)

Hey user, ever thought abiut your role in this story? Your are not a programmer and you have no idea what you can do?
You can contribute in many ways by donations and thinks like that. Help the developers to cover their affords and give them something since they have spended a lot for you. They need money for meetings, textbooks, travel cost, ... . KDE is a community project ... and you are the community! Here is how you can support KDE:
http://www.kde.org/support/

But what do I see here?
http://www.kde.org/support/donations.php
What do I have to read? Last donation on Nov 28, 2006? Oh dear, isn'that poor? Remember, some little helps if a lot of people help a little! ;)

by Georg Grabler (not verified)

I'm working daily with kde / kate to work off my development tasks. Though, it sounds very interesting, but the possibility to suppress those other windows from even opening would be worth a thought.
I'm thinking on developing on a machine with kate, code being on another machine (compilefarm / testing), connecting by fish. By opening thos files, a hundret and three dialogs open which i don't really want to see.
I'm aware that this is not what this interface is supposed to do, but still worth the tought.

The folding idea on subtasks sounds interesting, like "program groups", as application you see on the right side on the screenshot (processing x tasks and on a closer look the list).

Besides all those other projects i work on i could even imagine on getting my hands on this, since it could be a important feature for daily work, to see status of downloads copies transfers cd buring or ripping collected, without openeing one after the other application just to take a short look.

Kind regards,
Georg

by Anton (not verified)

All of those improvements are good but I think it will be too hard to release on the current system. When I look at the mockup I understand that something looking like this is unreal without HTML+CSS-like technology. I think the better GUI improvement of KDE that can be is an ability to use XML+XSLT+CSS for rendering windows.

by asd (not verified)

why do you think limiting the rendering to html would be a good idea?

kde styles are written in c++. they are programmed. you can do almost everything you want in a kde/qt style right now, while they are still very fast.

using html based rendering for styles would only make kde realy slow. just look at firefox and its memory usage.

by Anton (not verified)

Why limiting? And why html? I meant just the stylesheet model but not the html self.

Yes, you can do everything you want in a kde/qt style. But why then
a lot of kde applications looks so ugly and awkward? And why all the kde mockups are so differ from their realizations? Because C++ programmers in most cases are unable to make good UI and good designers cant use C++. I'm saying this as a web-developer and a former C++ programmer.

Firefox is not so slow as efficient with all of its extensions written by many enthusiasts. And firefox styles are also programmed ;)

by Pebete (not verified)

"If you'd like to get involved in KDE 4 development, adding support for the new KJobs progress monitoring is a fairly easy entry point to KDE programming. It takes only a few lines of code to adapt an application to display progress, and a few more lines to make the action buttons useful."

How can I get involved?

by Troy Unrau (not verified)

Contact ereslibre on irc.kde.org... he may be able to help you out with an example application or somesuch.

by Pebete (not verified)

Thank you.

by Colin (not verified)

Its great that we will finally get a unified and tidy way of displaying and keeping task progress together and I'm sure this framework will make lots of developer's lives easier.

I currently use Metamonitor (http://metamonitor.sourceforge.net) for getting notifications of certain events through system logs (such as plugging in USB devices and my backup scripts running, etc). Would it be possible to combine this functionality with this progress manager?

They're both concepts that require a dialog to be shown for a certain period of time, usually near the system tray area for neatness and consistency and may be stacked up with several other dialogs. I also think it would be useful for it to show a recent history of notifications too as you can sometimes miss what these notifications say because of the popup auto-close (which is normally a good feature IMHO). Applications/services such as Kopete and KBluetooth could then use the same framework for their notifications.

by Rafael Fernánde... (not verified)

Hi,

First of all. This app is really great. I find it very useful.

Well, all the available methods you can call are on kdelibs/kio/kio/observer.h.

I don't recommend you to use them right now, because they can suffer strong modifications from monday to monday. Anyway you can take a look there and see what you find :)

Bye,
Rafael Fernández López.

by Maciej Cencora (not verified)

That's great idea.
But please add an option "Shutdown computer after all jobs are finished" so user could go sleep, not waiting for printing/downloading/file copying being finished :)
Can't wait for KDE 4 release.

But what about doing something about the taskbar hell????

I Believe Steve Jobs got it right with Just showing icons for launchers, and then when you launch them they get an arrow to show they're active, and if you click it it shows the windows. Of course there is expose for kde, and its great, but I think kde guys should try to think something to improve the taskbar hunting hell...
virtual desktops helps... sometimes... sometimes its more dificult to look for each desktop looking for the window you want.

My biggest friend now is expose for kde because the taskbar is so slow to use.

Let's learn from the masters! KOOLDOCK for everyone, or something even better? Some brainstorming in this area is needed!!!!

Feel free to make some mockups :-)

by Vladsinger (not verified)

It would be nice if this could be implemented in a way that looks a bit more like the mockup, with a clean, borderless menu. That would be perfect.

by Jonathan (not verified)

It's look will be plasma's job. Just wait for it, I'm sure it will come