MAY
11
2006

Phonon and the Future of KDE Multimedia

Linux.com is running a very informative article on Phonon, the new multimedia layer for KDE 4. It explains the rise and fall of Phonon's predecessor aRts and elaborates on the ups and downs of an audio abstraction layer. The article also gives an overview of common use cases and provides some example code. The Phonon website itself provides more code examples and documentation for using the Phonon API and for writing a backend. In addition to the existing NMM backend by Bernhard Fuchshumer, Tim Beaulen is working on a Xine backend for Phonon.

Comments

Meanwhile, quite a debate is raging in blogospace:

First, Christian Schaller (a GStreamer dev) explains why he thinks Phonon is a dead end: http://blogs.gnome.org/view/uraeus/2006/05/11/0

Many replies are on that weblog itself, but since blogs are the new usenet as they say, there are also separate weblogs replying:

Aaron Siego (no introduction needed, right?), for instance: http://aseigo.blogspot.com/2006/05/id-like-another-black-eye-please.html, who comments on how KDE is not planning to make the mistake it made with Arts again. And on http://amarok.kde.org/blog/archives/91-Backends,-Phonon,-GStreamer.html, Ian Monroe, an Amarok developer, comments on the real world issues with supporting multiple MM frameworks as encountered in Amarok. On http://www.kdedevelopers.org/node/2007, last but not least, Scott Wheeler comments on some of the reasoning (back in 2004!) that has ultimately lead to the creation of Phonon, and how the points raised then are still valid today.


By Andre Somers at Thu, 2006/05/11 - 5:00am

Christian Schaller arguments against Phonon are not valid.
He basically says: You've high-level abstraction, but what to do if you want to get down to the bits and bytes level?

Answer is simple:
Imagine you use GStreamer + Phonon.
All the basic apps like your media-player, your VoIP-Software, System notifications can rely on Phonon and be sure to work in _every_ environment. But you want to use some special audio-editing software, too. This audio software is relying on gstreamer directly. So no problem. Everything will work fine, without getting in the way of each other.

What if your special audio application uses backend XYZ? Simply choose XYZ as backend for Phonon and your "normal" desktop apps will continue working while having your audio-app open.

For me, Phonon is really the way to go!


By thomas at Fri, 2006/05/12 - 5:00am

No, that won't work. Phonon is a simplified wrapper for various backends. As such, it will provide the lowest (or at least, a relatively low) common denominator of features. It simply won't support everything you can do with a particular backend, even if you're using the best one.

For me, supporting many backends is a mistake. It makes it easy for people distros, but end users don't care; they want the best stuff, and their distro should take care of providing it reliably. KDE should choose best-of-breed technology, and adopt it wholeheartedly, rather than hedging its bets and being average.


By Lee at Sat, 2006/05/13 - 5:00am

> KDE should choose best-of-breed technology, and adopt it wholeheartedly,
> rather than hedging its bets and being average.

That's what they're doing. They decided to not rely on the broken GStreamer framework - which by the way is developed since 1999 (7 years) and still offer nothing that end users can use reliable enough.

I've been struggling with GStreamer since it became part of GNOME. It initially sounded like a good idea (and still is) but the implementation inside GNOME (3rd party components) is hackish and incomplete (from a developers point of view) and the easy of use is not given since I encounter crashes since the whole mess became part of GNOME. GNOME is known for hallfassed implementations and incomplete features. Why adopt mistakes from one desktop over to another one ?


By avatar at Sat, 2006/05/13 - 5:00am

The idea is sound. At the moment, lots of different multimedia projects exist, reproducing the same efforts to make codecs etc. This is an insane repeatition of previous mistakes, where proven approaches have already existed for decades. GStreamer is the closest we have to that approach, and if we all get behind it, then free desktop multimedia will focus and improve rapidly.

I don't like the idea of coding multimedia to a GNOME-like C api either, but sooner or later we'll all have to grow up and work together. Frankly, it's been too long in coming already.


By Lee at Sun, 2006/05/14 - 5:00am

GStreamer being the closest is an opinion. And by using the Phonon approach KDE developers can still contribute to GStreamer without having to make a fork when they break their ABI compatibility.


By Corbin at Sun, 2006/05/14 - 5:00am

We had the same problem with printing and the problem was solved.
Phonon is the way to go.

further software patents are a real risk and dependencies on one single engine could create real trouble due to legal risks.

One issue e.g. what concerned me in Suse 10 was: When I play an mp3 with Amarok and then play an mp4 audio (default format in iTunes I guess), it opens in Kaffeine, so I listen to two audio files at the same time. This should not happen.


By Hup at Thu, 2006/05/11 - 5:00am

It would be nice if applications like these could (optionally!) play nice with each other. In case of music or videoplayers this could mean requesting other players to fade out to a pause if the user starts playback of a song or movie, but in case of an incomming VoIP call the volume could just be turned down a bit, for instance. Would be cool...

Still, it should be possible to just play two or more songs at the same time. If somebody want to do that, why not?


By Andre Somers at Thu, 2006/05/11 - 5:00am

"but in case of an incomming VoIP call the volume could just be turned down a bit, for instance."

Thats actually going to be a planned feature of Phonon. Applications can categorize themselves into categories, like Communication, Notifications, sound... player... thingies (I don't remember what they are official, but stuff like that).


By Corbin at Fri, 2006/05/12 - 5:00am

I thought that was for things like routing sounds to a specific piece of hardware? Ah well, maybe it can and will be used more broadly for features like these. Stuff like fading out an allready playing song when starting playback on a new one could then be just policy for that category, as a service provided by Phonon... Hmmm... Sounds pretty cool to me! Go Phonon! :-)


By Andre Somers at Fri, 2006/05/12 - 5:00am

This was discussed in kde-artists before it went down. But I hadn't heard it was actually being planned. Can you confirm that or point me to the docs that say it? I was the one who proposed it, so this might be a bit biased, but if they're planning to do that through Phonon, then that's a unique feature that would justify a new layer, I guess.


By Lee at Sun, 2006/05/14 - 5:00am

http://developer.kde.org/documentation/library/cvs-api/kdelibs-apidocs/p...

"These categories can also become usefull for an application that controls the volumes automatically, like turning down the music when a call comes in, or turning down the notifications when the media player knows it's playing classical music."


By Corbin at Sun, 2006/05/14 - 5:00am

You can't escape software patents damages. When they will decide to sue you if you decode mp3, no backend will save you (the same for jpeg and incredible number of audio/video/file format).
When they will enforce the patent of "lissening music with a computer", no frontend will save you also.
The only answer to software patents is... not let them be allowed in your country!
So never give up in fighting them, and spread consciousness about the incredible risks among developers, that too often seem not to understand the danger.
http://wiki.ffii.org/SwpatcninoEn
http://www.nosoftwarepatents.com
http://www.ffii.org


By Marco Menardi at Thu, 2006/05/11 - 5:00am

Xine has a license advantage to GStreamer: Xine is licensed under GPL and enjoys the full copyleft protection of the GPL.

The Free Software Foundation warned against the use of the LGPL, but GStreamer developers didn't listen -- they instead choose to sell out their users to the entertainment cartel.

Xine will always be able to remove the DRM from any crippled multimedia plugin, since the GPL ensures that all plugins must be Open Source.

I encourage all Freedom-conscious developers to stop working on GStreamer, NMM, Helix or any other backend that allows the Media Mafia to ram DRM down consumers’ throats. Work for Freedom, help out Xine!

Go Xine, Go GPL, Resist draconian DRM!


By ac at Fri, 2006/05/12 - 5:00am

In case you didn't notice, Helix is available under the GPL. I don't know about NMM.


By Shriramana Sharma at Fri, 2006/05/12 - 5:00am

Helix is LGPL, and thus can include proprietary, non-free, closed source, DRM components that cripple your fair use rights. DRM in Helix is not just theory -- it's reality:

https://devicedrm.helixcommunity.org/

Just a couple of days ago, Real's executives made a push for DRM in the Linux kernel, saying that otherwise Linux will die:

http://linux.slashdot.org/article.pl?sid=06/04/11/152234


By ac at Fri, 2006/05/12 - 5:00am

Yep, helix sucks. From day one on Linux, they've done nothing that cooperates with Linux multimedia efforts (such as GStreamer), except where it makes their own product more famous, since their windows market is dying quickly.


By Lee at Sat, 2006/05/13 - 5:00am

"The Free Software Foundation warned against the use of the LGPL, but GStreamer developers didn't listen -- they instead choose to sell out their users to the entertainment cartel."

I agree actually. You can't reconcile the two. Does free software exist to allow usage without pointless and artificial restrictions, or does it exist to enforce those restrictions by allowing people to put them in? Xine is also a quality back-end.


By segedunum at Fri, 2006/05/12 - 5:00am

Linux needs DRM to survive in the home.
DRMs are neither a good nor a bad thing as long as you know what you are getting.
What linux users are certainly not getting today is any commercial content (unless you steal it). Linux in the work place or the geeks bedroom my not need it, but I wonder how long it will make sense to use in Linux in the living room?

HB


By Hotbelgo at Sat, 2006/05/13 - 5:00am

No. DRM is a slippery slope, which will change everything if you give an inch. DRM *IS* bad; it enforces "rights" that content producers aren't legally entitled to, and thereby illegally infringes on users' freedom. Under no circumstances should anyone give into that just because it makes it easier to watch another holywood moneyspinner with no real depth. There are already real alternatives, like podcasting and vodcasting, which will give you more to watch and listen to than ever before, without giving up your rights.


By Lee at Sun, 2006/05/14 - 5:00am

Look, when you use a stick to lever a rock, you either move the rock, or break the stick. DRM is too heavy a rock to be moved by Linux stick, so please, don't break what market share Linux has by stripping ability to play DRM media from Linux.


By ac at Sun, 2006/05/14 - 5:00am

There is a simple reason why to not integrate DRM into linux-kernel: Linux is Open Soucre and under the GPL.

A license text does not stand above the the applicable law, and in some nations (e.g. Germany) it is a law that you are not allowed to write software or alter software in a way that makes it able to bypass a DRM system.

If we integrate a DRM system into OSS than we defacto destroy its OSS status.

So even without the GPLv3 DRM-terms DRM and OSS are implicitly incompatible.

Eagles


By Eagles at Sun, 2006/05/14 - 5:00am

Unfortunately, MP3 is NOT a software patent. It is a patent for a compression algorithm. As I see it, the problem is not that the patent exists but that their licensing method doesn't work for OSS. That is, I have no objection to paying Thomson my $2.50 for a license for the Fraunhofer patents. The problem is that they don't do business that way.


By James Richard Tyrer at Fri, 2006/05/12 - 5:00am

"Unfortunately, MP3 is NOT a software patent. It is a patent for a compression algorithm."

True, and that's a huge miscpnception people have. Because it is a patent for a compression and audio algorithm, what on Earth can you apply it to? Can you apply it to a format that perhaps uses the same principles? The water just keeps on getting muddier.

"That is, I have no objection to paying Thomson my $2.50 for a license for the Fraunhofer patents. The problem is that they don't do business that way."

True, and it's a trap open source projects have fallen into to the total detriment of everyone.


By segedunum at Fri, 2006/05/12 - 5:00am

One issue e.g. what concerned me in Suse 10 was: When I play an mp3 with Amarok and then play an mp4 audio (default format in iTunes I guess), it opens in Kaffeine, so I listen to two audio files at the same time.

This should not happen.

It's easy: you stop the playing in Amarok, then you play your other file in Kaffeine. Since you're already listening to the first one, you'd have to be quite dumb not to realise that there's something being played.

Plus, what prevents you from changing the order of programs associated with mp4 so that it's played in Amarok instead of Kaffeine?


By Obviously anonymous at Thu, 2006/05/11 - 5:00am

mp4 is usually a video format, so Kaffeine is just fine -- as long as it is video.

It goes like this, you click on the file in your playlist and it opens with Kaffeine.

"you'd have to be quite dumb not to realise that there's something being played."

Come one, you don't manually stop song a and then play song b. you are playing song a and then want to listen to a special audio file (who cares about codecs) b while song a is still playing. As you usually only listen to one song at one time, the default is that your music player stops song a and starts song b. This is the way it *always* works.

That it opens with Kaffeine is no problem, but that Amarok does not stop playing the former file a automatically is. Same goes usually for videos. You don't want to listen to Beethoven and watch a video from Linuxtag together, at least my multitasking capabilities are limited.

It is as you point out just an annoyance, no real problem which could not get fixed by personal activities. But it does mean that I don't like sound on Linux which is not fully ready for me for that small usability annoyance.

A common sound framework is helpful here to improve things.


By hup at Fri, 2006/05/12 - 5:00am

Just for your information: The example code in the article is from the ArtsPlayer class in JuK. Of course there's a lot more code around that (for example the setVolume code in the original ArtsPlayer expands to some heavy code for inserting the volume control into the signal path). Phonon hides all those details in the backends and, of course, remembers the output volume between media files.


By Matthias Kretz at Thu, 2006/05/11 - 5:00am


By Matthias Kretz at Thu, 2006/05/11 - 5:00am

WOW!


By thomas at Fri, 2006/05/12 - 5:00am

That's really impressive. There is one thing I'm wondering about, shouldn't the destructor delete the m_media, m_path and m_output objects created in the constructor? Or is there some sort of parent/child relationship going on.

Either way, I think Phonon is looking very good.


By Bryan Feeney at Fri, 2006/05/12 - 5:00am

You guessed right: All three objects inherit from QObject and are hence deleted as soon as the parent gets deleted.

Cheers,
Daniel


By Daniel Molkentin at Fri, 2006/05/12 - 5:00am

Those two implementations aren't comparable. In the artsplayer::play() method, there are checks for null which handle it and proceed, while in the phonon implementation, the method just returns without doing anything, as if it wasn't even called. Furthermore, the artsplayer implementation seems to handle that engine being disfunctional, whereas the phonon implementation presumably handles multiple engines internally, but is setup and checked for functionality elsewhere in juk. Having said all that, it does look a little nicer to work with.


By Lee at Sun, 2006/05/14 - 5:00am

Christian Schaller of Gnome is not happy about Phonon, according to this rant on http://blogs.gnome.org/view/uraeus/2006/05/11/0 :

quote:

..My final objection to Phonon is that even if they manage to prove me wrong on their ability to provide a truly useful limited cross framework API and demonstrates that having a menu option offering your grandma to play her music using framework X,Y or Z actually solves more problems that it creates, I still think that it falls short. Because it wouldn't provide an API to do applications like Pitivi, Diva, Jokosker, Buzztard, Flumotion and so on which I think is where we want to be at today in order to provide a competitive desktop. MacOS X and Windows Vista are showing us that this is the role that the desktop is heading towards.


By AC at Fri, 2006/05/12 - 5:00am

What still I wonder is: there will be mutual exclusion? or media things can co-exists?
From what I understood, phonon will become the media server of kde4 every audio and video apps will output to it, and then everything will pass to the choosen media engine, that can be system-wide or application specific right?
Then non kde apps how can play and connect to their media engine?(for gnome gstreamer i suppose) and then particular apps, like music software, that most of time need realtime support can continue to use directly the hardware, or use a low liatency server like JACK?
This is a little confusing to me, how those different media interact with each other? (note I'm talking on linux platform since it's the one I use)


By ra1n at Fri, 2006/05/12 - 5:00am

1. Phonon is not a server, it's a thin wrapper API around whatever media framework you like (btw: gstreamder doesn't have a sound "server" component either.

2. Thanks to alsa dmix, it is possible to run multiple frameworks side-by-side. Thus there is no reason why phonon can't coexist peacefully with each of the multimedia frameworks when used via their native APIs.

No server => no significant latency. I have no deeper knowledge about jack and gstreamer but from what I understood you could have a jack sink in gstreamer that just pours the sound into jack. But again: there is no need for that if you use alsa on linux (which all modern distributions do).


By Daniel Molkentin at Fri, 2006/05/12 - 5:00am

Right, its just as problematic to run two gstreamer apps as it is to run a gstreamer and a xine app. And its only a problem if you don't have dmix or a modern sound card.


By Ian Monroe at Sat, 2006/05/13 - 5:00am

Reading all that noise sounded familiar to me. And then I remembered: Corba vs DCOP. When KDE chose DCOP over Corba, exactly the same kind of objections were reaised, the same kind of noise was made. We were dooming the project, Corba is better, DCOP will never make it, you need a super good IPC mecanism that must be hardware independant, programming language independant, network transparent or you are dead. The overhead of maintaining DCOP will kill you while Corba and orbit is there.

The result:
4 years later, Gnome has struggled with Corba to the point where they almost ditched bonobo. Very few applications in Gnome use the features that super Corba powered bonobo was supposed to provide. On the other hand, DCOP has been so successful that it was picked up as a basis and rewritten by freedesktop folks to provide a wider linux/unix IPC mecanism. All KDE applications support DCOP transparentely and you can do a lot of cool stuff with it.

So, let them talk, let them brag, let them predict the doom of KDE and its multimedia framework. KDE developers have shown that they know how to pick up excellent technical solutions that last over the years (arts being a notable exception). Phonon is the way to go and the people who refuses to see it now will be happy that KDE did that in a few years from now.


By Philippe Fremy at Fri, 2006/05/12 - 5:00am

Agree!

Normal users of DE like me, like the work KDE devs are doing. Keep doing what you believe is right. Phonon will turn out to be an elegant piece of software.

I am sure there are many like me who watch silently from the sidelines but trust the KDE devs to do the right thing. We are with you.

I have installed kde as the default DE on all my customer machines (even on RHEL) and they all like it. I have NEVER had a complaint.


By aries at Sat, 2006/05/13 - 5:00am

Wow! Corba, in the early days of KDE2 development... That seems so long ago! And by the way: You are absolutely right. What a good example. Take that gnomies.


By TheTeeBat at Sat, 2006/05/13 - 5:00am

what is the picked up & rewritten implementation of DCOP at fd.o?


By me at Wed, 2006/05/17 - 5:00am

He's talking about D-BUS (http://www.freedesktop.org/wiki/Software_2fdbus).
(He didn't mean a 1:1 copy though.)


By cm at Wed, 2006/05/17 - 5:00am

@Thomas Vander Stichele

Please, do not repeat again that crap about "GStreamer 0.10 begin API/ABI stable". Just a few days ago you yourself (http://permalink.gmane.org/gmane.comp.video.gstreamer.devel/15342) made a proposal to break API/ABI compatibility in 0.10!

That's exactly why Phonon is the right choice for KDE4. It may be right for Gnome developers to change their software everytime GStreamer changes API/ABI (which is too often, sadly), but it's not for KDE.

If KDE used GStreamer, either KDE app developers stick with GStreamer 0.10.5 for the whole KDE4 lifetime, or KDE app developers change their apps for GStreamer 0.10.6, 0.12, 0.14, etc.

GStreamer releases so many versions with so many changes so often that it's a very fast-moving target, and that's definitely BAD.


By An Onymous at Sat, 2006/05/13 - 5:00am

Heh yea, its kind of ironic that the gstreamer devs would want KDE4 to use gstreamer 0.10. If it actually did, it would mean KDE pretty much forking gstreamer when gstreamer's development moved on in a few years.


By Ian Monroe at Sat, 2006/05/13 - 5:00am

I don't see the problem here. If a new version of GStreamer comes out with a new API, then it's up to KDE to port to it, like they would for any other new API, like CUPS etc. This talk that's going around in blogs, that GStreamer folks should maintain KDE's audio layer etc. is totally backwards.

Now, if KDE needs a certain interface for the lifetime of KDE4, all they have to do is open a specific gstreamer library version. It has always been the way, that a simple symlink lets that requested version point to the latest compatible version. And if building with multiple gstreamer versions present is an issue, it only takes an argument to configure, to say where the older header files are located.


By Lee at Sun, 2006/05/14 - 5:00am

Hmm, I think you should go and care for your own business. The KDE people have decided to use Phonon and that's it. We should stop forcing a broken multimedia framework like GStreamer down the throats of users.


By avatar at Sun, 2006/05/14 - 5:00am

> If a new version of GStreamer comes out with a new API, then it's up to KDE to port to it

Having the layer called Phonon and porting only that layer is an efficient
way of doing just that.

> Now, if KDE needs a certain interface for the lifetime of KDE4,

Yes, that's a "hard" requirement.
The KDE team cannot automagically port any third party
apps that it has promised ABI stability to. Not without Phonon.

> all they have to do is open a specific gstreamer library version.

That would mean missing any improvements not done in that
old library version, e.g. performance or bug fixes.

- Who wants to support a five year old version?
Does the GStreamer team guarantee that long a support? I don't think so.

- Can the KDE team do it? Not easily, as it's unfamiliar C code and
maintaining a multimedia framework is not necessarily everyone's forte.

- Would maintaining an otherwise obsolete version be a duplication of effort
(that's what some ignorant people keep accusing the KDE project of)? Hell, yes!


By cm at Sun, 2006/05/14 - 5:00am

I don't know why this is, but xine has always worked best for me. It's either an accident or not an accident. In case it's not an accident, please keep xine available.


By xin at Sun, 2006/05/14 - 5:00am

Same here, Xine is the only backend thats at all reliable for me in amarok on both of my systems (gstreamer has NEVER worked once), also Kaffine has always been great at playing back at any movie I throw at it (using the Xine backend of course). I've often wished that I could replace aRts on my laptop with a xine based program (when arts plays a sound its all crackily, but not when played with a Xine based player).


By Corbin at Sun, 2006/05/14 - 5:00am

The problem is, xine is just a playback framework. It doesn't help people who want to do other multimedia things, like actually make a video. And, mplayer duplicates much of that effort. GStreamer, on the other hand, breaks this all down into components, so that people who know how to encode/decode Quicktime really well can do that, and people who know MIDI can do that, etc. Even if each person has their own project, like Amarok, or Xine, or Blender, they could all contribute whatever improvements they want to gstreamer, and EVERY project would benefit. So, having that common multimedia layer that everyone puts their weight behind is much better in the long run, even if it takes a little longer to make it really come to life. It's been too long already; we really need to start working together on this.


By Lee at Sun, 2006/05/14 - 5:00am

> GStreamer, on the other hand, breaks this all down into components

The thing is.. GStreamer is broken.. It's in development for 7 years now and still is nowhere where people can rely on it or use it.


By avatar at Sun, 2006/05/14 - 5:00am

Pages