Phonon: Multimedia in KDE 4

Thursday, 27 April 2006  |  Mkretz

After many months of work on the new Multimedia API for KDE 4 it is time to finally announce Phonon. Phonon will provide a task oriented API for multimedia, making it easy for KDE applications to use media playback and capture functionality (and more) resulting in application developers being free to concentrate on the user interface aspects. The number of possibilities to integrate multimedia into the desktop experience make Phonon especially interesting.

Phonon uses exchangeable backends to do the real work which can be implemented using GStreamer, NMM, Xine, Helix or whatever else you can come up with. In turn KDE applications do not need to develop media engine abstractions anymore as Phonon provides it for them.

The folks from Motama have already started work on a Phonon-NMM backend and we will give a joint presentation at upcoming LinuxTag this year (also see their announcement). Meet us on Saturday, May 6th at 10:00 in Hall 6.2 or later at the KDE booth.

As there is still a lot of work that needs to be done until Phonon is released with KDE 4.0 it is a great chance to get involved now, for example doing a Google Summer of Code project.

Phonon is supported by basysKom GmbH, Motama GmbH, KDE-NL who are organising a KDE Multimedia meeting and most importantly the community for providing feedback & code.

Comments:

Video? - liquidat - 2006-04-27

If you do this presentation, please, please include a Video for the people who cannot come to the LinuxTag - it would be great to see something from Phonon in real life :)

Re: Video? - Matthias Kretz - 2006-04-27

This is out of my control. Last year there were cameras to record the talks, but I never saw the files for download. I guess it will be the same this year.

Re: Video? - somekool - 2006-04-30

same for me, I'm in Japan, I really would like to watch the presentation. please.

Streams - David - 2006-04-27

Hey, the website looks really good! I was wondering if it is also going to be possible to write a phonon program that gets frames from a source (mic or just a file) and then the possability to modify the data (for instance real time effects) and then send it back to phonon, like jack. Do you know if this is going to be possible?

Re: Streams - Matthias Kretz - 2006-04-27

> Hey, the website looks really good! Kudos to Nuno Pinheiro. He really did a great job on the artwork! > gets frames from a source (mic or just a file) and then the possability > to modify the data (for instance real time effects) and then send it back > to phonon Capture will be supported, an interface for getting the audio data is there already, I'm working on the video data interface now. Feeding it back into Phonon is technically possible, but I don't think it's a good idea. If you want to do processing of the audio or video data it should be done on a lower level. Phonon supports to use effects for audio and video for that.

Thank God - RJakiel - 2006-04-27

PLEASE tell me when this comes to pass arts will be dead and gone. If so that will be the best news I have heard in a LONG time.

Re: Thank God - FreqMod - 2006-04-27

I think arts will stay in KDE 3 because of binary compatibillity, and then it will be replaced in KDE 4.

Re: Thank God - RJakiel - 2006-04-27

That's what I meant. KDE 4 + Phonon = Death of arts, if that is the case I will be too excited for words.

Re: Thank God - Narishma - 2006-04-27

That's the case.

Re: Thank God - anonymous - 2006-04-28

Not at all. aRts is still a sound server, and if there will be a Phonon backend for aRts, then aRts could serve as a sound backend for a KDE 4 environment.

Re: Thank God - SadEagle - 2006-04-28

aRts is unmaintained. Unless that changes, I don't see how that would be such a hot idea.

Re: Thank God - Steven Brown - 2006-04-28

Arts/esd are already dead. Current versions of ALSA come standard with dmix enabled (previously, you had to configure it which not everyone did) so ALSA-using programs can already multiplex sound just fine. Combine that with libao with an ALSA backend for platform-independent traditional audio and GStreamer with an ALSA backend for platform-independent modern audio and you're all set. Note that for some reason the OSS emulation of ALSA isn't able to be multiplexed, so make sure you aren't running apps that use the OSS devices like /dev/dsp.

Re: Thank God - kaiwai - 2006-05-05

What about those who don't use Linux; such as me? I run KDE on FreeBSD, and I think that the abstraction between the soundcard/lower levels of the sound framework, and the desktop is very important. If you don't maintain this abstraction, you'll have for ever and a day, people writing directly for ALSA, that will not only cause problems for ALSA later on, when it wishes to make radical moves, but also those whose platforms don't have ALSA, and would require major application re-working to get it running with their OS of choice. Remember, KDE is a desktop environment for UNIX, not just Linux.

Re: Thank God - mikeyd - 2006-04-28

Why? As a user, arts is wonderful.

Simultaneous requests from different users - ac - 2006-04-27

How will Phonon deal with simultaneous requests from the same user and simultaneous requests from multiple users (ie. two users logged in to different X sessions by "Switch Users" or by "su - " on the command line)? Currently, when a KDE alert appears it plays a "gong" sound. However, if amaroK is playing while the alert appears, the "gong" is only heard _after_ amaroK has finished playing the whole song! Another problem I've encountered is that when two users are logged in, only the first user can play sounds.

Re: Simultaneous requests from different users - superstoned - 2006-04-27

2 requests from the same user should work if he uses ONLY phonon: if you use arts in amaroK, you'll hear the sound immediately. the problem is, most users don't use arts in amarok. if you use gstreamer in amarok and aRts or whatever in another app, they can't play at the same time. phonon will not fix this - but at least all KDE apps will use the same backend (unlike now) so it will get better. i don't know about the two-user problem, but i really think phonon SHOULD try to do something about it.

Re: Simultaneous requests from different users - Jakob Petsovits - 2006-04-27

The amaroK thingie is for sure going to be solved with Phonon. It stems from the fact that amaroK uses a different sound engine than the standard KDE one, so all the KDE sounds have to wait until the amaroK sound backend releases its claim for the sound card. With Phonon, all of the sounds (including amaroK's) will be redirected to the same sound engine, so all of the applications can make sounds at the same time. (This should also work with non-KDE apps.) About the user switching use case, I don't know if this is going to be tackled.

Re: Simultaneous requests from different users - Ian Monroe - 2006-04-27

This isn't really the case. Phonon won't be a sound server. What has actually solved the issue described by the parent post is that ALSA now has software sound mixing and most sound chips (even those builtin to the motherboard) have hardware sound mixing. When aRts was created this wasn't the case so a sound server made a lot of sense. This isn't really the case anymore. The backend to the Phonon system could be a sound server (aRts, NMM) but it doesn't have to be (Xine playing to ALSA isn't).

Re: Simultaneous requests from different users - Corbin - 2006-04-27

You need to setup ALSA to do software mixing on your computer because it seems like your sound card doesn't support hardware mixing. You can have ALSA do software mixing by setting up DMIX yourself, or upgrading to a rather recent version of ALSA (which has it enabled by default). If you're using OSS (Open Sound System) still, you have to upgrade to ALSA. This only applies to Linux, if you're using *BSD or another *nix then you may be out of luck (I don't know if they support software mixing, or how to set it up). Mixing sounds coming from multiple applications isn't (shouldn't be) the job of a userland sound server (like aRts), but really should be on the other side of the sound API so the applications don't have to know or deal with it.

Re: Simultaneous requests from different users - Corbin - 2006-04-27

To actually answer your question (guess I kinda forgot to), if you're using software or hardware mixing, then phonon doesn't need to do anythin special to allow sounds to play at the same time, in fact you'll be able to have multiple apps/users output directly to ALSA and not have any problems (you can't do that with OSS). I don't believe Phonon will include anything special to solve the problem people that don't have hardware/software mixing have.

Re: Simultaneous requests from different users - Diego - 2006-04-27

As I see, phonon will replace arts, but with improvements, trying to standarize multimedia apps. About the hw/sw mixing, it all depends on what sound card you use and NOT if you use ALSA or OSS. Nvidia Nforce2 sound card uses an OSS driver and has hw mixing, i.e., you can use xmms, amarok, flash, mplayer and xine at the same time regardless what backend they use. I think this matter is way beyond Phonon scope.

Re: Simultaneous requests from different users - Ian Monroe - 2006-04-27

Well, ALSA can do software mixing (dmix). So it does matter if you use ALSA or OSS.

Re: Simultaneous requests from different users - BSD Nerd - 2006-04-29

How to enable multiple sound sources simultaneously in FreeBSD is described in the FreeBSD handbook (online) chapter 7.2.3. It's easy :-) Write the values to /etc/sysctl.conf to make them permanent.

Re: Simultaneous requests from different users - mikeyd - 2006-04-28

Change amarok to the arts engine, then it works.

Project scope? - Simon Edwards - 2006-04-27

Phonon developers, How does the scope of the Phonon API compare NMM, Jack, gstreamer etc? What is the philosophy here. Does Phonon aim to provide a base level of multimedia support aimed at most 'simple' applications, and if you need advanced features do you then by pass Phonon and code on the API from NMM, Jack and friends? sorry if this is a FAQ. -- Simon

Re: Project scope? - superstoned - 2006-04-27

phonon USES NMM, Jack and Gstreamer. Phonon just makes it a lot easier for application developers to add rich multimedia features to their applications without having to worry about gstreamer, NMM, Jack or whatever. and you don't bypass Phonon, Phonon should be powerfull enough in its own right!

Re: Project scope? - mabinogi - 2006-04-28

Well no. Especially not when it comes to Jack. Not every media library has the same goals, and Phonon shouldn't try to be a superset. It should try to be a reasonable subset that covers the common cases. It's not the desktop environment's job to provide audio and video capabilities.

Re: Project scope? - Matthias Kretz - 2006-04-27

The Phonon API is a high-level multimedia API, NMM and GStreamer are rather low-level multimedia APIs while Jack provides an audio in-/out API especially geared for pro-audio needs. The idea is to make multimedia application development easy. The average developer doesn't know what a (de)muxer is or how he needs to set up the flow graph/pipelines correctly. Like I said in the article, the API is designed from looking at common tasks ("task oriented API") instead of looking at what features can be provided. In the end that means, that some things which are possible with GStreamer will not be possible with Phonon. But yes, in the cases where the Phonon API is really too limited (you should first ask me before you decide whether the Phonon API is not good enough for you) you can use whatever else you want. The only thing to take care of is that the soundcard output works nicely together with Phonon applications. I guess I should come up with a Do's and Don'ts document for such a case.

Re: Project scope? - Janne - 2006-04-27

"The Phonon API is a high-level multimedia API, NMM and GStreamer are rather low-level multimedia APIs" Why do I get the feeling that we have Phonon, which talks to Gstreamer/JACK/NMM/etc. and they talk to ALSA. So that's three layers on top of each other. Doesn't that make debugging harder? How about bloat?

Re: Project scope? - hugelmopf - 2006-04-27

Adding overhead by yet another layer (besides ALSA and gstreamer/NMM/...) is exactly what I was wondering about. Is it really that hard for developers to use gstreamer & co. for playing audio/video? Or is there another reason for wrapping it? I can't imagine that another layer is going to simplify anything for the user, but you may prove me wrong ;-)

Re: Project scope? - anon - 2006-04-27

Gstreamer sucks, so right there is one good reason to have a wrapped API. That way when the next overhyped beta-quality media engine get shoved in people's faces, it should be easy enough to switch to it. Then switch back to Xine.

Re: Project scope? - Ian Monroe - 2006-04-28

And even if you think gstreamer is great, it might have an ABI change before KDE 5.0.

Re: Project scope? - Brandybuck - 2006-04-28

It's a GNOME project, so will most likely have an ABI change before KDE 4.0.

This thread made me laugh. - sundher - 2006-04-29

So funny and so true. Xine rocks.

Re: Project scope? - meatman - 2006-04-28

Gstreamer isn't to bad the latest release works well and even surpasses Xine in some areas, Xine is a pain to get "win32" codecs running with sound. With GStreamer all you have to do is install the "Ugly" branch. Xine still is a great, don't get me wrong but. Gsteamer right now I think is a good framework

Re: Project scope? - Ian Monroe - 2006-04-27

Its not like Phonon is some sort of running process that has to "talk" to xine, gstreamer etc. It just layer of API in keeping with good OOP design. So, it adds a couple of extra function calls. The ability to change multimedia backends while retaining binary compatability is a large benefit. We don't want KDE 4.x to be stuck on one multimedia system like KDE 3.x is stuck with aRts.

Re: Project scope? - Matthias Kretz - 2006-04-27

First: "talk" is a term which suggests that it takes a lot of time for the Phonon -> GStreamer -> ALSA "communication", which doesn't have to be that case (of course it can be slow if you program it like that). And it would make debugging harder if those were processes (or threads), but actually debugging will be easier than with aRts, and not harder than without Phonon. The term bloat is unjustified. Phonon at this point has about 9500 lines of code including example code and the fake backend. libphononcore has 189892 bytes (stripped) in comparison to 2000212 for libkdecore or 385852 for libkutils.

i'm impressed - superstoned - 2006-04-27

really. look at these sites: http://phonon.kde.org http://plasma.kde.org http://solid.kde.org don't they all look BEAUTIFUL? really wonderfull, professional and such? respect to those who made them... i think all KDE sites should use this look'n'feel, even if there are differences like between phonon and the other two - they don't have to look exactly the same, just similar.

Re: i'm impressed - Ian Monroe - 2006-04-27

I agree, http://www.kde.org could learn something. :)

Re: i'm impressed - pinheiro - 2006-04-27

:) thanks i feel embaressed we in oxygen work very hard to make kde visual elements beter and beter. We are working on lots of other kde related sites and i can promisse kde is loking beter by the day.

Re: i'm impressed - superstoned - 2006-04-28

you should feel proud of yourself :D

Re: i'm impressed - Psiren - 2006-04-28

My issue with the phonon site is the fixed width, requiring me to scroll left and right. Very little point, considering the lack on content in it. It may look good, but for me it's unusable.

Re: i'm impressed - pinheiro - 2006-04-29

Has monitors get biguer and bigher the fix withe is imo the best aproach try to read a page on a 1900 screen in full screen mode, you get intire text's in one line and you constantly lose the starting point couse your eys lose th reference points.

Re: i'm impressed - Jimmy - 2006-04-29

I think 59em must be the optimum width for a page, because then you can fit ten words per line, which apparently is easiest to read. And the average word length is apparently 5 characters. So 5 characters times 10, plus 9 spaces would be 59em.

Re: i'm impressed - Corbin - 2006-04-29

If you're scrolling on a monitor with the resolution set to 1024x768 and an almost full screen window thats a bit too large, considering most people still use 800x600. Other than making you scroll on a small monitor I really love the new sites (the art is really nice, I can't wait for Oxygen!).

Re: i'm impressed - Gary Greene - 2006-05-04

Actually, I'd have to disagree about the 800x600 comment there.... Most I've seen in the last two years are at 1024x768, as monitors (LCD and CRT) have INCREASED in size.

Re: i'm impressed - Kolbjørn Barmen - 2006-05-03

I'm not impressed, what a waste of space. And it's not close to validating, but I guess that's part of having "professional look". I just hope they're not indications on what will happen with KDE4

How to use - Koos - 2006-04-27

What is the idea with the central configuration and choosing the right backend. I can imagine that depending on a certain use, eg. local playback vs. web movies, one would like to choose for an different one. Or can/should a movie player query all backends and let the user decide? Btw http://developer.kde.org/documentation/library/cvs-api/kdelibs-apidocs/phonon/ui/ifaces/html/classPhonon_1_1Ifaces_1_1Backend.html link is broken. Who is handling network streaming for eg. mms and rtsp protocols, is that phonon or should the backends handle those (and what happens with password protected links then)? Is there a way to specify whether the backends should be in- or out-process, eg. for konqueror web browsing one might not want to pull in the gstreamer libs but for a movie player that might not be an issue.

Re: How to use - gerd - 2006-04-28

Everything should "just work", you should not even have to think about it

Re: How to use - Koos - 2006-04-28

That sounds pretty boring, count me out then :-)

Re: How to use - Carlo - 2006-04-28

While central configuration should of course be available, it should also be possible to override that per application, imho. It would be unfortunate, if the user would be forced to go with one backend for all (KDE) applications, instead being able to choose backend X for app A and Y for app B. There can be differences in functionality, e.g. supported (or licensed) codecs needed for a specific task compared to less cpu usage by another backend that suffices for usual tasks.

Re: How to use - gerd - 2006-04-28

All applications which ship with KDE should use one single interface backend.

Re: How to use - Carlo - 2006-04-28

I think I said already that this is probably not a good idea, since it would restrict all applications, which use Phonen to the limitations of one backend.

Re: How to use - ac - 2006-04-28

But having multiple backends concurrently will lead to the problem mentioned above, where one backend hogs the soundcard and the other backend can't play any sounds. For example, you won't be able to hear system notifications (beeps or gongs) that are outputted via one backend if you are watching a movie that is outputted via the other backend.

Re: How to use - Koos - 2006-04-28

Re-read earlier comments, that´s a sound server issue. If you have an uptodate linux system with alsa then it´s not an issue, else make sure to set up the backends to use the _same_ sound server.

per-application settings? - Andre Somers - 2006-04-28

I was wondering if phonon willmake it possible to have per-application settings for things like volumne and/or soundcard and channel on that card? I think something like that would be very usefull, enabling one to route system sounds to small speakers connected to the simple on board sound system, the music you are playing to the amplifier connected to your additional soundcard and your VoIP session to your headset, while giving them all their own volumne....

Re: per-application settings? - superstoned - 2006-04-28

theoretically, yes. and actually, i think they will implement these things.

Multiple soundcards? - Mere - 2006-04-28

How will phonon deal with multiple soundcards? I prefer to be able to choose which soundcard to use on a per-application basis. Will this be possible with phonon?

Re: Multiple soundcards? - superstoned - 2006-04-28

theoretically, yes. and actually, i think they will implement these things. :D

Mixers - a.c. - 2006-04-28

as the site says, the mixer is kind of pain. It strikes me that this should be broken into 2 parts; the input and the output. As in 2 different view or even 2 different apps. For the output, it should be nothing more than a balance and a total output. So imagine a trackball (a circle with a pin) where you can push the output and a side volume for total volume. Nice and simple; that is, set the volume, and then move the pin. Push it straight forward, and all the output goes to front. Push to right side, and output goes to right front/right back. Push it to NE corner and you only get right front corner. Finally, borrow from KNOB. that is a SIMPLE way to turn total volume up and down, with a quick mute. One last thing would be a checkbox to indiacte that the volume is either avg or max vol. i.e. hate it when I set the vol. and then an add comes increasing the DB by 10. (or 2 slides, but...). For the input, it sounds like you have the right idea; class of apps (defaults), followed by individual app overrides. It would be nice to provide some filtering ala arts (the idea was awesome; the imp was slow).

LOGIN Screen and screensaver - a.c. - 2006-04-28

It would also be nice if we can have sound at the login/screensaver. that way, either a screen saver can display or even a radio play in the background. I was even thinking that I might feed a baby monitor through it (for the parents here).

Re: LOGIN Screen and screensaver - AC - 2006-04-30

You can hack into the configuration of XDM/KDM and make a soundapplication start with it.

Also on FISL - Iuri Fiedoruk - 2006-04-28

During FISL I was present on a presentention of Helio de Castro who talked about Phonon, so I kind if knew already about this, and I even can answer the major question about it: No! It's not a engine like gstreamer, arts and such, it's more like a layer, you make your program for the layer, and it will call the apropriate (or chosen) audio engine. It's kind of like what amarok does for playing audio files. IMHO it's a good idea to make KDE NOT try to solve the linux audio/video problem, let someone else do it, because they probally do a better job, because they don't have an entire desktop to care.

Small typo - Hugo Costelha - 2006-04-28

Hi and congrats for the work torwards KDE 4.0. I just wanted to point out a small typo in http://phonon.kde.org/cms/1030 . In the last paragraph it says "There will probably more when Phonon", but probably should say "There will probably be more when Phonon". Regards.

Re: Small typo - Matthias Kretz - 2006-05-02

Thanks. I just fixed it.

Free Software - apokryphos - 2006-04-29

Excellent designs on these websites, but on a side issue: I think it would be nicer if these sites adopted "...it's a Free Software project" as the tagline on the bottom (as http://kde.org does). http://www.gnu.org/philosophy/free-software-for-freedom.html

Re: Free Software - Ian Monroe - 2006-04-29

Open source desktop is the term used for the conferences etc. It wasn't made up for the sites. :)

Re: Free Software - apokryphos - 2006-04-30

Not sure I follow... not saying the term was made up for the site, just think it'd be nicer to have "free software" instead of "open source".

Re: Free Software - AC - 2006-04-30

Why would it be nicer?

Re: Free Software - superstoned - 2006-04-30

because for many of us, the 'free' aspect is an important reason why we use Linux and KDE. also, it is just a wider and more encompassing term. KDE and linux are not *just* OSS, but they are FREE as in speech. “In a world where speech depends on software, free speech depends on free software.” — Don Marti jr. this is for me, and many other linux users, a important point. i value my freedom of speech, and i'm committed to keeping it.

Re: Free Software - Martin - 2006-05-02

The KDE.org site has no such tagline, as far as I can see.