SEP
11
2005

KDE to Migrate to bksys/SCons Build System

At the build system BoF at aKademy it was decided to start moving the KDE 4 build system from unsermake to the SCons/Python based system bksys. To find out more about this important future technology, KDE Dot News talked to its lead developer Thomas Nagy about the reasons behind the change and what it will mean for KDE developers.

Thomas is ita on IRC

Please introduce yourself and your role in KDE.

My name is Thomas Nagy (I am also known as 'ita' on
freenode). I have spent some time writing some KDE
utilities (solvers, small games) and adding
some features to KDE applications such as KOffice and Kalzium
when i was still a student. Since i graduated i have
spent more time trying to create complete applications
like kdissert, a mind-mapping tool for the creation of
documents and its build system bksys.

What is SCons and what is bksys?

SCons is a software construction tool which is used to
build and install programs. It is written in Python
and its design is modular, making it easy to extend
for use with unsupported compilers or complicated
build rules.

bksys (as in Build Kde SYStem) extends SCons and aims
at replacing the autotools. It has been written in the
main place for compiling and installing kdissert and
became more modular over the time. The most important
feature is the object-oriented style API that can be
accessed from either Python code or throughout XML
description files.

Some more documentation describing the framework can
be found in the the slides from my aKademy 2005 presentation.

Why is bksys separate from SCons?

bksys aims at replacing completely autotools for
common projects, so this goes way beyond SCons' main
goals.

In which way are they better than autotools?

There are many improvements over the autotools, but I
will try to describe the most striking ones:

  • Reliability: targets are rebuilt if compilation
    flags are changed or if md5sums of the files changed
  • Size: 100KB (miniSCons included) vs 600KB
    (autotools)
  • Simplicity: one language (Python) versus at least 3
    (m4, posix sh, make) for autotools (GNU/make,
    automake, autoconf, libtool); no intermediate file to
    generate for the compilation process (Makefile.in,
    Makefile)
  • Configuration: the configuration process is fast,
    easy to control and to modify
  • Flexibility: SCons can be extended easily thoughout
    modules. Besides, helpers can be written easily to
    help programmers (take all source files in a
    directory, or check automatically the consistency of
    the builds for example). SCons also has excellent
    support for win32 and OSX platforms, it handles many
    languages like Java or Fortran out of the box.
  • Evolution: the API and the code written for
    compiling the targets are re-usable so projects using
    it will not be locked.

How do they compare to unsermake?

Unsermake is a replacement for make and automake only.
Bksys replaces the whole autotool chain.

Can you summarise the build system BoF at aKademy?

Several candidates were evaluated. QMake is way
too limited and cannot be used at all. Unsermake is
not to be kept, as it is only a partial solution and
emulating the behaviour of automake and make consumes
a lot of resources. Two solutions remained in
competition for the future KDE build system: CMake and
SCons.

CMake is a Makefile generator like QMake; it uses a
macro language which is known to present several risks
as non-standard targets arise frequently in KDE. On
the other hand SCons is much more flexible as it is
based on Python, and was proven to work on fairly
complex applications already (KDE Games, Rosegarden,
..). SCons was then chosen and the conversion of
kdelibs 4 has started.

So what work needs to be done now?

So far, the main goal of bksys has been to replace
autotools for usual KDE projects. Now for kdelibs the
situation is a bit different as the tool will be the
base of all KDE applications, so a new framework has
to be designed, in particular:

  • We have to agree on a new set of API (stardard
    targets, naming, features...)
  • New bksys modules have to be written for detecting
    the system configuration (full autoconf replacement)
  • Some work has to be done for writing config.h files
  • More helpers will have to be written to make
    programmers lifes easier (scan for sources
    automatically..)

At the very moment, the bksys scripts have been added
to KDE SVN, and 3 main directories in kdelibs 4 are
already compiling (dcop, libldlt, kdefx). There is
some documentation in trunk/KDE/kdelibs/bksys/design.
The main aspects of the future framework are
presented, and some questions are left opened to
discuss.

In the past, the kdegames module was converted in
about 2 days, but for kdelibs it will take a lot more
time as even after the framework is complete some
reverse-engineering will have to be done on the
Makefile.am to achieve the conversion. I am striving
at the moment to get a first version of the framework
ready (most configuration modules complete and a few
folders compiling) to show how the new system will
look like and the main advantages of using it.

Who is going to do the work?

.. or who is working on it already? Three people have
committed code on bksys in svn so far, Simon
Haussmann, Stephan Kulow and I. Interesting ideas have
been raised on the IRC, and I am awaiting
more comments, when the first version of the framework
is ready for testing (a request for comments will be
posted).

Are non-KDE projects taking up SCons/bksys?

Most projects using SCons do it directly and use their
own framework, like Blender or Rekall.
The projects using the bksys layer i know of are
almost all KDE-oriented: Rosegarden, kio-locate,
skim...

Comments

I don't have any beef with SCons, but I'm not sure why you included perl in the autotools size measurement? What does perl have to do with anything?


By kundor at Wed, 2005/09/14 - 5:00am

Because the autotools require Perl to run, and Perl is bigger than Python.

Try running 'rpm -q --requires autoconf' or 'aptitude show -v autoconf' (depending on which distribution you're on).


By Chris Lee at Wed, 2005/09/14 - 5:00am

Using bksys instead of auto* on my projects has prevented me from having to commit suicide. Which at the very least is an advantage for me, and I'd like to think some other people too.

Developers have to learn a new build system, true. But after 3 hours of using it they will very quickly understand that their lives will be infinitely better forever more.

The users will benefit from quicker configure checks, smaller downloads and not having to deal with the horrendous autotools if the build breaks. I don't think a build-dependency on python/SCONS is a problem for 95% of users or developers.


By Max Howell at Tue, 2005/09/13 - 5:00am

> Using bksys instead of auto* on my projects has prevented me from
> having to commit suicide.

If you'd read the auto* documents then you wouldn't need to commit suicide.


By ac at Wed, 2005/09/14 - 5:00am

> If you'd read the auto* documents then you wouldn't need to commit suicide.

I wouldn't say that reading the auto* documents is an experience that's going to cheer you up, however.


By The Badger at Wed, 2005/09/14 - 5:00am

* KDE 4 already required python (Unsermake) and for months
* The poster "ac" is not a KDE developer (it even looks like he works on Gnome)

Do not feed the troll, Thanks.


By anonymous_curious at Thu, 2005/09/15 - 5:00am

a) Unsermake was also used by KDE 3 for people who liked using it, but there was no nesessarity for it too since people were doing fine using autotools for getting KDE compiled. So basicly no python requirements.

b) Could you please refer me to the site on *.kde.org where it's written that you must be a developer to participate to KDE ? You are just attacking all the people who keep contributing to KDE (including me). We are not developing KDE by providing code (nontheless I am coder and developers but work on other stuff) but the KDE participants are still able to contribute with translations, webpresentations, documentations, audiovisual stuff and other things such as bugreporting and excessive testing. Still though I am skilled anough to talk my mind out of things that bothers me, after all the sentence 'free software as in free speach and free beer is still valid'.

For the part of not feeding the Troll. Please thanks for the advise, I am not going to feed you any further but I think it would have been fair to give you at least a normal answer.

Till now I thought that KDE people were much more open minded for feedback given by others. It would be a pity to pop this bubble.

I recall good enough that some weeks ago people kept praying the pester about Scons on IRC, at least I had that feeling (might be wrong but that time I surely had) I keep wondering myself what kept them changing so fast. Or is it just because you can flame ahead someone ?


By ac at Thu, 2005/09/15 - 5:00am

Do you by a chance mix up this board with a mailing list where all this discussion already took place? Either way you are late with your concerns and not involved enough to make those who are believe in your case.


By ac at Thu, 2005/09/15 - 5:00am

Oh and by the way I never mentioned any KDE 4 since KDE 4 doesn't exist as of now. People started porting things from KDE 3 to QT 4 starting maybe 1 month ago or so and this will be KDE 4 one day. So why bother for things that don't exist yet.

And for being acused to mortify scons and python loving people, please take my apologizes but reading about autotools being shit, perl sucks, etc. of course is not offending those who write it or contribute to it. My bad..


By ac at Thu, 2005/09/15 - 5:00am

Fact: people have started working on kde4 at least 4 months ago http://www.kdedevelopers.org/node/1094


By Anonymous at Thu, 2005/09/15 - 5:00am

I must say this change is worrying me a bit.
Does the new build system provide cross compiling capabilities?


By Luciano at Tue, 2005/09/13 - 5:00am

That would not be a regression, as currently KDE cannot cross-compile.

Have a nice day!


By Nicolas Goutte at Tue, 2005/09/13 - 5:00am

Maybe, but the build system is shared by the kdenox project, which needs to be built with a cross compiler.
I suppose I will have to stick with the autostuff, then.


By Luciano at Tue, 2005/09/13 - 5:00am

There is much more control over the compilation variables with this new system, so it will not be a problem.


By Anonymous at Tue, 2005/09/13 - 5:00am

Autoconf/Auromake was already pretty bad, but at least (sometimes) only bash was required, now I will need to install another one of those scripting languages I really don't need nor want to have installed. Ok ok, so bash is probably not the most elegant way to program something, but at least bash is compact in the sense that it doesn't spill a shitload of files all over the filesystem (often in the wrong places). I was pretty pissed when I found out one had to have perl installed to get Gtk and GIMP to build. Has Jam been considered as an alternative?


By Me at Tue, 2005/09/13 - 5:00am

Yes, pretty much all alternatives have been considered, and the only serious contenders where scons/bksys and cmake.

Second, have you ever tried to maintain an auto*-based build ? I have, and like anyone who has I can tell you how incredibly brain-damaged this system is. And it does "spill a shitload of files all over the filesystem", much more than python.

As for not wanting to install python, seriously... what's the big deal. Apparently you're resisting it for purely "religious" reasons, well, too bad.


By Guillaume Laurent at Tue, 2005/09/13 - 5:00am

No, I don't maintain any auto-* based build. This would be a nightmare, I DO agree with you on that.

As for not wanting python installed, that's not religious at all, it's just that I think that integrated solutions, neat and clean is a better way to go than "just install everything anyone could every want by default". I don't mind python or ruby or TCL or ECMAScript or whatever. People need to settle on just ONE DEFAULT ONE, integrate it the best possible in the system. But I think the situation where just everyone decide they'll use their favorite scripting language is more than suboptimal. It is OK however when the scripting language is used through a plug-in (that is choice) and you don't force those who don't need/want to use ten thousand scripting languages to install them. On the other hand I would choose TCL or Ruby over Perl and Python because these two last ones take up 50 megs of disk space.


By Me at Tue, 2005/09/13 - 5:00am

Are you serious?

First, you're upset over 50 megs of disk space? Hard drives cost 50 cents per _gigabyte_ these days. Python costs you two and a half pennies to store. Give me your address and I'll send you a quarter, so you can store 10 copies of Python, and stop worrying. :)

Second, it's no more feasible to pick one scripting language and make it fit everyone's preferences than it is to pick either Gnome or KDE and make everyone happy. You wouldn't like it if someone decided that Gnome was The Linux Desktop and forced (somehow) everyone to stop working/using KDE, would you?

Third, this is only a build-time dependency. If you're so concerned about not having Python installed, then get a pre-built version of KDE.

Fourth, is there actually a mainstream Linux distribution that doesn't include Python? I thought it was almost as standard as Perl and a C compiler.


By Dolio at Tue, 2005/09/13 - 5:00am

>First, you're upset over 50 megs of disk space? Hard drives cost 50 cents per >_gigabyte_ these days. Python costs you two and a half pennies to store. Give >me your address and I'll send you a quarter, so you can store 10 copies of >Python, and stop worrying. :)

The main point was the first part of the reply but you chose to ignore it for some reason.

>Second, it's no more feasible to pick one scripting language and make it fit >everyone's preferences

No realistically, it's just about not wanting to take a decision not to hurt the feeeeeeelings of 2 or 3 people.

> than it is to pick either Gnome or KDE and make >everyone happy. You
> wouldn't like it if someone decided that Gnome was The >Linux Desktop and
> forced (somehow) everyone to stop working/using KDE, would >you?

KDE, GNOME, Linux, *BSD, [add your favorite open source project here] will never succeed if there's no standards, integration, consistency. If it's going to be eternally just bits from here and there slapped together with no coherence whatsoever, that is if everyone has their way.


By Me at Tue, 2005/09/13 - 5:00am

> No realistically, it's just about not wanting to take a decision not to hurt the feeeeeeelings of 2 or 3 people.

No. People invent various languages because they're unsatisfied with the existing ones, and think that theirs is better. Programmers gravitate towards languages that they prefer. Until you turn everyone into mindless automatons, there will never be a single language that everyone prefers over every other, and therefore, there will always be more than one language.

It's not a decision that anyone can make or enforce. If you think otherwise, you must be living in some alternate reality that I'm not aware of.

> KDE, GNOME, Linux, *BSD, [add your favorite open source project here]
> will never succeed if there's no standards, integration, consistency.
> If it's going to be eternally just bits from here and there slapped
> together with no coherence whatsoever, that is if everyone has their way.

First of all, that's bull. Windows has just as much inconsistency, lack of integration, and lack of standards compliance as Linux, if not more. That has not prevented it from capturing and maintaining a hold on around 90% of desktop PCs.

Second, it's not as if Gnome, KDE and whoever don't work together on standards and integration when it makes sense. However, there are some things that are impossible to integrate. Qt is not GTK, and it probably never will be. As long as there are people who prefer each, they will continue to exist, and no amount of whining or philosophizing is going to change that. It's also not going to prevent Linux from making inroads on the desktop, because it doesn't really matter to most people.

People coming from Windows are used to inconsistency. They're used to using programs with 7 different widget sets, and a myriad of different user interface decisions; they can live with two on Linux if they absolutely must. Suggesting that this is a significant stumbling point of Linux in the view of the average person is laughable. Linux desktops are already much more consistent than Windows desktops, yet that fact isn't harming Windows in any significant way.

Similarly, the average person doesn't care whether their programs are written in Python, Perl, Ruby, C++ or Brainfuck (let alone what languages are required for the build system of their programs). All they care is that they work. And at 2.5 cents or less per language, they can afford to have all of those installed, so that programmers can work in whatever they find suits the task best. Suggesting we should get rid of all but one because you find it to be somehow philosophically cleaner is nonsense.


By Dolio at Wed, 2005/09/14 - 5:00am

Yeah, and people need to settle on one single desktop, and stop making bugs in their code, etc... ain't gonna happen, in the meantime let's be a tad bit pragmatic. Python is quite well enough established for this to be a reasonable choice.

(and choosing Tcl over Python because it takes less disk space is simply asinine).


By Guillaume Laurent at Tue, 2005/09/13 - 5:00am

> Yeah, and people need to settle on one single desktop, and stop making bugs
> in their code, etc... ain't gonna happen, in the meantime let's be a tad bit
> pragmatic. Python is quite well enough established for this to be a
> reasonable choice.

Within one particular distro yes, it would make sense to pick one single desktop.

Besides I use Slackware, Python is a not a dependancy at all, it's there though, if you want it you can have it. But you're free not to install it if you don't need it.

> (and choosing Tcl over Python because it takes less disk space is simply
> asinine).

Religious? Anyway we'll talk about that again in a few months when another scripting language will be all the rage...


By Me at Tue, 2005/09/13 - 5:00am

> Yeah, and people need to settle on one single desktop, and stop making bugs
> in their code, etc... ain't gonna happen, in the meantime let's be a tad bit
> pragmatic. Python is quite well enough established for this to be a
> reasonable choice.

Within one particular distro yes, it would make sense to pick one single desktop.

Besides I use Slackware, Python is a not a dependancy at all, it's there though, if you want it you can have it. But you're free not to install it if you don't need it.

> (and choosing Tcl over Python because it takes less disk space is simply
> asinine).

Religious? Anyway we'll talk about that again in a few months when (yet) another scripting language will be all the rage...


By Me at Tue, 2005/09/13 - 5:00am

> Besides I use Slackware, Python is a not a dependancy at all, it's there though,
> if you want it you can have it. But you're free not to install it if you don't
> need it.

I fully agree with you.


By ac at Tue, 2005/09/13 - 5:00am

> Religious?

No, I've used both, Tcl doesn't scale (in terms of lines of code) as well as Python.

> Anyway we'll talk about that again in a few months when (yet) another scripting language will be all the rage...

Oh please. There are only 3 "viable" scripting languages : Perl, Python and Ruby. Perl is thankfully fading away, Python has its quirks but is the most mature, and Ruby is gaining ground but still needs "packaging" (its libs need to be documented for instance). Even though Ruby has been around for a long time it has only received wide exposure for about three years now. When was the last time a new scripting language was actually "all the rage" ? Python is as standard as a distrib component can get, get over it.


By Guillaume Laurent at Wed, 2005/09/14 - 5:00am

> Perl is thankfully fading away

Any values to back this up ? Perl is stronger than ever these days.


By ac at Wed, 2005/09/14 - 5:00am

None whatsoever, I gladly admit this is based both on wishful thinking and random observation (I see more headlines about Python and Ruby than about Perl, and Larry Wall's "Apocalypses" about the upcoming Perl 6 are, hmm, well...).


By Guillaume Laurent at Wed, 2005/09/14 - 5:00am

> None whatsoever, I gladly admit this is based both on wishful thinking and
> random observation (I see more headlines about Python and Ruby than about Perl.

And I keep hearing more about Ruby than Python these days so what.


By ac at Wed, 2005/09/14 - 5:00am

True, even though Python is more used, Ruby is "in" those day.
Still Ruby will need more than a web framework to sustain the hype.

Anyway I fully support Guillaume in hoping that Perl will die, though it will take time it is probably more used than Python+Ruby together.
But there are troubles in the Perl world: they don't manage to make Perl6, which (IMHO!) sucks also anyway..


By renox at Sun, 2005/09/25 - 5:00am

Perl was all the rage in the late 90s with the web boom and the widespread emergence of web apps, for which Perl was usually the language of choice. By the end of the 90s Java was eroding the high end of that market and PHP the low end. As a general purpose scripting language Python and Ruby weren't as established.

I also think that some of the shift has been due to paradigm shifts in people learning to program. Despite it being possible to do OOP with Perl, it's ugly; Perl really is just better suited for iterative programming. In the mid-90s new programmers were starting off with iterative languages, so Perl was a more natural transition.

Today, on the other hand most new programmers start with object oriented languages and learn to think in object oriented terms. As such it's natural that those people would gravitate towards object oriented scripting languages such as Ruby and Python.

(And I should note at this point that I know Perl very well and don't know Ruby or Python -- so this isn't a Python-love-fest, just something I've observed.)


By Scott Wheeler at Thu, 2005/09/15 - 5:00am

> get over it.

Why should I?


By Me at Thu, 2005/09/15 - 5:00am

Because you won't be the one to fix it anyway?


By ac at Thu, 2005/09/15 - 5:00am

> the only serious contenders where scons/bksys and cmake.

Besides people from KDE no one else ever heard of scons or bksys.

> Second, have you ever tried to maintain an auto*-based build ?

I did and I am still doing.

> I have, and like anyone who has I can tell you how incredibly brain-damaged this system is.

There are good documents people should consider reading, if you are not willing to spent time reading the auto* documents then you are not suited to make good files that work perfectly with your software. The problem with KDE rejecting auto* is that everone I've met so far complained about auto* but in reality it's because no one really spent time reading the manuals, they all want some quick hacks so it works and they start to copy stuff from other modules and wonder why it doesn't really work. How comes millions of open source and free software except the stuff inside KDE works perfectly fine with auto* ?

> As for not wanting to install python, seriously... what's the big deal.
> Apparently you're resisting it for purely "religious" reasons, well, too bad.

That's not the point. I want to have and keep my choice of what I want to have installed on my system and not you decide what I should install. Scons will be only used in KDE and no one else in the world will probably give a damn about it except KDE. This is another step of alienation again from the KDE team of what is standard in open source. Standard still is the auto* stuff. If Scons would have been that great then by now the majority of project have switched to it and this even outside of KDE... And the reality of course looks differently.. Personally spoken for myself I do feel upset for this Scons stuff it has only been decided to be good because the author and some other two followers kept praising the hymn out of it.


By ac at Tue, 2005/09/13 - 5:00am

> That's not the point. I want to have and keep my choice of what I want to have
> installed on my system and not you decide what I should install. Scons will be only
> used in KDE and no one else in the world will probably give a damn about it
> except KDE. This is another step of alienation again from the KDE team of what is
> standard in open source. Standard still is the auto* stuff. If Scons would have
> been that great then by now the majority of project have switched to it and this
> even outside of KDE... And the reality of course looks differently..

that's a bad argument. every new system, good as it is, needs one of two things to become "famous" (or standard if you like):
1) a LONG LONG time of proving itself, slowly.
2) a big "showcase project"

when (2) happens, and big project starts using the new system, two things happen:
1) system gets better, since a lot of testing is put into it
2) system can get a chance to prove itself much much faster

so, in this case, the project for bksys can be kde, and it might just prove itself better then autohell.

honestly, i don't have a clue about bksys but i know auto* is pain.


By Shie Erlich at Tue, 2005/09/13 - 5:00am

Why are you ignoring and skipping comments and arguments and then come up with own replies that are far OT to what's actually written ?

Tell me the benefits of Scons over Auto* and I return here in a couple of hours crushing all your arguments to dust.

;--- from another comment.
d) If you read the autotool documents correctly then you would figure that autotools has to serve on a lot of POSIX compliant systems with dozens if not hundrets of different toolchains, compilers, headers and whatever. It's not limited to linux, it's not limited on windows, it's not even limited on operating systems such as MorphOS, AmigaOS (which are totally NON POSIX). That are the big advantages of autotools. You enter configure and then make afterwards. But you of course know perfectly the difference between scons and auto* since you never ever read a damn doc about it.


By ac at Wed, 2005/09/14 - 5:00am

If something works as well without reading docs like you get the other to work only after spending much time reading docs the former is surely the more comfortable solution, no?


By ac at Thu, 2005/09/15 - 5:00am

Can you tell us which autotool-based project you are maintaining ? Thanks.


By anonymous_curious at Tue, 2005/09/13 - 5:00am

> Besides people from KDE no one else ever heard of scons or bksys.

I don't think you quite know what you're talking about here. bksys is an scons extension specific for KDE. And if you haven't heard of scons, well, I hope the rock you're living under is comfy.

> There are good documents people should consider reading [...].

First of all, I think the people who wrote all the autotools-based build framework for KDE (S. Kulow being one of the first) are knowledgeable enough about it. Second, if a build system requires you to do more than quickly browsing through a few example files to be understood and used on your code, then it's just plain too complicated. A build system should never get in the way of development, and should not require extensive reading. That's plain broken.

> How comes millions of open source and free software except the stuff inside KDE works perfectly fine with auto* ?

You don't know anything about KDE's build system, do you ?

> I want to have and keep my choice of what I want to have installed on my system and not you decide what I should install.

Yes. Whatever. If you don't mind, we'd like to move on, but feel free to remain on your high horse.


By Guillaume Laurent at Wed, 2005/09/14 - 5:00am

> And if you haven't heard of scons, well, I hope the rock you're
> living under is comfy.

There is no need to get sarcastic here. I initially heard about Scons through the KDE team and 2 people spending all their time on IRC convincing everyone how great it is while 3/4 of the developers kept crushing all their arguments to dust on IRC. If Scons would have been that great then by far 1/5 of the open source or free software world would have switched to it.

> First of all, I think the people who wrote all the autotools-based build
> framework for KDE (S. Kulow being one of the first) are knowledgeable enough
> about it.

See, you simply answered one of my arguments above. From this sentence I understood it so. S. Kulow is the only skilled person around the KDE team who understands the autotools stuff. Which clearly says that the others who work on KDE (please pardon I don't want to sound offensive now) never bothered to read the documents of said toolchain. This most likely includes you too otherwise you would have realized that:

;--- from another comment.
d) If you read the autotool documents correctly then you would figure that autotools has to serve on a lot of POSIX compliant systems with dozens if not hundrets of different toolchains, compilers, headers and whatever. It's not limited to linux, it's not limited on windows, it's not even limited on operating systems such as MorphOS, AmigaOS (which are totally NON POSIX). That are the big advantages of autotools. You enter configure and then make afterwards. But you of course know perfectly the difference between scons and auto* since you never ever read a damn doc about it.

> Second, if a build system requires you to do more than quickly browsing
> through a few example files to be understood and used on your code, then it's
> just plain too complicated.

There are dozens (if not millions) of example files for autotools floating around all over the net. Dozens of manuals, howto, tutorials. But then there is that 'reading' and 'understanding' problem again. What good does all tutorials, docs and howtos give if people won't spent time reading them ?

> A build system should never get in the way of development, and should not
> require extensive reading. That's plain broken.

It doesn't since it's part of the development process.

> You don't know anything about KDE's build system, do you ?

I do and I consider it a hack with all the *.in.in files that get concatenated to an *.in file which then get passed through the autotools to generate working configure and makefiles. Of course the best example in howto NOT use the autotools. A clear sign that the whole concept of the toolchain has not been understand correctly.

> Yes. Whatever. If you don't mind, we'd like to move on, but feel
> free to remain on your high horse.

Please pardon but that's the worst bit of ignorance that I've ever read from your side. You keep ignoring comments done by me and some other guy here, you only reply on the things you see fits best in your picture and pull other text under the carpet and you make Scons sound so ultimatively great without giving one example in what areas it improves.

I would like to call you up for a challange to write a nice Scons tutorial and have a real life comparison between where you think Scons is better than autotools instead braging out how great it is. Until now I haven't seen anything about it except 2 people hyping hymns about it. The autotools* toolchain work on all sorts of systems, taking acount of all types of different hardware, toolchains, compilers, it even works on non POSIX systems such as Windows, AmigaOS, MorphOS and whatever other OS' and the best, it doesn't require Python.

Please also stop talking about HARDDISKSPACE is so cheap. This clearly shows that you live in a wasteful world and never really used a small system which is fast, maintainable and functional. The point is to have small systems and not monster systems. My entire system here is just 1.5 gb of space this includes X11R6, teTeX, Linux (+ Kernels), GNOME, Evolution, Abiword, Firefox, my home dir with all my files. Tar'ed up the entire System would be smaller than 590 mb of size and you can be sure I am not missing anything. I listen to music, watch movies, look at pictures, do documents, print, even play games. The point for using KDE is not just because of its superior architecture because I could save again 300 mb by simply switching from GNOME to KDE and getting rid of Python and the docbook crap. If you live ok with having a 240 gb harddisk and live perfectly by installing everything you find then it's your choice. But my choice (talking about choices here) is to keep and prefer small systems - a point you seem to not understand.

KDE 3.5.x perfectly works without Python, I am not missing anything, everything works and I would have enjoyed to see the same for KDE 4.

Sometimes I have the feeling (and please apologize here) that KDE is walking alongside of the entire open source and free software movement (I said it's just a feeling). It starts with the way how KDE installs the files (pulling all includes in the top include dir, installing all libs without versioning) while the majority of open source and free software projects agreed to install their headers nicely sorted in subdirs so people know they are all keep together, give their libraries good versioning, make use of man files, info files. But under KDE all is stuffed in as it shows up. But ok chances are great that with KDE4 these issues are dealt with, there is a top voted bugreport for this on bko which is in the first 10 places of most voted bugs.

And now with Scons, the vast majority of project imigrated to autotools these days this even includes XOrg and a few other really large and major projects without any problems (XOrg and Mozilla are quite big ones and yet they seem to handle the oh so bad autotools perfectly). And again KDE need to walk alongside with a silly buildenvironment.


By ac at Wed, 2005/09/14 - 5:00am

> I initially heard about Scons through the KDE team

Whenever you look for an alternative to 'make', scons comes up pretty quickly. I heard about it a couple of years ago I think.

> If Scons would have been that great then by far 1/5 of the open source or free software world would have switched to it.

There's such a thing as inertia, and the Open Source community excels at it. Have you bothered to look at http://www.scons.org/refer.php ? Plus there's little doubt that KDE adopting it will give it a lot of spotlight.

> the others who work on KDE (please pardon I don't want to sound offensive now) never bothered to read the documents of said toolchain

Yes, because they shouldn't have to. Again, a development tool should not get in the way of writing code. You should not have to waste time understanding it or making it fit your needs, otherwise it's broken, period. Yes, it's a part of the development process, but a part that must be kept as small as possible.

> Dozens of manuals, howto, tutorials

I've googled my share of autotools docs in the 5 years I've maintained RG's buildsystem, and those never turned up. I must have been blind or not entered the right keywords.

> A clear sign that the whole concept of the toolchain has not been understand correctly.

Feel free to explain Coolo how he's an idiot. Please don't forget to include patches.

> Scons sound so ultimatively great without giving one example in what areas it improves.

I've switched RG's build system from autocrap to scons/bksys. I had very little problems doing so, Thomas was very responsive to all requests, and the end result is a build system that's faster, lighter, and both easier to use ('scons' compared to 'make -f Makefile.cvs && ./configure && make') and maintain. For instance after 5 years we still had recurring problems with the simple addition of a '-fexceptions' flag to g++. Or adding a new QObject would require restarting 'make -f Makefile.cvs' otherwise you'd get obscure link errors. So we're saving time both for us developers (less problems in building the whole thing, less time wasted in answering users who have trouble in building it) and users (easier for them to build it).

> This clearly shows that you live in a wasteful world and never really used a small system which is fast, maintainable and functional.

Dude, 1.5 Gb ?? Do you have *any* idea how wasteful that is ? Back in the 80's my //gs had a 40 Mb (read that : MEGA-bytes) HD, all with OS, GUI, and office stuff. And even back then that was already considered a waste of resource (after all the apple //e was just as useful with a couple of 128Kb floppies). 1.5Gb is a super-computer which should be serving dozens of users. And you have all that just for you ? What a waste.

Now seriously, if you can't spare the $50 a 80Gb costs these days, tough luck, but on my side the limited resource is *time*, not HD space, so I'm not going to waste it because you insist on using a 10 year old system. I don't care about your choices, I don't even care about leaving you choices when it diverts me from writing useful code for real non-geek users, and the same reasoning is applied by all sane open-source developers. It's pointless to hurt the majority of real users by wasting time just to satisfy the silly needs of a few uber-nerds.

As for the whining about KDE not following standards, yes, let's all stick to C (after all if you can't read on how to write OO code in C, you've no business programming, right ?) and Motif (which was standard 10 years ago).


By Guillaume Laurent at Wed, 2005/09/14 - 5:00am

> Dude, 1.5 Gb ?? Do you have *any* idea how wasteful that is ?

Please stop that sarcasm, we all know that we talk about 2005 and not what was 25 years ago with out 20 or 40mb harddrives. I come from these times and know pretty well. The demands and requirements nowadays are a hell lot differently and 1.5 gb for Linux measures are quite small but yet a full featured system.


By ac at Wed, 2005/09/14 - 5:00am

You're putting down the work of pretty much everybody here who's trying to provide useful software for people, pompously saying that "people should read the autotool docs", or that KDE's build system is an example of misunderstanding the tool chain, that standards should be respected and other stereotypical statements usually coming from people who have no experience in software development, all this because you cling to an obsolete system, and you expect me to remain polite or even take you seriously ?

Just what software are you working on, so we can all download it and bask in the light of your infinite wisdom ?


By Guillaume Laurent at Wed, 2005/09/14 - 5:00am

There is no need for you to get offensive.

a) I apologized many times that the stuff I write shouldn't be taken as offense. It just is my very own opinion about what I see and what I hear.

b) I very well respect the work KDE developers have done to provide a serious plattform for good software and integrated desktop experience.

c) I still believe that the problems around autotools is just a *placebo* thing that got repeated so often till everyone started to believe it while this argument is plain holdless. Personally I believe that the autotools system is quite mature and works perfectly. The problem is that people need to learn it, to read documents, read tutorials and of course that people do help each other to get around of children's disease. You say the programmer should concentrate on programming. I say that to learn programming you need to learn the language, the api, the system, the architecture and so on. That's quite a lot of what you need to learn to start getting something serious done. I talk from quality software here and not banal stuff as written in teaching books. Since I am a long years experienced programmer I do know what I am talking here. To learn autotools you need to be willing to spent time reading about it. The same would be a valid sentence for Scons and anything else that is new.

d) I never said that Coolo is a moron (as you described it). I only said that Coolo seem to be the only person who get used to the autotools (non)issue because he was willing to read about it and skilled enough to handle the issues for the entire KDE software that exists. Look one person alone to fix everything. Now imagine everyone would have at least half of the autotools skills of Coolo and by now no one would complain about it.

c) I probably asked you a couple of times now to name at least a few 'advantages' of Scons over autotools and yet you keep insulting me on a public forum like dot.kde.org for no reasons. And still I also said (I hope I did) that I have no issues with Scons per se. I have issues with the requirements of Python as buildtime. Not that it just is an requirement at buildtime, you also build other .py files and install them from other KDE components (that you usually won't get installed and compiled if no Python is found). So what was a buildtime requirement ends up in becoming a runtime requirement as well.

d) Please don't ask me on what software I was working on because it's by far offtopic to this conversation and would only end up in deflecting from the real problem. I by far have programmed, worked, maintained far more software than many other people around I believe, I must admit that personally I have only 3 or 4 programs that require the autotool chain, the rest usually were Makefile based, some Imake based stuff from former times and a shitload of ASM 68k software that from mechanism and build procedure works a bit differently (handled through a macro assembler, IDE, editor, debugger usually written with ASM-One on former Amiga times). Today I maintain some parts of the Linux Kernel, work on my own build system, some own software that I keep working at times and some other things.

But please would you start to be willing to name a few advantages of Scons over autotools ? Since you seem to be a clever guy that keeps defending Scons like it's written by you (maybe it is who knows) you might be willing to come up easily about a few advantages. It's your luck that python recently got ported to systems like AmigaOS4 or MorphOS otherwise we wouldn't be able to use the glory Scons system without Python but we would have been able to use autotools since 1996. I don't want to know how many non posix systems exists where you don't have python existing.

And please, I would respect you even more if you would reply in a mature way to me, even if I am an anonymous person I still try to be respectful to you and be as honest as even possible. Something you probably wont see every day from AC's.


By ac at Wed, 2005/09/14 - 5:00am

OK, you're right, I'll try to calm down.

> You say the programmer should concentrate on programming. I say that to learn programming you need to learn the language, the api, the system, the architecture and so on.

I agree with that, but the build system is not, or should not, be part of this. Would you use a compiler which required extensive reading just to understand how to compile a single C file ? Unless you had no choice, you'd probably prefer it to be as simple as "cc foo.c", right ? It's the same for the building system, or the text editor, the IDE, the version control system, the documentation generation system, etc... I make a clear distinction between development tools and development platform (for instance, C++/QT/KDE, or Java, or C#/.NET). Learning a platform is mandatory of course, even though it should also be as simple as possible. But your IDE, compiler, and build system should never ever get in the way, and should not require you to dive into tons of tutorials and stuff just to get simple things done. And in that regard, the autotools fail utterly, while scons is rather successful.

> Now imagine everyone would have at least half of the autotools skills of Coolo and by now no one would complain about it.

And everyone would spend time on autotools rather than on their code. Again, time is the critical issue.

> I probably asked you a couple of times now to name at least a few 'advantages' of Scons over autotools

I did in my last post. To sum up, it's just much simpler to maintain and use, simply not having this 2-level generation of files is a huge win. Also having all in one script (configuration and building) helps immensely. It also handles conditional compiling very easily. Just try it for yourself, you'll see.

Note that I'm not a huge fan of scons, I prefer Ruby over Python, but the only ruby-based build system (rake) is even more exotic than scons is, and doesn't handle configuration tests. But scons is still much better than autotools.


By Guillaume Laurent at Wed, 2005/09/14 - 5:00am

ac, you want a few advantages of scons over auto*? (Haven't others already given some, like that its simpler to customize, or the links to the experience with scons on other platform? You seem to constantly ignore them and demand other advantages. On the other hand you complain about misinterpreting if only a small part of you posts is not directly answered.)

Anyway, here are a few facts:

-) The ability to build one target with all its dependencies (and only the dependencies). E.g. in kdepim you can build only korganizer/korganizer and the required libs (libkcal/libkcal.so, libkdepim/, etc.) are also built, but the rest of kdepim won't be built.

-) Do parallel builds in a compile farm (like -jN in automax) without launching the linker N times locally, too (which basically locks up your machine).

-) Dependencies (on libs in other dirs, generated headers in other dirs, etc.) are really generated from the actual code, not implicit like in automake (where the order of the subdirs is relevant because they will always be compiled in the order given).

-) No PID overflow on systems like windows (see Jan's mail, link above, which you constantly ignore)

Of course, this list if not exhaustive.

One thing you claim is that no-one except KDE apps use scons: How about id Software (Doom) using scons, or blender using scons on all their platforms (Linux, Windows using both Microsoft Visual C and Cygwin, Solaris, Mac OS X, IRIX and OpenBSD)?

Also, by now we all seem to agree that coolo has a fairly good knowledge about the auto* systems etc. Why do you think was he the first to start a replacement for automake for the KDE build?
Yes, he's the guy who wrote unsermake (which is now recommended to build KDE; and btw it is a python script...) to cure the problems he couldn't overcome with the auto* toolchain.

Since you don't seem to use KDE anyway (let alone build it...), why is it so important to you whether the KDE build needs python (scons) or perl (autoconf)?


By Reinhold Kainhofer at Wed, 2005/09/14 - 5:00am

First of all, I would be thankful if you could stop saying that I am permanently ignoring things, which I clearly don't. I haven't ignored those mails if you refer to clee's posting which contains a bunch of links. To say the truth I kept reading them and even replied to clee but that mail didn't made it to dot.kde.org for some unknown reasons.

I also mentioned that scons is by far useless on systems that don't have python and there are a few of them (maybe even hobbiest systems) which are not unix like and not posix conform (which they mustn't). These systems however have perl and the autotool chain. So talking about cross plattform, it doesn't make sense to use something regardless how great it is if the host system don't support python. But this just as a sidenote and also not the worries I tried explaining. Again since you seem to have ignored more of my writings than I igored a few links posted by someone. My problems are not Scorns related, my problems are the dependency of python that I'd like to avoid. And no I also don't tell developers what to use as some others told me on IRC, I only asked whether the autotools stuff can still be supported for those who dislike having python around.

There are also people who keep telling me that perl is bigger than python and started doing some weak calculations that with scons I can get rid of perl, autoconf, automake, m4, libtool and only keep python and scons. This would be nice in a real life scenario but if you have ever dealt with building up Linux systems then you realize how many damn packages (there are tons of them) do require perl, m4, automake, autoconf, libtool and other things. So I am not able to get rid of them. The calculation 100kb scons vs. 600kb autotools is wrong too, saying that I can save 30mb perl + 6 mb autotools, libtools, autoconf is wrong as well. Correctly said it has to be (for the upcoming KDE and for the underlaying system to get build)

Perl 30mb
Libtool 3mb
Autoconf 2mb
Automake 1mb
M4 1mb

And now with the dependency of python you can add

Python 40mb
Scons 100kb

Ontop of that. Till now even by reading the Links above I don't see the problems with autotools on Cygwin, Solaris, Mac OS X, IRIX and OpenBSD and honestly I do believe that these are just excuses. I don't see any issues since there were no issues posted. Only the reminder that it would be better to use scons because they had issues with autotools, but which ones are not mentioned. But then I only bothered reading the Links posted there.

Instead of slamming the entire autoconf, automake, m4 and libtool people (since I get accused for slamming the scons people) why not helping them to improve autotools to become better, fix the bugs that exists ?

The thing with Id-Software and Blender should be seen differently:

a) I don't have to compile Id-Software, most of their stuff is propritary for years anyways till it gets released open source one day (if we refer to doom here)

b) Blender is something not really necessary for my system, it's more or less a small tool that can be converted to configure if someone likes doing this.

c) KDE is a different story I somehow expect it to seamlessly fit into the unix world, fit into the way and common arrangements of installing files (headers, libs, manfiles, infofiles) as most other (basicly everything these days) are doing. KDE still does not:

http://bugs.kde.org/show_bug.cgi?id=86986

And I would also have loved to see that KDE feels more unix'ish beneath it's desktop. Better autotools support, better dealing with libtools, better organization of the files. Manfiles, touching of files when they are installed (tons of files are still copied). These are the things I'd really expected. Right now where we finally get rid of all the different build mechanisms such as Imake (which isn't bad by the way) where we get rid of custom written Makefiles that are hard to deal with. Right now where major solutions such as XOrg started switching to autotools. Finally I thought that everything would pull on one rope and start moving to one direction.

I also prefer KDE for not depending on so many languages for getting its core build. There are bindings that people can use to code tools in the language they prefer but it was no dependency for building KDE from source which made it an interesting thing because you could delete bloat around it (no requirements for extra doocbook installation, no requirements for python, you also need curious mono dependency and other nasty thing, you simply ran a small kde build script which executes configure and you are done).

This little rant is also not about cheap diskspace or why I and others have to justify why we should or should not install python. It's simply because we still have the choice to shape our Linux systems the way we want it. We can easily add python, maybe ruby one day and then mono because someone had a brilliant idea, but we don't see the reasons for installing big bulky packages of nonsense just because some small script requires it otherwise there would be no progress further to get your desktop of choice compiled.

Of course Scons may be the better choice over autotools and that's still not what worries me, but I would really have prefered to stay without python. What's so hard to understand here ?


By ac at Thu, 2005/09/15 - 5:00am

> I only asked whether the autotools stuff can still be supported for those who dislike having python around.

As you have now volunteered to maintain the autotools based KDE build system and be available for any third party developer that might have questions regarding autotools, can I look forward to see your improvements in the 3.5 release or do you need more time, given that you have to maintain it also for the 4.0 branch

You are really taking on a huge job here, there should be more contributors like you


By Kevin Krammer at Thu, 2005/09/15 - 5:00am

I still wait for some of my autotools patches to get committed to SVN they are waiting on bko for quite some months now. Even asking back whether it's ok to committ them didn't return any answer so far.


By ac at Thu, 2005/09/15 - 5:00am

What bug number? (So that I can check what has happened?)

As for commiting yourself, as it is probably in kde-common/admin, you cannot do it.

Have a nice day!


By Nicolas Goutte at Thu, 2005/09/15 - 5:00am

Well, I tried to work with autotools and tried to read the documentation. I didn't have success. You need to learn:
-m4
-libtool
-autoconf
-automake

These are just too many things to keep them all in your head if you use them only from time to time.
While I myself prefer cmake over scons, scons will definitely an improvement over autotools.

Alex


By aleXXX at Wed, 2005/09/14 - 5:00am

Pages