aKademy Interview: Avi Alkalay of Linux Registry Project
Sunday, 29 August 2004 | Ateam
During the KDE Contributor and Developer Conference at aKademy one of the speakers Avi Alkalay talked about his project Linux Registry. We met him afterwards and asked him several questions about this project.
Please give us some background information on yourself.
<p>My name is Avi Alkalay, I worked with Linux for 10 years and the last 2 years
I'm doing Linux and consulting for IBM in Latin-America. Helping customers migrate
to Linux and Open Standards.</p>
<p><strong>What's the focus of this project and how did it start?</strong></p>
<p>The objective is to provide an infrastructure to make the Linux configuration
nightmare disappear. This can be achieved through a way to let programs
integrate automatically, without having to ask users to manually
edit their configuration files.</p>
<p><strong>How many people are working on this ?</strong></p>
<p>30+ members. Many contributions, language bindings and suggestions.</p>
<p><strong>Are you paid to work on this project.?</strong></p>
<p>No.</p>
<p><strong>What's wrong with the current way we are managing configuration files?
In your talk you question that we need to evolve the underlying OS structure.
Why is that?</strong></p>
<p>A lot happened in the desktop space: DCOP, XML-RPC, components, widgets, colorfull themes, fonts, etc etc. but
nothing evolved in the base OS organization. You can't have a wholly integrated desktop (as a computer for my mother)
without evolving the underlying OS. KDE or Gnome don't really deal with plugged webcams, new video cards, etc.
And the way you integrate this kind of things today in the system makes use of your human brains and eyes
to edit text files like modules.conf or XF86Config. A software can't really write this configurations
for you effectively, without a considerable amount of complexity or artificial intelligence :-)
It is programatically difficult to parse and change some configuration bit in a human readable configuration file.</p>
<p>Each software project (KDE, Gnome, Apache, Samba, /sbin/init, modprobe, etc.) has its own way to define the
format of their (text) configuration files, so this make them completely separate and unintegrable software.
Configuration is the soul of software.</p>
<p>Current UNIX systems organization was designed 30 years ago. It has many strong points, like the
filesystem hierarchy, etc, but nothing was defined regarding the format of the configuration files.
I'm talking about a common format.</p>
<p>Also, commodity hardware (video cards, webcams, mice, keyboards, USB, etc.) vendors use to not support
Linux (even if the kernel has drivers for them) because they don't want to deal with complex,
multi-distribution setups, like "in distro A, edit this and that file this way; in distro B version N,
run tool XYZ, then edit those and those files in this other way", etc. That's awful.</p>
Is there any interest from other developers or projects for this concept of using key trees for storing configuration data
A lot of interest is showed from many sides. We have people writing patches, GUI tools, language bindings, etc.
Check the project website to see some of them.
Waldo Bastian showed some interest to see how KConfigXT (KDE Config framework) will behave with a
Linux Registry backend.
<p><strong>How would the Registry integrate with existing frameworks like KConfigXT and Gconf?</strong></p>
<p>These frameworks are more high level, designed for more specific use: desktop software. Registry was designed to
be the most generic and low level configuration system you can get, so it can be used as a backend for these
frameworks. Then, you'll be able to access Gnome programs configurations with KDE tools, and vice-versa. And also,
access base system configuration with KDE or Gnome tools.
KDE and Gnome applications source code will still use KConfigXT and GConf API, but their infrastructure will be a registry.</p>
<p><strong>So what do you think are the advantages for Linux and especially for KDE? Is this something which is
of importance in Userland?</strong></p>
<p>Linux Registry was designed to be the configuration store of everything on the system, not only for the
desktop part. So if we get it adopted, we'll be able to see programs more easily integrating with other programs.
You don't need to burden users to edit config-files when software can do that on his own. This will make life
easier also for software and hardware vendors to deploy their products on Linux. So that way a new program is
easily pluggable into the system.</p>
<p><strong>The key names in your presentation at aKademy were similar to existing configuration file names, but still confusing to the
newcomer (e.g. user/env and system/init) Do you not think it would be a good opportunity to rename these keys
(e.g. from "init" to "startup")?</strong></p>
<p>Those were just examples .. I choose those names just to be easy.
But keep in mind that keys should not be accessed directly by users. Stable semantic-oriented GUI configuration tools will
appear asking things like "click here to install and configure your new webcam". Then, the real keys and key
structure that is happening bellow that is sysadmin stuff, and hopefully he'll have to deal with them directly
only in rare situations.</p>
<p>The layout of nowadays configuration files are defined at the packaging time. In the office of RedHat, SuSE, etc.
This leads one Linux to be slightly different to another Linux. But this small differences makes impossible
automatic integration and configuration.
Switching to a tree of keys and values concept, makes the application development team the owner of this layout
(which should be unique). So distro A, B and C, X.org, a video card vendor, a GUI configuration tool, etc,
will allways find the current video card driver in the key system/sw/XFree/Device/Videocard0/Driver, instead of
line 34 of file /etc/X11/XF86Config on distro A (if user does not edit it), and line 42 of file
/etc/X11/xorg.conf on distro B (if user didn't edit it).</p>
<p><strong>How about using XML .. isn't XML invented for this kind of stuff</strong></p>
<p>XML is a wonderfull standard to represent any information, make it portable, human and
machine readable. It adds a considerable amount of complexity too.
Registry was designed to be available anywhere, anytime on the system. For this, all required libraries should
be there anywhere, anytime. If Registry used XML for its storage, libXML would be required, and the XML library
may not be available for some early boot stage programs, like /sbin/init. Well, Registry keys can be used for
/sbin/init configurations, instead of the 30 years old /etc/inittab.</p>
<p>Anyway, Registry uses XML as its universal format to import and export keys into/from the key database.
This helps on transferring software configurations from one machine to another, backup, etc.</p>
<p><strong>Wouldn't using this concept of trees of keys be unreadable for the human eyes. Better said: can an
administrator still edit the files?</strong></p>
<p>Yes, he can. This was a long discussed subject, and one of the weaknesses of the Windows implementation.
Registry storage are plain text files, editable with vi or ed. But, thinking out of the box
(considering time), an administrator will not have to edit keys directly. A registry infrastructure makes semantic
configuration GUIs development be very easy to do. So with adoption and time, administrators will use more semantic
configuration tools, and they'll be able to edit keys directly too, if they need.</p>
<p><strong>Don't you think that this name, Linux Registry, can be problematic? Many system administrators had bad experiences
with the Windows Registry.</strong></p>
<p>In my opinion the Windows Registry is a good idea behind a bad implementation. It is one file to store it all.
If you loose this file, or it gets corrupted, you loose your system.</p>
<p>On the other side, the Linux Registry is not a daemon (so no single point of failure), does not store
all configurations on a single file, was designed with security in mind, and stores user's configurations
in each user's $HOME, and system's on /etc.</p>
<p>Anyway, I receive many many e-mails from people liking the design but feeling the name as an obstacle to
adoption. This name was chosen to make an automatic awareness in people's mind. Sometimes it works for
the wrong side. So in a few days, the name of the project will change to something very cool. Stay tuned!</p>
<p><strong>Where do you see the future of Linux Registry going?</strong></p>
<p>Hopefully it will be accepted by major projects like KDE, Apache, Samba, etc, and distributions.
Also, more storage backends should appear, more language bindings, etc.
Linux/BSD/Unix today is a 1+1=2 system, and Linux Registry can help it become a 1*1=1.</p>
<p>The project needs more programs using it, more awareness, more adoption. The advantages of a wholly
integrated system will only be felt when many programs will be using it, so use it!</p>
<p>It is free, designed for Unix, by an old Unix user, to old Unix users.
So give it a try: <A href="http://registry.sourceforge.net">http://registry.sourceforge.net</A>!
Comments:
The windows registry - Pinaraf - 2004-08-29
I think a lot of people don't understand the aim of the windows registry. The windows registry is here to be big, hard (impossible ?) to understand... Why ? Because the registry is the best way to store datas after removing a software. With the registry, a shareware can write when it was installed, and without a manual remove of the key, we won't be able to re-install the shareware...
Re: The windows registry - Corbin - 2004-08-29
Luckily with the Linux Registry you will just be able to search the registry for any keys created on Date X, or search for keys that haven't been accessed in Y days, it will kind of improve that situation (it would allow you to use any normal search methods you use for normal files)
Re: The windows registry - Pinaraf - 2004-08-29
It will be a text file, so it will be easy to make a diff between each install... But I don't like the idea because if the file is broken by a software, what happends ?
Re: The windows registry - Avi Alkalay - 2004-08-29
Different from the Windows Registry, the Linux Registry does not have a single file for storage. Keys are apread through the filesystem.
Re: The windows registry - Johannes Wilm - 2004-08-29
As an example: johannes@brinsley2: ~$ rg set -c "My first key" user/example/key "Some nice value" johannes@brinsley2: ~$ cd .registry/ johannes@brinsley2: ~/.registry$ ls user johannes@brinsley2: ~/.registry$ cd user/ johannes@brinsley2: ~/.registry/user$ ls example johannes@brinsley2: ~/.registry/user$ cd example/ johannes@brinsley2: ~/.registry/user/example$ ls key johannes@brinsley2: ~/.registry/user/example$ cat key RG002 40 <DATA> Some nice value johannes@brinsley2: ~/.registry/user/example$
Re: The windows registry - Richard Moore - 2004-08-29
This could be very problematic on some filesystems, especially things like CDROM. I also cautious about the performance impact of this arrangement. If you have multilayered KDEDIRS (eg. because you are using kiosk to lock different aspects of the system down for different groups of users), then the number of files involved is immense.
Re: The windows registry - Johannes Wilm - 2004-08-29
I don't get it, what does it have to do with CDROMs? Is it a question of ~/.registry/ being writable to? Wouldn't that also apply to ~/.kde/share/config/ Also, I don't get the part about kiosk. Sure, I get that you apparently have groups of different levels of what can be changed by the user. But isn't that pretty much useless if it only applies to KDE apps anyways? Shouldn't that be something that you implement in some future version of the registry itself so it will work for ALL apps? I think this is really needed very urgently, as we should be able to simple link the entries for several entries in gconf and kconfig and with a bit of luck the entries are formated in the same way and we should be able to use the same folder for desktop and the same fonts, colors, etc. And it would actually be a clean way of doing it.
Re: The windows registry - Richard Moore - 2004-08-30
> I don't get it, what does it have to do with CDROMs? IIRC there is a pretty low file system limit on the number of files in the ISO9660 format. In a live CD running KDE, the large number of files this design requires could be a problem. > Also, I don't get the part about kiosk. I'm referring to large number of files that would need to be opened during application startup. > we should be able to simple link the entries for several entries in gconf and kconfig This is a very bad plan. It will quickly lead to corrupted configuration due to version skew. In any areas where there can be a mapping between the two desktops, it needs to be explicit rather than just common config entries.
Re: The windows registry - Johannes Wilm - 2004-08-30
ok, I get the CDROM-argument. Well there is always still KConfigINIBackEnd, and that won't go away. The whole thing is set up to allow for different backends as far as I read the sources, but as there is only one implementation currently, it is hardcoded as being the one used. Also, if you watch the video from akademy, the linux registry is set up to have different backends (so for example when running windows it uses the windows registry as it's backend), so it should be possible to have a XML backend as well. Or am I wrong? About the linking: the registry allows for symbolic links which make one and the same key available at two different locations. Without changing anything but the backends used in gconf and kconfig, some keys will probably be the same to start with (that is in format of the contents). A tiny little python or bash script should be able to link all these to oneanother just for now. (just for testing of course or until the next kde comes out... and that could be in 1.5 years) Then probably a freedesktop standard can be formulated over these and they can be put into /user/sw/freedesktop with other global kde stuff being in /user/sw/kde and gnome global stuff in /user/sw/gnome (all normal apps simply having their stuff in /user/sw/APPNAME, and if it is system wide and not just for one user it will be in /system/sw/... instead).
Re: The windows registry - Richard Moore - 2004-08-30
As you say, it sounds like it should possible to have multiple backends, though this would seem to remove one of its advantages. > the registry allows for symbolic links which make one and the same key available at two different locations The trouble is that there are often semantics associated with these values and it is easy to get into an undefined configuration set (ie. one that an application would never generate itself). If configuration is to be shared, then the contract between reading and writing applications must be explicit.
Re: The windows registry - Johannes Wilm - 2004-08-30
What advantage does it remove that the LR supports multiple different backends? At almost all times you would probably use the one-key-one-file solution, an alternative backend for an xml-file could be a compile time option that is used for the rare cases where it is useful. The reason why the standard interface doesn't work with with a XML-file is library dependencies. About the keys: /user/sw/kde/Paths/Desktop would for example have the value "$HOME/" and /user/sw/nautilus/preferences/desktop_is_home_dir would have the value "true" given my current configuration. Now obviously, although these two things say the exact same thing, they are incompatible and for that a freedesktop standard is needed, I agree with that. But you can see how this will be so much easier to try out, etc. right? And it would not be possible with the current gconf/kconfig solution.
Re: The windows registry - a.c. - 2004-08-30
>though this would seem to remove one of its advantages. Actually, I think that is one of the better things about this. It will provide a nice clean seperation of policy vs. implementation.
Re: The windows registry - koos - 2004-08-30
If large parts (that what isn't needed for boot) of the LR can be symlinked to either an untar'ed tmpfs directory or a loop cramfs, it would solve both the CDROM and KIOSK issues. Although each entry requires a system call, both methods make LR actually a shared memory implementation.
Another possibility - Jakub Stachowski - 2004-08-29
I like idea of consolidating configuration system but I don't think Linux Registry is best tool to do it. To be useful it has to be adopted by majority of applications and frameworks or it will be "yet another configuration system". And how big chance has system, that forces different API *and* configuration files structure? Small if you ask me. Another problem is that Linux Registry only deals with simplest case - configuration files stored locally on disk. No network-wide configuratio, no LDAP etc. Instead of that I suggest looking at UniConf (http://open.nit.ca/uniconf.pdf). It has concept of many backends that deal with actual configuration files so it doesn't restrict developers to one particular format (like LR) but instead gives them ability to use *any* format undestood by backends (you happen to like XML? or maybe INI-style? fine, so KDE will use it). These backends can not only access local files but also get information from LDAP server, SQL, whatever. They can also be "stacked" allowing something following scenario of searching for config key: "first check if key is in read-only /etc/overrides.xml, if not check in user-writable ~/program.ini if still not found get default value from LDAP server. Ah, and our LDAP server is damn slow so its responses should be cached". Eh, I imagine simple GUI tool for admins to specify how programs should access their config - simplicity and user-friendliness meets power and possibilities. Multiple backends also mean that UniConf is useful even if not adopted by everybody. And this is where LR could have its place - as UniConf backend.
Re: Another possibility - koos - 2004-08-29
> but I don't think Linux Registry is best tool to do it. To be useful it has > to be adopted by majority of applications and frameworks .. Why doesn't this apply to your UniConf too? > These backends can not only access local files .. NFS? > They can also be "stacked" .. KDE has that as long as I remember, ie. $KDEDIR/share/config gets overruled by .kde/share/config. And LR seems to have that too, a /etc/ global and $HOME/ userlevel override. You forgot to mention how UniConf solves the access part on each entry. And how does it perform with a middleware?
Re: Another possibility - Jakub Stachowski - 2004-08-29
>> but I don't think Linux Registry is best tool to do it. To be useful it has >> to be adopted by majority of applications and frameworks .. >Why doesn't this apply to your UniConf too? Just to clarify: I'm not involved with this project. I just found it some time ago and thought that it was interesting. >> These backends can not only access local files .. >NFS? LDAP? SQL? Caching? Persistent caching (for fault tolerance) ? >> They can also be "stacked" .. >KDE has that as long as I remember, ie. $KDEDIR/share/config gets overruled >by .kde/share/config. And LR seems to have that too, a /etc/ global and >$HOME/ userlevel override. But these are arbitrary defined, not admin defined. My example was only example and nothing more - admin can setup it anyway he wishes. >You forgot to mention how UniConf solves the access part on each entry. And >how does it perform with a middleware? Huh? What "middleware"?
Re: Another possibility - koos - 2004-08-29
> Huh? What "middleware"? LDAP? SQL? Caching? Persistent caching (for fault tolerance) :-) That what sits in the middle of the application and the final storage on disk. Which is IMHO a major slowdown on UniProcessor systems (as all look-ups require a server round trip)
Re: Another possibility - Eike Hein - 2004-08-29
> To be useful it has to be adopted by majority of applications and frameworks or it will be "yet another configuration system". I think the key to getting this off the ground is getting X.org to adopt it. If LR were to be installed with that key piece of software, other application developers (and X11 desktop environments) would soon start using it as well. And since X.org is just at the beginning of becoming the most popular X11 implementation (with distros switching to it left and right), the time and opportunity for that change to occur is now.
Re: Another possibility - Avi Alkalay - 2004-08-29
From my perspective, UniConf has several issues: - It is too complex, with all its plugin systems - It is a middleware, which again leads to complexity - It is a daemon, so it stands for a single point of failure - It does not provide a universal single namespace for key names. Each program define its own - It is C++, not suitable for base system programs like /sbin/init. And, from the author, the C binding are still not good. - It has too much dependencies like the wvstreams library, which is under /usr/lib, which can be unavailable to whatever runs before you get /usr mounted - I tryied hard to find source code and download it to have a look (so then I can provide you guys more info about it :-), but it looks to be unavailable - Weaker ecosystem then LR. Be carefull with multiple backends, because in the end they just add too much options, and we'll end up having distro A using backend X, distro B using backend Y, distro C developing their own backend. And we don't want it. Anyway, LR was designed to, after being compiled, to not have multiple backends, on purpose. What is simple is good. What is good is simple. If its not simple its not good. If its not good its not simple (some chinese proverb). One more point: the central points of LR is the key namespace and the API. So in the future, if somebody really need it, he can develop some fat plugin system. This is just operational details. LR does not want to compete with network systems like NIS and LDAP. Its focus is only with those local multiformat human readable text files (99.9% of configurations). Period. Besides, please don't think that migrating a local configuration to NIS ir LDAP is that easy. LDAP, as any other DB system, demand rules and an architecture.
Re: Another possibility - Jakub Stachowski - 2004-08-29
> - It is too complex, with all its plugin systems More features causes complexity. KDE is more complex than fvwm+xterm and I still like it. > - It is a daemon, so it stands for a single point of failure Not true. It can run without daemon just fine (as library). > - It does not provide a universal single namespace for key names. Each >program define its own I don't understand the difference - both UC and LR define tree of key-value pairs and every program should define 'root' in this tree. > - It is C++, not suitable for base system programs like /sbin/init. And, >from the author, the C binding are still not good. /sbin/init doesn't need features of UniConf so I agree that LR would be sufficient. And UC can coexist with LR just fine. > - It has too much dependencies like the wvstreams library, which is under /usr/lib, which can be unavailable to whatever runs before you get /usr mounted Agreed. I don't like wvstreams dependency either. > - I tryied hard to find source code and download it to have a look (so then >I can provide you guys more info about it :-), but it looks to be unavailable It is part of wvstreams source tree. cvs -d:pserver:anon@cvs.nitmtl.ca:/cvsroot co wvstreams >- Weaker ecosystem then LR. And what do you mean by that? > Be carefull with multiple backends, because in the end they just add too >much options, and we'll end up having distro A using backend X, distro B >using backend Y, distro C developing their own backend. And we don't want it. I really don't get that - it is like saying that we should have one distro, one window manager, etc. Why do you want to restrict admins always to use simplest case - bunch of files on local disk? Multiple backends are really no problem - it is transparent for applications and admin can set it as he like (I happen to like XML for example). >One more point: the central points of LR is the key namespace and the API. So >in the future, if somebody really need it, he can develop some fat plugin >system. This is just operational details. That seems good. >LR does not want to compete with network systems like NIS and LDAP. Its focus >is only with those local multiformat human readable text files (99.9% of >configurations). Period. And this is a problem. Suppose that you convinced Apache developers to switch to LR. It makes using central configuration via for example database impossible (because they would have to implement another API for that). With UC it is up to admin how and where he wants to store config data. As I understand, LR is created to solve one particular problem: multitude of config file formats by unifying them. UC solves that (another way - by making any format available to any application) AND gives additional benefits.
Re: Another possibility - Paul de Vrieze - 2004-08-30
I agree that current linux registry seems to be too limited. What I would like is to be able to say: "This part of the registry needs to be stored in ldap (accounts)", "This part should live in a file (/etc/inittab)", or "this part should be stored flat in an ini style file". In such a setup one would provide one or a few standard modules that live in /lib, but provide an ldap plugin in /usr/lib. It would even be possible to have that plugin do some caching with a daemon. What I further would really like is that applications would be forced to use a single key in an identified location. Making it very hard not to do so would probably also work. In any case a windows-registry like mess should be avoided. The namespace should not become too deep or cluttered
Re: Another possibility - Nurul - 2004-08-31
The registry should not be the palace applications look for their config files, but we do need some way other application can interrogate the configuration. My strong feeling is that there would be a hybrid system: 1. there is a /proc like file system for configuration. e.g. /configfs 2. Each config file has an associated script that will populate this /configfs - the config FS would look just like the registry 3. The subsystem associated with the config file should provide the script that populates the /configfs 4. For each config file there should also be a script that would perform a consistency check on the /configfs 6. Changes can be made to the /configfs dynamically but it will not have any direct effect on the underlying application or the underlying config file, bat a save operation would run the reverse script that would reverse the data back into the config file and a 'propagate' operation that would notify/restart the underlying application. Benefits - we get a simple way for other applications to check configuration without complex ad-hoc parsing, existing applications do not change (what good is a global registry if only a few applications use it). We are not totally dependent on the application maintainers to benefit from the /configfs - although it would be best if the application maintainers write the scripts that would populate the /configfs but in fact anyone could dot it.
No daemon? - Mike Hearn - 2004-08-29
Hmm, without a daemon how can applications be notified of updates to the configuration? This is absolutely central to gconf for instance, and nothing that wants to replace it can lack this feature
Re: No daemon? - jmfayard - 2004-08-29
Yep, Linux Registry has its drawbacks, some of them are described on freedesktop.org wiki : ===== http://freedesktop.org/Standards/config-spec registry.sourceforge.net This proposal stores every (key, value) pair in a separate file in the filesystem. On most filesystems, this is very inefficient, although systems such as reiserfs support it well. It also makes accessing the keys slow if many values need to be read (which is typically the case when an application starts, and especially at boot time when it causes many disc seeks). The registry does not support change notification, locking, networking or cascading, all of which are required for a desktop registry. ======
Re: No daemon? - Eike Hein - 2004-08-29
Beats the myriad of configuration files we use nowadays, though. Linux Registry is simple, straightforward. That's its biggest advantage over many other "the config system to end all config systems"-type projects, which may be superior in the feature department, but also have a much higher barrier of entry/adoption. I say let's go with something simple like LR - it can be improved upon and extended at a later date fairly easily. Also, keep in mind that LR doesn't aim to replace e.g. GConf - it can act as backend for GConf, though.
Re: No daemon? - Avi Alkalay - 2004-08-29
Exactly. For GConf or KConfigXT, LR is a perfect common backend. So you keep apps source code compatibility.
Re: No daemon? - Avi Alkalay - 2004-08-29
This is an old post. It currently supports change notifications etc. A key point some people (specially FD.o folks) don't see is that LR was designed for super generic and base system usage. Then it can be used from that low level to anything up. If all their requests are satisfied, LR will not be usefull for very base system programs like PAM, /sbin/init, etc, because it will have too much dependencies, and it will be too much optimized for desktop. If you optimize something for certain use, you'll deoptimize it for some other. So LR was designed to be the simplest possible, and additional features (and dependencies) can be added on top of it easilly.
Re: No daemon? - Some random KDE user - 2004-08-29
But one application could have a single key/value pair stored in the registry, with the value being the "real" configuration file of that application. This way, there would be at least a facility to conveniently and unambiguously locate a configuration file. Nowadays, most applications have a complex configuration file search mechanism, so that difficulties arise to determine the exact files which are chosen by the application.
Re: No daemon? - Avi Alkalay - 2004-08-29
LR provides key monitoring and change notification, across programs and across users. It uses a mechanism similar to SGI's libFAM. This is the API documentation you can use to get fine tunned change notifications of entire key sets or a single key: http://registry.sourceforge.net/api/html/group__registry.html#ga16 http://registry.sourceforge.net/api/html/group__registry.html#ga17 You can get notifications like: - When value changes - When only key access permissions changes - When key comment (description) changes - Etc - Any combination of all
Project status? - Simon Edwards - 2004-08-29
What is the status of this project? What is working right now and what still needs to be done? and most of all, what is the status of all the un-fun work such as integrating LR with existing programs and distributions? or are the LR people just doing the fun part of making a framework and hoping that other projects will jump onboard and do the thankless integration work? Just trying to be realistic here. -- Simon
Re: Project status? - Eike Hein - 2004-08-29
> are the LR people just doing the fun part of making a framework and hoping that other projects will jump onboard and do the thankless integration work? With Avi Alkalay attending aKademy and giving a talk about the project, they're obviously investing both time and money in getting developers interested. Which is a fine way to get the ball rolling; to spark discussion of the issue LR tries to address. Once the manpower is there, I'm sure the development of patches to existing software is going to be the next logical step. You have to admit: Convincing people "on the inside" is likekly to be more important than submitting patches as outsiders - we're talking about getting projects to replace their configuration backend with a new external dependency, after all. Evangelization is key.
Re: Project status? - Avi Alkalay - 2004-08-29
Simon, you touched the central point. Actually this project is about 0.2% of software and 99.8% of evangelization and patching. Actually, the software done, fully documented, etc. No we are making awareness.... Anyway, check http://registry.sourceforge.net/#ecosystem to see some things happening. GLibC is being patched too. And yes, you can also help make that list bigger :-)
Non-Linux systems - Pedro Fayolle - 2004-08-29
What about non-Linux systems? Most software running on Linux is able to run also on a plethora of other OSs. Wouldn't the implementation of this tie software to Linux? I know this could most likely be easily implemented anywhere, yet the current name of the project is "Linux Registry", and as far as I've seen it's also meant to be tightly bound to a Linux system.
Re: Non-Linux systems - Avi Alkalay - 2004-08-29
This took a considerable amount of attention. The idea is to let Apache use LR on Linux, BSD, Unix, Windows, etc, each port using a special platform backend, but calling the same API. Linux is in the name only for "marketing". It uses only POSIX calls, so it is portable to Windows, Mac, Unix, BSD, etc. So on Windows for instance you'll use the LR API and namespace, but keys/values will be actually stored in the Windows Registry I've successfully compiled it in BSD, already. See the presentation I delivered in the aKademy, about this subject. It is here: http://registry.sourceforge.net/registry.sxi
Re: Non-Linux systems - David - 2004-08-31
It seems to me that this "marketing" name is not aimed at the primary audience, the developers, but towards the secondary audience of suits and PHBs. "Registry" isn't going to appeal to the developer because of its connotations with Microsoft's centralized information model, and "Linux isn't going to appeal to those developers using a different operating system. I'm even imagining the reaction to my customers when I tell them they need a "Linux Registry" to run the native OSX application I made for them. This project really needs a new name. It's not a Linux clone of the Windows registry, yet that's what its name implies. Not only are you marketing this to the wrong people, it seems that you're anti-marketing it to the right people.
Re: Non-Linux systems - Eu - 2004-08-30
Look, at some point, this is bound to happen. It is cost prohibitive to develop hardware drivers and software for every possible unix-like system. Some people don't seem to realize, particularly older unix users, that changes such as these are necessary to bring Linux up to the level of usability of existing proprietary systems. What we need to do, the whole FLOSS community, and in here I include the BSDs and Linux folks, is set our egos aside and ask ourselves whether we really need 10 different kernel or rather a really good one. If we ever expect Free Software to succeed, we will need to sacrifice some of our own pet projects for the greater good, by which I mean acceptance of FLOSS, greater hardware compatibility/availability and more ISVs developing to our platform. Linux has become the defacto standard that many in the industry have been looking for in the past 20 years and I think that that is a damn good thing, even if it means that I'll have to switch my OpenBSD server to Mandrake, Red Hat or whatever.
Re: Non-Linux systems - Roey Katz - 2004-08-30
>Look, at some point, this is bound to happen. It is cost prohibitive to >develop hardware drivers and software for every possible unix-like system. So what happens when some new app comes out for Linux and there isn't yet a fixed standard for the apps's Registry keys? One distro will start the keys at one point in the tree and another distro will start the keys in a completely different place. Who is going to mandate standards for Registry's key heirarchy? Are the big commercial Linux vendors going to follow it? What's in place today to discourage divergent heirarchies among Linux vendors? - Roey remove the turtles from my name to respond.
it's prohibitive to develop windows drivers too .. - AntiGuru - 2004-08-30
win95/98/me/NT35/NT4/Win2K/WinXP ... etc. etc. as long as standardization on drivers for windows is driven and owned by hardware makers (and not MS doing another embrace and extend WinModem WinPrinter move) then it benefits Unix too. From ZD.net: FreeBSD 5.3 will ... introduce a software layer that lets Windows network drivers work with FreeBSD. This layer, dubbed Project Evil, means that wired and wireless network cards should be able to work with FreeBSD even if the manufacturers have not written any drivers for the operating system. "Bill Paul (a fellow core team member) has done the work there," said Long. "His method was to write a Windows emulation layer that provides the API (application programming interface) hooks that the drivers use. It seems to work for quite a number of drivers so far."
Re: it's prohibitive to develop windows drivers too .. - Roey Katz - 2004-08-30
still, hardware maker adoption of Registry without some standard to conform to almost guarantees that different vendors will come out with different key heirarchies. Besides, hardware vendors write their own drivers. They can stuff keys wherever they want in the Windows Registry, because it's only their own drivers that have to deal with it on that level. Yuck. As an administrator, I find it a pain to have even two ways (between Red Hat and Debian) to configure the same programs. If these FHS and LSB standards (1) provided usable standards and (2) were adopted by the big Linux vendors, my job would be easier. How about FHS/LSB adopting the Linux Registry? - Roey remove the turtles from my address to reply.
Re: Non-Linux systems - Avi Alkalay - 2004-08-31
The utopia is to let the developers define their keys structure. This should be done in the development time, and not in the packaging time (which leads to differences between distros). LR documentation provide about 4 bullets of good practices, and that should be enough, I think. They are here: http://registry.sourceforge.net/#bestpract
Re: Non-Linux systems - David - 2004-08-31
Funny, your same exact argument is often used to tell people to stop working on KDE and join the GNOME effort. The Free Software community has managed just fine without a centralized dictatorship, and I think we will continue to survive without it.
quick hack - Johannes Wilm - 2004-08-29
Hey, does anyone have some somewhat intimate knowledge of kdelibs/kdecore/kconfigbackend.cpp and kconfigbackend.h? KConfigINIBackEnd is hardcoded as being the one used so a quick hack to try out using linuxregistry as the backend would probably be very usefull. I have tried, and I might try again if noone else does, but my main problems are that I don't have an intimate knowledge of C++ and especially conversion from one thing to another (`KConfigBase*' to `KeySet*'). I would probably find out, but it would take a while. It is probably a real easy way to get a lot of credit within a few minutes.
Re: quick hack - jmfayard - 2004-08-29
See this mail from Waldo Bastian. http://freedesktop.org/pipermail/xdg/2004-April/003711.html Hope that helps.
Re: quick hack - koos - 2004-08-30
Recently I was looking at this (after reading a mail on kde-devel IIRC, about using threads for config files) but although the multi-backend and abstract KConfigBase, KConfig contains a 'KEntryMap aEntryMap;'. Which means that parallel reading from disc and querying entries (eg. block, join config thread, until entry is being read or all entries read) is simply not posible :-(
ESmith approach - AntiGuru - 2004-08-30
is more flexible ... www.esmith.org Linux Registry is scary ... at least GConf only targets user and desktop applications.
REGISTRY IS A BAD IDEA! - Douglas Phillipson - 2004-08-30
Haven't we learned our lesson from windows about a all encompassing registry? I like all my settings for all different programs in separate files thank you! I would NEVER buy into another registry no matter how many other problems it might solve! Doug P
Re: REGISTRY IS A BAD IDEA! - Nurul - 2004-08-30
I agree completely, but we do need a better solution for handling configuration meta-data. I have posted a suggestion that would keep the current configuration information intact but extract the data from all the configuration files to act as a meta configuration knowledge base. This would allow other applications to easily interrogate the configuration. Security is an important concern so this meta data would need rather fine grained security. That has to be clearly thought through.
Re: REGISTRY IS A BAD IDEA! - Avi Alkalay - 2004-08-31
The problem with the Windows registry is its implementation: - single point of failure - binary blackbox - proprietary The design of LR was made exactly to avoid these issues.
nooooooooooooo - Fabs - 2004-08-30
I agree configuration may be painful but a registry may give root privileges to an automated userland process and virii will be a possible nightmare in Linux, its like [W/L]indows giving everybody root (or administrator) access just to make gadget configuration (like web cams or mem stick readers) "easy". NO to Linux registry Well as the Open Source world is a free world, anybosy is free to implement a Linux Registry but I really hope nobody will use it.
Re: nooooooooooooo - Nurul - 2004-08-30
The registry is totally broken in windows, let us not make the same mistake.
Re: nooooooooooooo - johnflux - 2004-08-30
"I agree configuration may be painful but a registry may give root privileges to an automated userland process and virii will be a possible nightmare in Linux" Stop FUDing. How is it going to give root privilleges? There isn't even a process(daemon) that could possibly be SUID.
Re: nooooooooooooo - Avi Alkalay - 2004-08-31
If you could just READ some points of the documentation BEFORE writing some random crap you'll see that this is an issue LR does not have. http://registry.sourceforge.net/#securityexample BTW, LR spends no single line of code to handle security. This is good because this is all delegated to the underlying kernel.
Why re-invent a broken idea - Nurul - 2004-08-30
I completely agree that the Linux configuration information is a big problem because it is so ad-hoc. The real problem is that Unix/Linux does not have meta-data about the configuration files: 1. Which are the configuration files 2. What are the interdependencies in the various configuration files 3. Large number of formats. 3. There is no consistent method of notifying components that the configuration has changed 4. There is no high level API to interrogate the configuration The good things about the configuration information in Unix: 1. Human readable 2. Mostly fairly simple to understand (with a little bit of documentation) 3. Fairly use to use version management of config file using CVS - although not as common as it should be 5. Most config files can be easily read by programs The windows registry was a great idea (10 years ago) that turned out to be badly broken. Perhaps the single biggest problem in windows is the registry and in particular corrupted registries. The problem is that it is really easy to semantically corrupt the registry and very difficult to discover where the problem lies. Having no real semantic information it is up to the applications that use the registry to make sure that everything is consistent . Over time these applications change, different developers add yet more keys to the registry often is places that may have seen like a good idea at the time but nobody bothers to document it and bit by bit you end up with this tangled pile of s**t that nobody understands. So the standard recommendation for MS Windows is "re-install windows once a year to clean out all the registry c**p. There is an old saying - every problem in computing can be solved by adding one extra layer of indirection. Clearly, configuration files are themselves one level of indirection: do not hard code values in the application place them in a configuration file. So add another level of indirection, this data would act more like the index files iin a ddatabase, that is the information can be reconstructed form the original configuration files. This meta-data would not be a replacement for the underlying configuration file but it would be a more convenient 'view' of the configuration files. Think of this as a knowledge base in the AI sense of the term of the underlying configuration. So how do we solve the problem of all those different formats of the configuration file - we don't. I propose that just as the shell has the: #!/bin/sh for shell scripts #!/usr/bin/perl for perl scripts there would be a similar arrangement to each configuration file that would would update the central meta data when the file has been changed. This update utility would also extra information about how to check for consistancy, allow changes to the meta-data to update the configuration file In order to set up such a meta databse we will require some meta-config files: a. that lists the locations of all b. list the file containing the consistancy checker rules c. Meta information about version managment d. Whom to inform if there are configuration changes IMHO we have to apply the tools of knowledge management to handle system configuration, the suggestion that The most significant benefit of this approach is that no application has to be changed in order to use the system. Over time many applicatios would cease using the configuration file directly and use the meta-data but that is their choice. I have made may for the basis of such an approach. I hope this this will encourage others to consider this approach and build upon it. This is a very crude outline of such a system, the next stage would be to create a prototype.
Re: Why re-invent a broken idea - Avi Alkalay - 2004-08-31
My dear, IMHO you have the wrong vision about how things happen in customers, are missinformed about what LR can do for you, WRegistry issues it avoids, what is good and bad on Unix, etc etc etc. Windows registry is a good idea with a bad implementation. They didn't think about security, single point of failure, etc at that time. The most painfull one is DB corruption. LR avoids it all. Read the documentation. About that "tangled pile of s**t that nobody understands" that grows over time, I totally agree with you. But go take a look in /etc and in you $HOME dotfiles. You'll find the same ammount of s**t. Anyway, LR provides mechanisms for you to find keys that were not accessed for more than a period of time, that are candidates to be removed. Notification is not that important. It is usefull for some desktop apps. And you probably don't want Apache to automatically restart itself right after you change something. Anyway LR supports change notifications.... About the good things on Unix config style you said: - Unix geeks are not humans. Humans are people that don't even know how to use vi. I'm talking about my mother to use Linux. - Config files version management is a tale. Real world people (not geeks) don't even know what is version control. I travel the world visiting and evangelizing Linux on real businesses, and 150% of them don't use it. Anyway, you can do this with LR. - About 0% of nowadays config files can be read by programs. Go to NVidia website (a big supporter of their products on Linux/BSD). Their products READMES tell you to edit X config files by yourself. It is simply too much work to do: you'll have to write configuration files compilers :-) About this all meta-data things you said, thats too complex, pure rocket science. Please go to see how real businesses use computers. They just want to make their stuff work. If such meta-data thing existed it would not be used on real world, or it would become an industry of inconsistency. That recalls me Linuxconf, an abandoned software, because it was too complex to maintain such a rich set of configuration files generators, and creates inconsistencies. Knowledge management for configurations? Oh gosh.... rocket science again. Think simple, please.
Re: Why re-invent a broken idea - Andre Somers - 2004-08-31
> Anyway, LR provides mechanisms for you to find keys that were not accessed for more than a period of time, that are candidates to be removed. Maybe it would also be usefull to see by what application they were put there. That may be more informative than the question when they were last accessed. >About the good things on Unix config style you said: > - Unix geeks are not humans. Humans are people that don't even know how to use vi. I'm talking about my mother to use Linux. Your mother will not be accessing the LR directly either, nor would she ever try to modify the windows registry for that matter. Sysadmins *do* read and edit configuration files, and are glad they can. > - Config files version management is a tale. Real world people (not geeks) don't even know what is version control. I travel the world visiting and evangelizing Linux on real businesses, and 150% of them don't use it. Anyway, you can do this with LR. True. However, backing up configurations is not. I find it a treat that I can back up my entire configuration by just packaging my homedir and /etc. > - About 0% of nowadays config files can be read by programs. Go to NVidia website (a big supporter of their products on Linux/BSD). Their products READMES tell you to edit X config files by yourself. It is simply too much work to do: you'll have to write configuration files compilers :-) Yes, but OTOH: migrating everyting in one go is too much work too. In some cases, it might be very profitable to create a configuration file compiler. That may be hard for some cases, but I doubt it will be harder than converting all important systems in one go. >Think simple, please. Yes, but don't oversimplify.
Re: Why re-invent a broken idea - Nurul - 2004-08-31
I actually happen to have a fair amount of experience with the Windows registry and I hate it with a vengeance. For those of you who hate Microsoft please bear with me - I am not here to praise Microsoft; I would like to benefit for their experience with the registry and show you why it is one of the most hated and feared component in the system. Far from being a tool that an end-user would change MS actively discourages sys admins from editing it. So the notion that editing a registry is simpler that a config file is not borne out by experience. I remember when it first came out I was very excited about the prospect of getting rid of all those INI files that were all over the place and with no consistency about naming of placement. Windows registry would give a simple hierarchical storage that was searchable, allows atomic updates, a single location to keep all the data, and easy to organize cleanly. As they say be careful what you as for you may get it - and boy did we get it. Far from creating simplicity we ended up with an organizational nightmare. Applications would leave their poop all over the place, they would update parts of the registry that would make you wander what was the developer thinking. You would uninstall an application and the uninstall would remove 80% of the entries the application had created, the remainder would be lurking somewhere in the 100 of thousands of entries that is the typical registry. The registry tries to solve a legitimate problem - manging configuration files in a complex highly interdependent system; it has been a miserable failure for Microsoft despite the fact that they dis not do anything too stupid implementing it. I completely disagree about your comment that Microsoft's registry was a failure because they had a poor implementation. IMHO the failure arises because MS did not realize that manage configuration is more than a simple matter of having a central repository of data. My basic points are as follows: 1. It is too hard to get every application to change is configuration storage scheme to the registry and the registry is only useful if everybody uses it. 2. Even if you could persuade everybody to use the registry you do not significantly simplify theproblem of managing configuration information and you loose the entire tool chain of utilities that have been developed over the years to process text files. 3. No user directly manipulates the registry to manage configuration information in the Microsoft world. 4. There is a legitimate need for other application to interrogate the configuration information, but is the registry the best solution? 5. The key concept missing in the MS registry is meta-data that can vaildate the consistency of the data across configuration files 6. A method by which other applications such as installers, configuration GUI can manipulate the data in a way that is both safe, consistent, auditable (see who has done what) and a means of reverting to known good configurations. 7. There should be a dynamic file system (a kind of registry cache) like the /proc file system that each configuration files is parsed and the entries are set in this 'virtual' registry so that other applications can query the configuration data 8. A reverse process that can take this virtual registry and create a configuration file 9 Consistency checking scripts that would be run against this virtual registry to make sure everything is makes sense. 10 The virtual registry would be a file system - no new API just file and directory operations 11. The configuration file will be the inly information that the assocated application uses, the registry is simply a preprocessed cache that orher utilities can use to interrogate the data. IMHO Windows registry is a bad idea with a horrible implementation. It is S**T squared. It creates a new namespace that resembles a file system and text files/ binary files Registry key = directory value-fields = files except we have decades worth of tools to manipulated and process files and directories, we have nothing useful to manipulate Windows Registry. Even if you were to do a hell of a lot better in terms of support tools that MS has done for its registry, it would be largely wasted effort. Hans Reiser has been working on this very problem for years - how to create a file system that would efficiently support thousands of very small files - but I an getting off topic. The problem as I see it (perhaps a very limited view) is as follows: 1. There are too many formats for config files, that makes it unnecessarily difficult to learn configuring all the myriad of components in a Unix/Linux system. 2. There is small but significant inconsistencies between distributions on where the files are to be located and perhaps even named. 3. It is very difficult for other applications to read and utilize the information in the config files 4. It is difficult for end users to do anything with the config files. Let me start off with item 4 - no windows administrator would let an end user within one mile of the registry. Most administrators use regedit with trepidation and will only touch only very well known locations. Nobody other than a FOOL would attempt to do any tidy up job on the registry (EVER) - unless it is a precursor to doing a fresh install of Windows. Item 3 (It is very difficult for other applications to read and utilize the information in the config files) yes the registry in theory make is relatively easy to access data, but to actually put out useful information for the registry? Microsoft usually recommends against it - the organization of the registry can vary from release to release - there is always ANOTHER MS API to get some system critical information. Sarcasm aside - as applications evolve they seem to find ever more complex ways of representing information in the registry. So it really is not any easier (or not by much) to extract useful information from the registry than form a text configuration file. You think XF86 config is complex - look at windows some time it will make your head spin. Item 2 should be better with the registry, but again Microsoft shows us you can even screw this one up even when you own company designs all the applications. Windows 9x and WinNT/2000 have these subtle changes to the location of certain information that always makes Windows programming ever so interesting - just in case writing the application was too simple a challenge. Finally item 1, the registry should get rid of all those subtly different formats for the config files, but you guessed it, MS yet again show us that no matter how simple something may be in principle you can always make it more complex. Life would be not fun if everything was simple. >About 0% of nowadays config files can be read by programs. Go to NVidia website (a big >supporter of their products on Linux/BSD). Their products READMES tell you to edit X config >files by yourself. It is simply too much work to do: you'll have to write configuration files compilers :-) You have the problem completely backwards - of course configuration files can be easily parsed otherwise those configuration files would be just junk on the disk. In actual fact most configuration files are relatively easy to parse that is why they are the way they are. But you are right that most developer do not want to put the effort into parsing someones else's configuration files to only get one or two pieces of information. On Meta-data Meta-data is a very simple concept it is descriptive data and relationships (in a machine processable form) that are not obvious simply looking at the data. Typically it exposes relationships that are hidden deep in the code that processes the data. The greatest problem in complex interrelated systems is that simple inconsistencies in the configuration can have knock effects in other sub-system that are hard to determine. With this meta data there is no simple way of validating configurations particularly when you go across systems. Configuration files often have complex dependencies in them, for example XFree86 and the NVidia driver implements its own GLX and incompatible with DRI, in order to use their driver you have to remove not only the open source 'nv' drive but also glx and dri. Examples like this are abound in configuration files. Some of this information becomes fold lore others are plain black magic. This is where the issue of knowledge management comes into the picture. Now I am not talking about some vast complex AI to manage configuration, but the simply information that provide the following: 1. Locations and names of config files 2. How to check for nonsistancy 3. How can you clean up data after components have been deleted 4. Ability to create 'virtual' data that is derived from several config files (using som kind of matching rules) Knowledge and understanding the inter-dependencies between configuration files are exactly the problem and within. Finally, I really hate criticizing people that are doing useful work but I strongly feel the the effort is futile, so please take is in the spitit in which it is given - constructive criticism.
Re: Why re-invent a broken idea - Andre Somers - 2004-08-31
Although I am critical to the current form of the LR, I think your critique takes it a bit to far. Therefore, a reply. >My basic points are as follows: > 1. It is too hard to get every application to change is configuration storage scheme to the registry and the registry is only useful if everybody uses it. I think it becomes usable before that, but I agree you need critical mass. That will be hard to accomplish, and in order to accomplish it you need to really offer something compelling for application developers and packagers alike. > 2. Even if you could persuade everybody to use the registry you do not significantly simplify theproblem of managing configuration information and you loose the entire tool chain of utilities that have been developed over the years to process text files. How is that? I think the idea of a single tree does simplify managing the systems configuration, if the system is implemented well. > 4. There is a legitimate need for other application to interrogate the configuration information, but is the registry the best solution? If not, what is it? The LR is a legitimate attempt at providing this functionality. > 5, 9. The key concept missing in the MS registry is meta-data that can vaildate the consistency of the data across configuration files I'll get back to that point later on. > 6. A method by which other applications such as installers, configuration GUI can manipulate the data in a way that is both safe, consistent, auditable (see who has done what) and a means of reverting to known good configurations. A means to return to a previous configuration and a good log would indeed be *very* usefull. I do think that such a thing could be far easier accomplished in a LR-like system than in the current situation, so that's progress. > 7, 8 and 10 There should be a dynamic file system (a kind of registry cache) Why is that? Why a filesystem? What is the advantage over a clear API targeted at this specific problem? I am very curious to your ideas about meta data, and especially the validating you claim you can achieve with them. I really don't have an idea how you could do that, even in the example of the 3d drivers you give. How in the world can I specify how exactly the values are to be validated, without basicly re-implementing the entire settings parsing of the whole system? Consistency checking is complicated even on relatively simple database systems that house almost homogenious data. How on earth would you propose to implement such a thing for the very heterogenious data stored in configurations?! How would you know in advance what kind of conflicts may occur with other installed packages? You summed up where Microsoft managed to screw up, but in my view, you give very little usefull hints on how to solve the issue at hand. The fact that they screwed up, does not mean that any project creating a hierarchy for configuration settings is doomed to screw up too. I think adopting such a system, if well thought out, could actually solve many occasions where users today in Linux _need_ to tinker with configuration files because it's so hard to write a good tool to can actually work with the different files.
Re: Why re-invent a broken idea - Nurul - 2004-09-01
I am strongly in favor of having a mechanism for managing configuration better, but I strongly believe that it should be something that is built on top of the text based system - that should be the underlying storage for configuration. If in the mean time we can get to a stage where there are only a few limited formats for configuration file that would be a significant asset. The higher level representation would be something that is created from the current active configuration them the sys-system is loaded (usually). There are cases where we would like to get the configuration of a sub-system that is not active at the time, for example X configuration when no X server is running. We can discuss how that would would be done later. The advantage of the scheme is that it represents the actual configuration that is running. There may be many alternative configurations for different situation that would not be in the dynamic registry. For example take the case of an Apache web server; there can be many instances of the server running on the same machine, for example on port 80 for the public site and on port 800 for an intranet. The two server would have completely different configuration files and that is easy to manage currently. With a central repository this starts to become more complex - you can imagine yet more complex scenarios. If i decide I would like to run my copy of Apache on port 8080 I would just copy the config file into my own directory and change the port to 8080 and I am done, I do not pollute a global registry. If I no longer reed the server I just delete the directory and everything is gone. IN the case of the registry if I do not do the house keeping on the global registry I now have c**p left after a while the registry will become a pile of poop. Coming back to the multiple Apache scenario, the simple original scheme of having one place where the Apache configuration resides is now out the window. I now need a forest of configurations and some scheme to manage them. That has been my point about the registry; in the real world the encoding in the registry will become ever more complex - we see that in Windows. >> 2. Even if you could persuade everybody to use the registry you do not significantly simplify the problem of managing configuration information and you loose the entire tool chain of utilities that have been developed over the years to process text files. >How is that? I think the idea of a single tree does simplify managing the systems configuration, if the system is implemented well. My basic proposal is tat we do not change the configuration files that are currently used. They are text file we have lot of tools to process test files, editors, macro generators, diff utilities, backup copies, multiple versions, and even manage version control using CVS - I have seen this dome in some large organizations. The registry tries to solve 2 problems: 1) A simple consistent method of storing configuration information 2) Since the method is uniform it allows any other application to interrogate the configuration information or even modify that information. On point 1. the principle of a registry is very good but the Microsoft experience seems to suggest that in real life a simple hierarchy is not sufficient and you end up with a convoluted scheme of links pointing to other links that are just as hard to navigate and interpret as text files. On point 2. I completely agree that the current configuration files are to numerous and convoluted and too frequently changed in subtle ways to make it useful for others to interrogate. My suggested solution is as follows: Create a registry of some sort - I have not fully thought through is the MS like registry is sufficient - I think not, but I will discuss those options a little later. This registry is created from the original config files. How would that be done in a sane manner? Clearly each sub-system development group know about their config files, it would be best if they write a script - typically written in some scripting language, for example perl or python - that would convert their config file into standard text representation that can be imported into the registry. This standard representation should be a "specification" that is created as a part of the 'config registry' project. I am in favor of having an 'minimal' XML format for this intermediate file, but a good alternative might be a LISP like s-expression form - these really are equivalent (so much of computing is rehashing old ideas in a new format and nomenclature, we seem to be very bad in computing when it comes to studying anything that was written more than a few years ago). I am guessing that the best verifiers for configuration system would be some type of simple rule-based systems that were so popular in the area of expert-systems that became popular in the 80's . Although these systems turned out to be unsuccessful in the arena of encapsulating large scale human knowledge, but I have a strong feeling that in the domain of encoding knowledge about system interdependencies and configuration verification. A simpler scheme, but not necessarily better, would be to just have verifier script associated with each sub-system that would walk the dynamic data a check-off the consistency of the data. I definitely do not think that a single verifier would be the way to go but a get of verifiers what would be tried in turn to validate the dynamic data. For example, NVIDIA driver if it is installed will not work if the 'nv' driver is loaded or if glx or dri is loaded. The verifier rules should be able to be written as simple rules such as this: ((Domain XFree86) (IF (display-driver 'nvida') ( (provides 'GLX') (provides 'twinView') (incompatible-with (module 'dri' ) (message 'nvidia driver cannot coexist with module dri') ) (incompatble-with (display-drive 'nv') (message 'nvidia driver cannot coexist with display driver nv')) (incompatible-with (alternative (provides GLX)) (message 'nvidia driver already provides glx, remove the glx module')) ) } I personally like the like pattern matching approach to verifying configuration, but a more traditional programming approach would also suffice. But the important point is that the verifier would work against the 'dynamic' config information that would be created from the text files. >7, 8 and 10 There should be a dynamic file system (a kind of registry cache) > Why is that? Why a filesystem? What is the advantage over a clear API targeted at this specific problem? The file system has a many advantages over some special API (although that could also be provided), all the tools to manipulate the file system and files would be available with no extra effort - find, index, grep ... This is a problem that has always plagued computing - namespace pollution. The whole motivation of OO programming was to reduce this namespace pollution, things that are similar (isomorphic) should have the same interface. It keeps the number of things that a person has to manage and remember much smaller and manageable. For example in windows file handles and socket handles are different, so the API get ever larger. I hope you get the point - see Plan 9 or Reiser4 for more detailed discussion on the benefits of unifying namespaces..
Re: Why re-invent a broken idea - Thomas Bettler - 2004-09-08
>except we have decades worth of tools to manipulated and process files and >directories, we have nothing useful to manipulate Windows Registry. Of course this may be true. But why? Because it's all property *without any docs or man* for deep system configuration. I believe, Elektra comes on Linux there *1 will be / and has to be manuals about the registry entries (what would all those cryptic config files be without *5 manuals?) *2 there will come easy to use tools (like for me mother) to configure settings all about the whole system.
Re: Why re-invent a broken idea - Mod - 2005-11-03
A large group(most of the pc users) dont even know what a config file is. This is part of the reason linux isnt used by people outside of the computer science field. Could you figure out how to tell a person from this group, over the phone or threw email, to change value x to some other value. Its hard eneuff to get them to find the file first. In windows if i want then to remove a regKey i make a .reg file send it to them in email and tell them to right click merge. The trick with the whole system is this. Admins who are going to use windows as there primary OS need to be able to use the registry. The users have to know nothing about it.
Re: Why re-invent a broken idea - David - 2004-08-31
You missed the main good thing about the Unix configuration "system": it's decentralized. When the single file known as the Windows Registry is corrupted the whole system is affected. But when a single file under /etc is corrupted, only the specific application its for is affected. The Unix system is even more decentralized than just /etc, because on most systems you can have a /usr/X11R6/etc and /usr/local/etc as well. Oh, and don't forget the dotfiles under the users' home directories. What missing in Unix isn't a Microsoft-style centralized authority, but merely a certain amount of consistancy and minor details. The numerous config file formats is unfortunate, but this can be solved easily without resorting to a central registry. Lots of dotfiles cluttering up the home directory is also unfortunate, but this can also be easily solved by using a single ~/.etc directory.
Re: Why re-invent a broken idea - Miro - 2004-09-01
1. I like the idea. The problem with windows registry is that there is no olicy enforced by the system. I mean an application can create a key (almost) anywhere. There is also no consistency with placing files on windows too (everything in c:\windows, even if you was dumb enough to install windows on d:\ *EVEL-GRIN*). The point is, LR must enforce a policy where an application is allowed to make changes. My proposal: 1. each application would only be allowed to crete 1. level of directories (keeping it simple) add files to writable locations (MIME -> for default app for filetype resolution), read where allowed 2. root apps could do anything. 3. Force a layout! and force application to follow User1. local overrides for Apps (only?) Apps (eg. /usr) MIME App1. App2. etc... System (eg. /etc) Sound Video etc... etc.. (no idea if needed:) 4. Keep a lot of info about who (App@user) when created, modified, deleted a key. I dont care if LR is 100MB large if it lets me actually my system easily) DONT believe developers will EVER follow a policy, and even if they do they do make mistakes. So PLEASE be radical, this is a radicaly new idea on unix.
It's time for registry - LuckySandal - 2004-08-30
I think most everyone can agree that the current linux configuration system is broken. There is such a wide variety of config file formats that no central configuration utility will ever be able to parse/modify them all. Registry seems like the obvious solution. Having everything in a standard locations, in hierachial tree, with comments, modification times, permissions, symbolic links, and the ability to read / write smoothly using a common API seems like the way to go. Unlike Windows, however, we must make sure that OUR registry makes sense, and that there is a well-defined standard for key locations by which applications must abide. Given the sucess of the UNIX filesystem hierarchy, this doesn't seem like it would be hard to achieve. How often do you have a UNIX program insist on installing files in the wrong place? Compare that to Windows. Fears over registry bloat as on Windows are unfounded, because this is already a problem with regular configuration files and can be fixed with access times. Saying that we shouldn't have registry on Linux because it failed on Windows is like saying we shouldn't have government because communism failed. Or that shouldn't have Desktop Environments because, after all, look at Windows! If there is ever a distribution that uses registry exclusively, or a Gentoo USE flag, I will use it! Kudos to open source for leading the UNIX world toward the future!
my vote is also no - jack - 2004-08-30
I have run into too many programs that use the windows registry for a database and it just keeps growing bigger and bigger also i do not think is a good to let just any program change values then we end up with the mess that drove me to linux in the first place Jack
Re: my vote is also no - LuckySandal - 2004-08-31
It's not geing to "let just any program change values." There will be permissions on every key, just as with files. And even if it "keep growing bigger and bigger," how is that any different than the dot files in your home directory? See my above post.
Re: my vote is also no - jack - 2004-08-31
I had a program that i used in my business called "Wirsbo Radiant Express" that after i installed i noticed that the registry had doubled in size it was then i noticed that the program was using the registry for its own database this then slows down every program on the system that has to query the registry not just itself which is the case with config or dot files one thing i don't under stand is how to manage the permissions on every key i can see it taking longer to figure if you want the program to change a certain key how to tell if that key belongs to that program or another i like the idea that i know that a certain config file belongs to which program Jack
Re: my vote is also no - Tom - 2005-10-30
Is this program worth th cost then?
Re: another thought - jack - 2004-08-31
The more i think about i believe if it is well thought out it could work but you have to take into consideration that as linux canes in popularity there will be allot more developers writing software for linux not all gnu programs like the one mentioned is writing for a limited number of users and not likely to be open sourced are there going to be enough checks to make sure they do not abuse registry. why do you have to call it a registry i think if you called it another name more people will look at it with an open mind and not revulsion that the window registry invokes Jack
Project name change: Registry --> Elektra Project - Avi Alkalay - 2004-08-31
The Linux Registry project changed his name today. New name is ELEKTRA PROJECT. The website is still http://registry.sf.net, but not for too long.
Good idea, but... - Andre Somers - 2004-08-31
First of all, I really think that in the basis, the LR is a good idea. I agree with the observation that the current situation is a mess and holds back good system integration. I also agree with the idea that we don't need a big, monolythic database like windows, but that we do need a universal key naming system. So far, so good. The problem is, so far, almost all of the profits of the proposed system are in the field of the system integration, something a distribution takes care of most of the time. I hardly see any profits for the application developers themselves, only a lot of work in porting loads of programs to use the LR. That is a major drawback. Another point - that has been mentioned before in the above reactions - is that configurations need to be layered. The very existence of Kiosk is testament to that. You say you don't want to solve that problem, and that the LR is a low level system on top of which this could be implemented. I think that is the wrong choice, for two reasons. The first is basicly the same as above: what is the profit for the application programmer if he needs to do the hard work himself anyway? Another problem is more fundamental to the design: it is a potential trap for the whole system you propose. The beauty of the LR was a single, consistent way of storing and retreiving settings. However, by ignoring layering of settings, you immediatly end up with the settings for the same thing on different places in the LR and probably outside of it as well (LDAP, SQL, etc.). You choose not to do all this, because you want to keep the system simple so it a) does not form a single point of failure, and b) is available for very low level services. I think these problems are solvable, even if you offer more than just a basic service. You could think of the configuration system in much the same way as the filesystem. That works roughly like this: Initially, only the boot partition is available, wich is in a format the kernel understands. Later in the process, more filesystems may be mounted, some needing higher level services that were not available at boot time (NFS for example). Right? Why not work in much the same way for the LR? Have a basic configuration system that is availble without any external dependencies. As the system is booted, more parts of the configuration system could be 'mounted', possibly with other and more complex backends. Each mounted configuration resource could form a layer, that augments and/or overrides the layers below it. Such a system, although I admit more complex, would offer significant advantages to application programmers, as it frees them from making configuration in layers possible themselves. They only need to access a single key for each setting, and in 99% of the cases don't care where that key was set. At the same time, it frees you from the problem that application programmers or distributions may organize their settings layering in different, incompatible ways, nullifying the advantages of the LR in the process. Next, it would make it possible to integrate some important sets of settings into the system without changing all programs at the same time. I think this will make the transition easier. I really think the LR will have a much higher chance of being accepted if it offers real advantages for all parties involved, if it really offers something more than is available now.
Re: Good idea, but... - Andre Somers - 2004-08-31
Another small point. This comes from the LR homepage: == There are very good reasons why types like Integer, Time, Font, List, etc were not implemented: The Registry was designed to be usefull for any type of program, so having more specific data types implicates in the definition of value limits, separators in the storage format, etc, that may be good for some application and bad for other. So the semantics of the data is handled by the application. A program or framework may define its own special data handling methods using these essential basic types. See the keyGetType() and keySetType() methods documentation in the registry(3) man page to understand how to set keys with your own data types. == I'm not quite sure I totally agree here. I think it would be *very* usefull to at least be able to store some basic keytypes in a consistent way. UTF8 for strings is a Good Thing (TM). If you can decide on an encoding for the basic string type, why not on an encoding for a small set of other basic types? I think that would only make it easier to get a consistent registry, and would promote integration. Applications that want to use other types can still define them, but an option to use at least basic types like integer, boolean, date/time and perhaps a _few_ others would be very welcome I think. In my experience, these are the types that are most often used in configuration files. Maybe it would even make sence to be able to define a list of values with the same type for a single key.
Yes!! - tomten - 2004-08-31
<RANT> I was a fulltime Linux (work/home) user for 3 years until 4 years ago(Suse 5-7/Debian). When Windows 2000/XP came along the whole blue-screen mess was rectified and I now use Windows fulltime, but I regularly install Linux to follow it's progress. At the *system-level* other than printing (CUPS) and dists like Knoppix I must say I'm not impressed. Time has stood still in UNIX-land as the competition at the same time is miles away from there 90s. One thing that stands out is Linux distros flakiness when it comes to updating configuration files. I still find myself often have to drop to the terminal to fix things. Just look at Suse, they probably still have a horrid Perl script updating all configuration files behind that flashy GUI after a small software install. My conclusion is that the (nonexistent) configuration system on Linux takes so much time to keep in shape that they newer have time to do serious quality control. Unix needs more projects like CUPS. Recognizing that LPR was shit they forged ahead ignoring the pro LPR-dinosaurs and designed a killer system that proved them wrong. I think Linux registry has the same potential but at a much grander scale and I wish it the best of luck. </RANT>
Re: Yes!! - Avi Alkalay - 2004-08-31
Thank you very much for your practical, real-world vision :)
we can do better than windows - Nurul - 2004-08-31
How about handling different configurations files (under different circumstances), or comparing changes to the configuration files. I could go on - the problem is not the registry per se but the fact that we through out all those test tools in Unix/Linux that have been developed over the years to handle all these thing and much more. The registry is conceptually good but let us not try to emulate the Windows crap. You talk about 99% evangelizing - perhaps some of that energy cold be put to better use if we could get more discussion about the issue. I am glad that your team has started us on the process - it is very important but let us go to a place that we can be really proud of instead of trying to mostly emulate an idea that failed to deliver on the Windows platform.
Re: we can do better than windows - Avi Alkalay - 2004-08-31
I see your point. But lets try to see it from a business (and not technicall) perspective: MS wanted to deliver a fully and completelly integratable system (programs make the configurations for you, preciselly). They found their way putting all together on a configuration tree that can be accessible by absolutelly any program. I admit this is not the only success factor for MS, but when I think about the power it provides, I'm pretty sure that this is a key point in Windows success. And I admit Windows is an extremelly successfull platform. How they did it, the technologies they used, the way it was implemented, etc... these are just operational details. From a successfull business perspective, it does not matter. And if its essence is adopted, good minds will join the effort and will help improving it. From a technical perspective, yes, we'd like to do a better job then the Windows Registry, but we also learned that too much complexity will make it unaccessible for simple minds, and maybe optimized for only one purpose (which deoptimizes it for other purporses). Oh yes... and also too hard to implement, more likelly to have dependencies, more bugs, etc. There is a wonderfull document by Doc Searls (one of the Linux Journal colunists), very inspiring, that explains why the Internet is simple because it is a success, and is a success because it is simple. If you have time, I really suggest reading: http://worldofends.com/ http://worldofends.com/#BM3 http://worldofends.com/#BM4 The entire document is about 15 minutes reading that will change the reader's life. (ok, not that much...)
Re: we can do better than windows - Nurul - 2004-09-01
Look don't get me wrong - I thing it is wonderful that you guys have put the effort in solving a significant issue in Unix/Linux. I have been very critical because I strongly believe that the Windows registry has not solved the configuration problem for Windows - it has actually made it a more brittle system. If you have been following advice from windows GURUs they recommend completely reinstalling windows once a year. I agree this is not only the fault of the register, but it is a large part of the problem. My suggested approach is a hybrid one - keep the current config file scheme. Encouraging developers to use a much small number of differing formats would be a great help, but not necessary. The hybrid approach is that we have the concept of a 'running' config repository that may well be your registry. This 'running' registry is updated when a sub-system is started. The update would typically be done with a script the would be provided (hopefully) with the application. After the running registry has been updated, we also could have a series of scripts for each subsystem that would do some consistency check on the configuration data, this could be a very complete check or even a superficial sanity check. But the very best aspect of this dynamic config registry is that it would allow other applications to check to the actual configuration information in use and to verify that the necessary precondition that the application requires has been met. This would not allow the configuration to be changed programatically, but that issue can be tackled later. An arrangement that i favor is to have a script (again provided by the sub-system developers - hopefully) that would create a config from the dynamic data. When all is said and done, text representation has been the most successful invention of computing. Over time we see reverting back to text representation that were once binary - that really does say something. AGAIN it is great that you and your group of developers have made this first step but I feel you have not gone far enough.
why can no-one read? - superstoned - 2004-09-01
most negative comments here are just because of the name "registry". if the name was different (or the ppl ranting had just *read* what the linux registry is all about) they whouldnt complain. please read before ranting. because its more like trolling, if you dont understand what you are talking about. ppl say "I want to be able to read my config files myself" well thats possible with the LINUX registry (no, not in windows, but are we talking about windows or linux registry?) ppl say "I dont want ONE huge database that can easilly get corrupted" well no problem, the linux registry IS NOT a database, but a bunch of seperate files etc etc etc so shut up if you dont know what you are talking about, but first see what it *really* is... (more like a common standard for config files so a central program, the linux registry, can access them easilly, notify about changes etc etc and let other programs change things too without difficult parsing etc etc) (sorry its very irritating to read through all these comments, and see most are just plain stupid)
Re: why can no-one read? - Christian Loose - 2004-09-01
Well said! Exactly my thought when I would through those posts. Christian
Re: why can no-one read? - superstoned - 2004-09-01
yeah, poor Avi replied to (almost) all of them, he's just too nice - I whould've said "RTFM" instead...
Re: why can no-one read? - Avi Alkalay - 2004-09-02
:-D
Re: why can no-one read? - Zooplah - 2004-09-01
Well, I had similar ill feelings when I read about it. But I read the article, and it sounds like it avoids my two problems with Windows registry. Now, I do think that it's a better implementation than Microsoft's (though the two aren't directly comparable). However, I don't see the big problem with the current configuration (/etc/prog.conf, ~/.progrc ~/.prog/*), but I guess an easy way to access the settings programatically without each program making it's own parser is nice. Let's just hope that it's: 1. Portable (Linux, BSD, Mac, Windows) 2. Accessible, with any programming or shell-scripting language 3. Easier to programmatically access than Windows registry (the norm for Linux is easier API) 4. Includes scripts to carry over your old configuration (apache2reg, for example).
Achilles Heel - Graham Daniell - 2004-09-02
The registry is the Achilles Heel of Windows. It is the worst Windows "feature" of them all. It is what makes Windows so hard to deal with and so impossible for ordinary users to understand. We need a Linux registry like we need a hole in the head. Graham Daniell
Re: Achilles Heel - Andre Somers - 2004-09-02
We don't need the bad implementation used in Windows, but we do need the integration. If you can come up with a better idea, please go ahead. If not, please stop trolling.
Re: Achilles Heel - superstoned - 2004-09-02
READ!!!! damn, can't you read? just in the post above yours I said there are so many stupid comments here because no-one actually reads more than the name: linux REGISTRY. but its not like windows registry, exept for the fact that it tries to solve the same problem. it has none of the problems the windows version has, just READ the damnit article. jeeez
manual for kdb configs? (according to 5* man) - Thomas Bettler - 2004-09-08
Apps using Elektra should of course provide own manuals about available settings in the kdb (as 5* manuals do for the config files) as metadata. The manual could also be translated into other languages. Woudn't it be helpful to specify a machine readable standard for such manuals (so any higher level UI to kdb can provide this metadata. (i.e. like Kconfig does for the linux kernel). Or is this behind the scope of your project and therefore already a higher level project itself?