aKademy Interview: Avi Alkalay of Linux Registry Project

During the
KDE Contributor and Developer Conference at aKademy one
of the speakers Avi Alkalay talked about his project
Linux Registry. We met him
afterwards and asked him several questions about this project.

Please give us some background information on yourself.

My name is Avi Alkalay, I worked with Linux for 10 years and the last 2 years
I'm doing Linux and consulting for IBM in Latin-America. Helping customers migrate
to Linux and Open Standards.

What's the focus of this project and how did it start?

The objective is to provide an infrastructure to make the Linux configuration
nightmare disappear. This can be achieved through a way to let programs
integrate automatically, without having to ask users to manually
edit their configuration files.

How many people are working on this ?

30+ members. Many contributions, language bindings and suggestions.

Are you paid to work on this project.?

No.

What's wrong with the current way we are managing configuration files?
In your talk you question that we need to evolve the underlying OS structure.
Why is that?

A lot happened in the desktop space: DCOP, XML-RPC, components, widgets, colorfull themes, fonts, etc etc. but
nothing evolved in the base OS organization. You can't have a wholly integrated desktop (as a computer for my mother)
without evolving the underlying OS. KDE or Gnome don't really deal with plugged webcams, new video cards, etc.
And the way you integrate this kind of things today in the system makes use of your human brains and eyes
to edit text files like modules.conf or XF86Config. A software can't really write this configurations
for you effectively, without a considerable amount of complexity or artificial intelligence :-)
It is programatically difficult to parse and change some configuration bit in a human readable configuration file.

Each software project (KDE, Gnome, Apache, Samba, /sbin/init, modprobe, etc.) has its own way to define the
format of their (text) configuration files, so this make them completely separate and unintegrable software.
Configuration is the soul of software.

Current UNIX systems organization was designed 30 years ago. It has many strong points, like the
filesystem hierarchy, etc, but nothing was defined regarding the format of the configuration files.
I'm talking about a common format.

Also, commodity hardware (video cards, webcams, mice, keyboards, USB, etc.) vendors use to not support
Linux (even if the kernel has drivers for them) because they don't want to deal with complex,
multi-distribution setups, like "in distro A, edit this and that file this way; in distro B version N,
run tool XYZ, then edit those and those files in this other way", etc. That's awful.

Is there any interest from other developers or projects for this concept of using key trees for storing configuration data

A lot of interest is showed from many sides. We have people writing patches, GUI tools, language bindings, etc.
Check the project website to see some of them.
Waldo Bastian showed some interest to see how KConfigXT (KDE Config framework) will behave with a
Linux Registry backend.

How would the Registry integrate with existing frameworks like KConfigXT and Gconf?

These frameworks are more high level, designed for more specific use: desktop software. Registry was designed to
be the most generic and low level configuration system you can get, so it can be used as a backend for these
frameworks. Then, you'll be able to access Gnome programs configurations with KDE tools, and vice-versa. And also,
access base system configuration with KDE or Gnome tools.
KDE and Gnome applications source code will still use KConfigXT and GConf API, but their infrastructure will be a registry.

So what do you think are the advantages for Linux and especially for KDE? Is this something which is
of importance in Userland?

Linux Registry was designed to be the configuration store of everything on the system, not only for the
desktop part. So if we get it adopted, we'll be able to see programs more easily integrating with other programs.
You don't need to burden users to edit config-files when software can do that on his own. This will make life
easier also for software and hardware vendors to deploy their products on Linux. So that way a new program is
easily pluggable into the system.

The key names in your presentation at aKademy were similar to existing configuration file names, but still confusing to the
newcomer (e.g. user/env and system/init) Do you not think it would be a good opportunity to rename these keys
(e.g. from "init" to "startup")?

Those were just examples .. I choose those names just to be easy.
But keep in mind that keys should not be accessed directly by users. Stable semantic-oriented GUI configuration tools will
appear asking things like "click here to install and configure your new webcam". Then, the real keys and key
structure that is happening bellow that is sysadmin stuff, and hopefully he'll have to deal with them directly
only in rare situations.

The layout of nowadays configuration files are defined at the packaging time. In the office of RedHat, SuSE, etc.
This leads one Linux to be slightly different to another Linux. But this small differences makes impossible
automatic integration and configuration.
Switching to a tree of keys and values concept, makes the application development team the owner of this layout
(which should be unique). So distro A, B and C, X.org, a video card vendor, a GUI configuration tool, etc,
will allways find the current video card driver in the key system/sw/XFree/Device/Videocard0/Driver, instead of
line 34 of file /etc/X11/XF86Config on distro A (if user does not edit it), and line 42 of file
/etc/X11/xorg.conf on distro B (if user didn't edit it).

How about using XML .. isn't XML invented for this kind of stuff

XML is a wonderfull standard to represent any information, make it portable, human and
machine readable. It adds a considerable amount of complexity too.
Registry was designed to be available anywhere, anytime on the system. For this, all required libraries should
be there anywhere, anytime. If Registry used XML for its storage, libXML would be required, and the XML library
may not be available for some early boot stage programs, like /sbin/init. Well, Registry keys can be used for
/sbin/init configurations, instead of the 30 years old /etc/inittab.

Anyway, Registry uses XML as its universal format to import and export keys into/from the key database.
This helps on transferring software configurations from one machine to another, backup, etc.

Wouldn't using this concept of trees of keys be unreadable for the human eyes. Better said: can an
administrator still edit the files?

Yes, he can. This was a long discussed subject, and one of the weaknesses of the Windows implementation.
Registry storage are plain text files, editable with vi or ed. But, thinking out of the box
(considering time), an administrator will not have to edit keys directly. A registry infrastructure makes semantic
configuration GUIs development be very easy to do. So with adoption and time, administrators will use more semantic
configuration tools, and they'll be able to edit keys directly too, if they need.

Don't you think that this name, Linux Registry, can be problematic? Many system administrators had bad experiences
with the Windows Registry.

In my opinion the Windows Registry is a good idea behind a bad implementation. It is one file to store it all.
If you loose this file, or it gets corrupted, you loose your system.

On the other side, the Linux Registry is not a daemon (so no single point of failure), does not store
all configurations on a single file, was designed with security in mind, and stores user's configurations
in each user's $HOME, and system's on /etc.

Anyway, I receive many many e-mails from people liking the design but feeling the name as an obstacle to
adoption. This name was chosen to make an automatic awareness in people's mind. Sometimes it works for
the wrong side. So in a few days, the name of the project will change to something very cool. Stay tuned!

Where do you see the future of Linux Registry going?

Hopefully it will be accepted by major projects like KDE, Apache, Samba, etc, and distributions.
Also, more storage backends should appear, more language bindings, etc.
Linux/BSD/Unix today is a 1+1=2 system, and Linux Registry can help it become a 1*1=1.

The project needs more programs using it, more awareness, more adoption. The advantages of a wholly
integrated system will only be felt when many programs will be using it, so use it!

It is free, designed for Unix, by an old Unix user, to old Unix users.
So give it a try: http://registry.sourceforge.net!

Dot Categories: 

Comments

by johnflux (not verified)

"I agree configuration may be painful but a registry may give root privileges to an automated userland process and virii will be a possible nightmare in Linux"

Stop FUDing. How is it going to give root privilleges? There isn't even a process(daemon) that could possibly be SUID.

by Avi Alkalay (not verified)

If you could just READ some points of the documentation BEFORE writing some random crap you'll see that this is an issue LR does not have.

http://registry.sourceforge.net/#securityexample

BTW, LR spends no single line of code to handle security. This is good because this is all delegated to the underlying kernel.

by Nurul (not verified)

I completely agree that the Linux configuration information is a big problem because it is so ad-hoc. The real problem is that Unix/Linux does not have meta-data about the configuration files:

1. Which are the configuration files
2. What are the interdependencies in the various configuration files
3. Large number of formats.
3. There is no consistent method of notifying components that the configuration has changed
4. There is no high level API to interrogate the configuration

The good things about the configuration information in Unix:

1. Human readable
2. Mostly fairly simple to understand (with a little bit of documentation)
3. Fairly use to use version management of config file using CVS - although not as common as it should be
5. Most config files can be easily read by programs

The windows registry was a great idea (10 years ago) that turned out to be badly broken. Perhaps the single biggest problem in windows is the registry and in particular corrupted registries. The problem is that it is really easy to semantically corrupt the registry and very difficult to discover where the problem lies. Having no real semantic information it is up to the applications that use the registry to make sure that everything is consistent . Over time these applications change, different developers add yet more keys to the registry often is places that may have seen like a good idea at the time but nobody bothers to document it and bit by bit you end up with this tangled pile of s**t that nobody understands. So the standard recommendation for MS Windows is "re-install windows once a year to clean out all the registry c**p.

There is an old saying - every problem in computing can be solved by adding one extra layer of indirection. Clearly, configuration files are themselves one level of indirection: do not hard code values in the application place them in a configuration file. So add another level of indirection, this data would act more like the index files iin a ddatabase, that is the information can be reconstructed form the original configuration files. This meta-data would not be a replacement for the underlying configuration file but it would be a more convenient 'view' of the configuration files.

Think of this as a knowledge base in the AI sense of the term of the underlying configuration. So how do we solve the problem of all those different formats of the configuration file - we don't.
I propose that just as the shell has the:

#!/bin/sh for shell scripts
#!/usr/bin/perl for perl scripts

there would be a similar arrangement to each configuration file that would would update the central meta data when the file has been changed. This update utility would also extra information about how to check for consistancy, allow changes to the meta-data to update the configuration file

In order to set up such a meta databse we will require some meta-config files:
a. that lists the locations of all
b. list the file containing the consistancy checker rules
c. Meta information about version managment
d. Whom to inform if there are configuration changes

IMHO we have to apply the tools of knowledge management to handle system configuration, the suggestion that The most significant benefit of this approach is that no application has to be changed in order to use the system. Over time many applicatios would cease using the configuration file directly and use the meta-data but that is their choice.

I have made may for the basis of such an approach. I hope this this will encourage others to consider this approach and build upon it. This is a very crude outline of such a system, the next stage would be to create a prototype.

by Avi Alkalay (not verified)

My dear, IMHO you have the wrong vision about how things happen in customers, are missinformed about what LR can do for you, WRegistry issues it avoids, what is good and bad on Unix, etc etc etc.

Windows registry is a good idea with a bad implementation. They didn't think about security, single point of failure, etc at that time. The most painfull one is DB corruption. LR avoids it all. Read the documentation.

About that "tangled pile of s**t that nobody understands" that grows over time, I totally agree with you. But go take a look in /etc and in you $HOME dotfiles. You'll find the same ammount of s**t. Anyway, LR provides mechanisms for you to find keys that were not accessed for more than a period of time, that are candidates to be removed.

Notification is not that important. It is usefull for some desktop apps. And you probably don't want Apache to automatically restart itself right after you change something. Anyway LR supports change notifications....

About the good things on Unix config style you said:
- Unix geeks are not humans. Humans are people that don't even know how to use vi. I'm talking about my mother to use Linux.
- Config files version management is a tale. Real world people (not geeks) don't even know what is version control. I travel the world visiting and evangelizing Linux on real businesses, and 150% of them don't use it. Anyway, you can do this with LR.
- About 0% of nowadays config files can be read by programs. Go to NVidia website (a big supporter of their products on Linux/BSD). Their products READMES tell you to edit X config files by yourself. It is simply too much work to do: you'll have to write configuration files compilers :-)

About this all meta-data things you said, thats too complex, pure rocket science. Please go to see how real businesses use computers. They just want to make their stuff work. If such meta-data thing existed it would not be used on real world, or it would become an industry of inconsistency. That recalls me Linuxconf, an abandoned software, because it was too complex to maintain such a rich set of configuration files generators, and creates inconsistencies.

Knowledge management for configurations? Oh gosh.... rocket science again.
Think simple, please.

by Andre Somers (not verified)

> Anyway, LR provides mechanisms for you to find keys that were not accessed for more than a period of time, that are candidates to be removed.

Maybe it would also be usefull to see by what application they were put there. That may be more informative than the question when they were last accessed.

>About the good things on Unix config style you said:
> - Unix geeks are not humans. Humans are people that don't even know how to use vi. I'm talking about my mother to use Linux.
Your mother will not be accessing the LR directly either, nor would she ever try to modify the windows registry for that matter. Sysadmins *do* read and edit configuration files, and are glad they can.

> - Config files version management is a tale. Real world people (not geeks) don't even know what is version control. I travel the world visiting and evangelizing Linux on real businesses, and 150% of them don't use it. Anyway, you can do this with LR.

True. However, backing up configurations is not. I find it a treat that I can back up my entire configuration by just packaging my homedir and /etc.

> - About 0% of nowadays config files can be read by programs. Go to NVidia website (a big supporter of their products on Linux/BSD). Their products READMES tell you to edit X config files by yourself. It is simply too much work to do: you'll have to write configuration files compilers :-)

Yes, but OTOH: migrating everyting in one go is too much work too. In some cases, it might be very profitable to create a configuration file compiler. That may be hard for some cases, but I doubt it will be harder than converting all important systems in one go.

>Think simple, please.
Yes, but don't oversimplify.

by Nurul (not verified)

I actually happen to have a fair amount of experience with the Windows registry and I hate it with a vengeance. For those of you who hate Microsoft please bear with me - I am not here to praise Microsoft; I would like to benefit for their experience with the registry and show you why it is one of the most hated and feared component in the system. Far from being a tool that an end-user would change MS actively discourages sys admins from editing it. So the notion that editing a registry is simpler that a config file is not borne out by experience.

I remember when it first came out I was very excited about the prospect of getting rid of all those INI files that were all over the place and with no consistency about naming of placement. Windows registry would give a simple hierarchical storage that was searchable, allows atomic updates, a single location to keep all the data, and easy to organize cleanly.

As they say be careful what you as for you may get it - and boy did we get it. Far from creating simplicity we ended up with an organizational nightmare. Applications would leave their poop all over the place, they would update parts of the registry that would make you wander what was the developer thinking. You would uninstall an application and the uninstall would remove 80% of the entries the application had created, the remainder would be lurking somewhere in the 100 of thousands of entries that is the typical registry.

The registry tries to solve a legitimate problem - manging configuration files in a complex highly interdependent system; it has been a miserable failure for Microsoft despite the fact that they dis not do anything too stupid implementing it. I completely disagree about your comment that Microsoft's registry was a failure because they had a poor implementation. IMHO the failure arises because MS did not realize that manage configuration is more than a simple matter of having a central repository of data.

My basic points are as follows:

1. It is too hard to get every application to change is configuration storage scheme to the registry and the registry is only useful if everybody uses it.

2. Even if you could persuade everybody to use the registry you do not significantly simplify theproblem of managing configuration information and you loose the entire tool chain of utilities that have been developed over the years to process text files.

3. No user directly manipulates the registry to manage configuration information in the Microsoft world.

4. There is a legitimate need for other application to interrogate the configuration information, but is the registry the best solution?

5. The key concept missing in the MS registry is meta-data that can vaildate the consistency of the data across configuration files

6. A method by which other applications such as installers, configuration GUI can manipulate the data in a way that is both safe, consistent, auditable (see who has done what) and a means of reverting to known good configurations.

7. There should be a dynamic file system (a kind of registry cache) like the /proc file system that each configuration files is parsed and the entries are set in this 'virtual' registry so that other applications can query the configuration data

8. A reverse process that can take this virtual registry and create a configuration file

9 Consistency checking scripts that would be run against this virtual registry to make sure everything is makes sense.

10 The virtual registry would be a file system - no new API just file and directory operations

11. The configuration file will be the inly information that the assocated application uses, the registry is simply a preprocessed cache that orher utilities can use to interrogate the data.

IMHO Windows registry is a bad idea with a horrible implementation. It is S**T squared. It creates a new namespace that resembles a file system and text files/ binary files

Registry key = directory
value-fields = files

except we have decades worth of tools to manipulated and process files and directories, we have nothing useful to manipulate Windows Registry. Even if you were to do a hell of a lot better in terms of support tools that MS has done for its registry, it would be largely wasted effort. Hans Reiser has been working on this very problem for years - how to create a file system that would efficiently support thousands of very small files - but I an getting off topic.

The problem as I see it (perhaps a very limited view) is as follows:

1. There are too many formats for config files, that makes it unnecessarily difficult to learn configuring all the myriad of components in a Unix/Linux system.

2. There is small but significant inconsistencies between distributions on where the files are to be located and perhaps even named.

3. It is very difficult for other applications to read and utilize the information in the config files

4. It is difficult for end users to do anything with the config files.

Let me start off with item 4 - no windows administrator would let an end user within one mile of the registry. Most administrators use regedit with trepidation and will only touch only very well known locations. Nobody other than a FOOL would attempt to do any tidy up job on the registry (EVER) - unless it is a precursor to doing a fresh install of Windows.

Item 3 (It is very difficult for other applications to read and utilize the information in the config files) yes the registry in theory make is relatively easy to access data, but to actually put out useful information for the registry? Microsoft usually recommends against it - the organization of the registry can vary from release to release - there is always ANOTHER MS API to get some system critical information. Sarcasm aside - as applications evolve they seem to find ever more complex ways of representing information in the registry. So it really is not any easier (or not by much) to extract useful information from the registry than form a text configuration file. You think XF86 config is complex - look at windows some time it will make your head spin.

Item 2 should be better with the registry, but again Microsoft shows us you can even screw this one up even when you own company designs all the applications. Windows 9x and WinNT/2000 have these subtle changes to the location of certain information that always makes Windows programming ever so interesting - just in case writing the application was too simple a challenge.

Finally item 1, the registry should get rid of all those subtly different formats for the config files, but you guessed it, MS yet again show us that no matter how simple something may be in principle you can always make it more complex. Life would be not fun if everything was simple.

>About 0% of nowadays config files can be read by programs. Go to NVidia website (a big
>supporter of their products on Linux/BSD). Their products READMES tell you to edit X config
>files by yourself. It is simply too much work to do: you'll have to write configuration files compilers :-)

You have the problem completely backwards - of course configuration files can be easily parsed otherwise those configuration files would be just junk on the disk. In actual fact most configuration files are relatively easy to parse that is why they are the way they are. But you are right that most developer do not want to put the effort into parsing someones else's configuration files to only get one or two pieces of information.

On Meta-data

Meta-data is a very simple concept it is descriptive data and relationships (in a machine processable form) that are not obvious simply looking at the data. Typically it exposes relationships that are hidden deep in the code that processes the data. The greatest problem in complex interrelated systems is that simple inconsistencies in the configuration can have knock effects in other sub-system that are hard to determine. With this meta data there is no simple way of validating configurations particularly when you go across systems.

Configuration files often have complex dependencies in them, for example XFree86 and the NVidia driver implements its own GLX and incompatible with DRI, in order to use their driver you have to remove not only the open source 'nv' drive but also glx and dri. Examples like this are abound in configuration files. Some of this information becomes fold lore others are plain black magic. This is where the issue of knowledge management comes into the picture. Now I am not talking about some vast complex AI to manage configuration, but the simply information that provide the following:

1. Locations and names of config files
2. How to check for nonsistancy
3. How can you clean up data after components have been deleted
4. Ability to create 'virtual' data that is derived from several config files (using som kind of matching rules)

Knowledge and understanding the inter-dependencies between configuration files are exactly the problem and within.

Finally, I really hate criticizing people that are doing useful work but I strongly feel the the effort is futile, so please take is in the spitit in which it is given - constructive criticism.

by Andre Somers (not verified)

Although I am critical to the current form of the LR, I think your critique takes it a bit to far. Therefore, a reply.

>My basic points are as follows:

> 1. It is too hard to get every application to change is configuration storage scheme to the registry and the registry is only useful if everybody uses it.
I think it becomes usable before that, but I agree you need critical mass. That will be hard to accomplish, and in order to accomplish it you need to really offer something compelling for application developers and packagers alike.

> 2. Even if you could persuade everybody to use the registry you do not significantly simplify theproblem of managing configuration information and you loose the entire tool chain of utilities that have been developed over the years to process text files.
How is that? I think the idea of a single tree does simplify managing the systems configuration, if the system is implemented well.

> 4. There is a legitimate need for other application to interrogate the configuration information, but is the registry the best solution?
If not, what is it? The LR is a legitimate attempt at providing this functionality.

> 5, 9. The key concept missing in the MS registry is meta-data that can vaildate the consistency of the data across configuration files
I'll get back to that point later on.

> 6. A method by which other applications such as installers, configuration GUI can manipulate the data in a way that is both safe, consistent, auditable (see who has done what) and a means of reverting to known good configurations.
A means to return to a previous configuration and a good log would indeed be *very* usefull. I do think that such a thing could be far easier accomplished in a LR-like system than in the current situation, so that's progress.

> 7, 8 and 10 There should be a dynamic file system (a kind of registry cache)
Why is that? Why a filesystem? What is the advantage over a clear API targeted at this specific problem?

I am very curious to your ideas about meta data, and especially the validating you claim you can achieve with them. I really don't have an idea how you could do that, even in the example of the 3d drivers you give. How in the world can I specify how exactly the values are to be validated, without basicly re-implementing the entire settings parsing of the whole system? Consistency checking is complicated even on relatively simple database systems that house almost homogenious data. How on earth would you propose to implement such a thing for the very heterogenious data stored in configurations?! How would you know in advance what kind of conflicts may occur with other installed packages?

You summed up where Microsoft managed to screw up, but in my view, you give very little usefull hints on how to solve the issue at hand. The fact that they screwed up, does not mean that any project creating a hierarchy for configuration settings is doomed to screw up too. I think adopting such a system, if well thought out, could actually solve many occasions where users today in Linux _need_ to tinker with configuration files because it's so hard to write a good tool to can actually work with the different files.

by Nurul (not verified)

I am strongly in favor of having a mechanism for managing configuration better, but I strongly believe that it should be something that is built on top of the text based system - that should be the underlying storage for configuration. If in the mean time we can get to a stage where there are only a few limited formats for configuration file that would be a significant asset. The higher level representation would be something that is created from the current active configuration them the sys-system is loaded (usually). There are cases where we would like to get the configuration of a sub-system that is not active at the time, for example X configuration when no X server is running. We can discuss how that would would be done later.

The advantage of the scheme is that it represents the actual configuration that is running. There may be many alternative configurations for different situation that would not be in the dynamic registry. For example take the case of an Apache web server; there can be many instances of the server running on the same machine, for example on port 80 for the public site and on port 800 for an intranet. The two server would have completely different configuration files and that is easy to manage currently. With a central repository this starts to become more complex - you can imagine yet more complex scenarios. If i decide I would like to run my copy of Apache on port 8080 I would just copy the config file into my own directory and change the port to 8080 and I am done, I do not pollute a global registry. If I no longer reed the server I just delete the directory and everything is gone. IN the case of the registry if I do not do the house keeping on the global registry I now have c**p left after a while the registry will become a pile of poop.

Coming back to the multiple Apache scenario, the simple original scheme of having one place where the Apache configuration resides is now out the window. I now need a forest of configurations and some scheme to manage them. That has been my point about the registry; in the real world the encoding in the registry will become ever more complex - we see that in Windows.

>> 2. Even if you could persuade everybody to use the registry you do not significantly simplify the problem of managing configuration information and you loose the entire tool chain of utilities that have been developed over the years to process text files.
>How is that? I think the idea of a single tree does simplify managing the systems configuration, if the system is implemented well.

My basic proposal is tat we do not change the configuration files that are currently used. They are text file we have lot of tools to process test files, editors, macro generators, diff utilities, backup copies, multiple versions, and even manage version control using CVS - I have seen this dome in some large organizations. The registry tries to solve 2 problems:

1) A simple consistent method of storing configuration information
2) Since the method is uniform it allows any other application to interrogate the configuration information or even modify that information.

On point 1. the principle of a registry is very good but the Microsoft experience seems to suggest that in real life a simple hierarchy is not sufficient and you end up with a convoluted scheme of links pointing to other links that are just as hard to navigate and interpret as text files.

On point 2. I completely agree that the current configuration files are to numerous and convoluted and too frequently changed in subtle ways to make it useful for others to interrogate.

My suggested solution is as follows:

Create a registry of some sort - I have not fully thought through is the MS like registry is sufficient - I think not, but I will discuss those options a little later. This registry is created from the original config files. How would that be done in a sane manner? Clearly each sub-system development group know about their config files, it would be best if they write a script - typically written in some scripting language, for example perl or python - that would convert their config file into standard text representation that can be imported into the registry. This standard representation should be a "specification" that is created as a part of the 'config registry' project. I am in favor of having an 'minimal' XML format for this intermediate file, but a good alternative might be a LISP like s-expression form - these really are equivalent (so much of computing is rehashing old ideas in a new format and nomenclature, we seem to be very bad in computing when it comes to studying anything that was written more than a few years ago).

I am guessing that the best verifiers for configuration system would be some type of simple rule-based systems that were so popular in the area of expert-systems that became popular in the 80's . Although these systems turned out to be unsuccessful in the arena of encapsulating large scale human knowledge, but I have a strong feeling that in the domain of encoding knowledge about system interdependencies and configuration verification. A simpler scheme, but not necessarily better, would be to just have verifier script associated with each sub-system that would walk the dynamic data a check-off the consistency of the data. I definitely do not think that a single verifier would be the way to go but a get of verifiers what would be tried in turn to validate the dynamic data. For example, NVIDIA driver if it is installed will not work if the 'nv' driver is loaded or if glx or dri is loaded.

The verifier rules should be able to be written as simple rules such as this:

((Domain XFree86)
(IF (display-driver 'nvida') (
(provides 'GLX')
(provides 'twinView')
(incompatible-with (module 'dri' ) (message 'nvidia driver cannot coexist with module dri') )
(incompatble-with (display-drive 'nv') (message 'nvidia driver cannot coexist with display driver nv'))
(incompatible-with (alternative (provides GLX)) (message 'nvidia driver already provides glx, remove the glx module'))
)
}

I personally like the like pattern matching approach to verifying configuration, but a more traditional programming approach would also suffice. But the important point is that the verifier would work against the 'dynamic' config information that would be created from the text files.

>7, 8 and 10 There should be a dynamic file system (a kind of registry cache)
> Why is that? Why a filesystem? What is the advantage over a clear API targeted at this specific problem?

The file system has a many advantages over some special API (although that could also be provided), all the tools to manipulate the file system and files would be available with no extra effort - find, index, grep ...
This is a problem that has always plagued computing - namespace pollution. The whole motivation of OO programming was to reduce this namespace pollution, things that are similar (isomorphic) should have the same interface. It keeps the number of things that a person has to manage and remember much smaller and manageable. For example in windows file handles and socket handles are different, so the API get ever larger. I hope you get the point - see Plan 9 or Reiser4 for more detailed discussion on the benefits of unifying namespaces..

by Thomas Bettler (not verified)

>except we have decades worth of tools to manipulated and process files and >directories, we have nothing useful to manipulate Windows Registry.

Of course this may be true. But why? Because it's all property *without any docs or man* for deep system configuration.
I believe, Elektra comes on Linux there
*1 will be / and has to be manuals about the registry entries
(what would all those cryptic config files be without *5 manuals?)
*2 there will come easy to use tools (like for me mother)
to configure settings all about the whole system.

by Mod (not verified)

A large group(most of the pc users) dont even know what a config file is. This is part of the reason linux isnt used by people outside of the computer science field. Could you figure out how to tell a person from this group, over the phone or threw email, to change value x to some other value. Its hard eneuff to get them to find the file first. In windows if i want then to remove a regKey i make a .reg file send it to them in email and tell them to right click merge. The trick with the whole system is this. Admins who are going to use windows as there primary OS need to be able to use the registry. The users have to know nothing about it.

by David (not verified)

You missed the main good thing about the Unix configuration "system": it's decentralized. When the single file known as the Windows Registry is corrupted the whole system is affected. But when a single file under /etc is corrupted, only the specific application its for is affected. The Unix system is even more decentralized than just /etc, because on most systems you can have a /usr/X11R6/etc and /usr/local/etc as well. Oh, and don't forget the dotfiles under the users' home directories.

What missing in Unix isn't a Microsoft-style centralized authority, but merely a certain amount of consistancy and minor details. The numerous config file formats is unfortunate, but this can be solved easily without resorting to a central registry. Lots of dotfiles cluttering up the home directory is also unfortunate, but this can also be easily solved by using a single ~/.etc directory.

by Miro (not verified)

1. I like the idea.

The problem with windows registry is that there is no olicy enforced by the system. I mean an application can create a key (almost) anywhere. There is also no consistency with placing files on windows too (everything in c:\windows, even if you was dumb enough to install windows on d:\ *EVEL-GRIN*). The point is, LR must enforce a policy where an application is allowed to make changes.
My proposal:
1. each application would only be allowed to crete 1. level of directories (keeping it simple) add files to writable locations (MIME -> for default app for filetype resolution), read where allowed
2. root apps could do anything.
3. Force a layout! and force application to follow
User1.
local overrides for Apps (only?)
Apps (eg. /usr)
MIME
App1.
App2.
etc...
System (eg. /etc)
Sound
Video
etc...
etc.. (no idea if needed:)
4. Keep a lot of info about who (App@user) when created, modified, deleted a key. I dont care if LR is 100MB large if it lets me actually my system easily)

DONT believe developers will EVER follow a policy, and even if they do they do make mistakes.
So PLEASE be radical, this is a radicaly new idea on unix.

by LuckySandal (not verified)

I think most everyone can agree that the current linux configuration system is broken. There is such a wide variety of config file formats that no central configuration utility will ever be able to parse/modify them all.

Registry seems like the obvious solution. Having everything in a standard locations, in hierachial tree, with comments, modification times, permissions, symbolic links, and the ability to read / write smoothly using a common API seems like the way to go. Unlike Windows, however, we must make sure that OUR registry makes sense, and that there is a well-defined standard for key locations by which applications must abide. Given the sucess of the UNIX filesystem hierarchy, this doesn't seem like it would be hard to achieve. How often do you have a UNIX program insist on installing files in the wrong place? Compare that to Windows.

Fears over registry bloat as on Windows are unfounded, because this is already a problem with regular configuration files and can be fixed with access times.

Saying that we shouldn't have registry on Linux because it failed on Windows is like saying we shouldn't have government because communism failed. Or that shouldn't have Desktop Environments because, after all, look at Windows!

If there is ever a distribution that uses registry exclusively, or a Gentoo USE flag, I will use it! Kudos to open source for leading the UNIX world toward the future!

by jack (not verified)

I have run into too many programs that use the windows registry for a database and it just keeps growing bigger and bigger also i do not think is a good to let just any program change values then we end up with the mess that drove me to linux in the first place

Jack

by LuckySandal (not verified)

It's not geing to "let just any program change values." There will be permissions on every key, just as with files.

And even if it "keep growing bigger and bigger," how is that any different than the dot files in your home directory?

See my above post.

by jack (not verified)

I had a program that i used in my business called "Wirsbo Radiant Express"
that after i installed i noticed that the registry had doubled in size it was then i noticed that the program was using the registry for its own database this then slows down every program on the system that has to query the registry not just itself which is the case with config or dot files

one thing i don't under stand is how to manage the permissions on every key i can see it taking longer to figure if you want the program to change a certain key how to tell if that key belongs to that program or another i like the idea that i know that a certain config file belongs to which program

Jack

by Tom (not verified)

Is this program worth th cost then?

by jack (not verified)

The more i think about i believe if it is well thought out it could work but you have to take into consideration that as linux canes in popularity there will be allot more developers writing software for linux not all gnu programs like the one mentioned is writing for a limited number of users and not likely to be open sourced are there going to be enough checks to make sure they do not abuse registry.

why do you have to call it a registry i think if you called it another name more people will look at it with an open mind and not revulsion that the window registry invokes

Jack

The Linux Registry project changed his name today.

New name is ELEKTRA PROJECT.

The website is still http://registry.sf.net, but not for too long.

by Andre Somers (not verified)

First of all, I really think that in the basis, the LR is a good idea. I agree with the observation that the current situation is a mess and holds back good system integration. I also agree with the idea that we don't need a big, monolythic database like windows, but that we do need a universal key naming system. So far, so good.
The problem is, so far, almost all of the profits of the proposed system are in the field of the system integration, something a distribution takes care of most of the time. I hardly see any profits for the application developers themselves, only a lot of work in porting loads of programs to use the LR. That is a major drawback.
Another point - that has been mentioned before in the above reactions - is that configurations need to be layered. The very existence of Kiosk is testament to that. You say you don't want to solve that problem, and that the LR is a low level system on top of which this could be implemented. I think that is the wrong choice, for two reasons. The first is basicly the same as above: what is the profit for the application programmer if he needs to do the hard work himself anyway? Another problem is more fundamental to the design: it is a potential trap for the whole system you propose. The beauty of the LR was a single, consistent way of storing and retreiving settings. However, by ignoring layering of settings, you immediatly end up with the settings for the same thing on different places in the LR and probably outside of it as well (LDAP, SQL, etc.).

You choose not to do all this, because you want to keep the system simple so it a) does not form a single point of failure, and b) is available for very low level services. I think these problems are solvable, even if you offer more than just a basic service.
You could think of the configuration system in much the same way as the filesystem. That works roughly like this: Initially, only the boot partition is available, wich is in a format the kernel understands. Later in the process, more filesystems may be mounted, some needing higher level services that were not available at boot time (NFS for example). Right?
Why not work in much the same way for the LR? Have a basic configuration system that is availble without any external dependencies. As the system is booted, more parts of the configuration system could be 'mounted', possibly with other and more complex backends. Each mounted configuration resource could form a layer, that augments and/or overrides the layers below it.

Such a system, although I admit more complex, would offer significant advantages to application programmers, as it frees them from making configuration in layers possible themselves. They only need to access a single key for each setting, and in 99% of the cases don't care where that key was set. At the same time, it frees you from the problem that application programmers or distributions may organize their settings layering in different, incompatible ways, nullifying the advantages of the LR in the process. Next, it would make it possible to integrate some important sets of settings into the system without changing all programs at the same time. I think this will make the transition easier.

I really think the LR will have a much higher chance of being accepted if it offers real advantages for all parties involved, if it really offers something more than is available now.

by Andre Somers (not verified)

Another small point. This comes from the LR homepage:

==
There are very good reasons why types like Integer, Time, Font, List, etc were not implemented: The Registry was designed to be usefull for any type of program, so having more specific data types implicates in the definition of value limits, separators in the storage format, etc, that may be good for some application and bad for other. So the semantics of the data is handled by the application. A program or framework may define its own special data handling methods using these essential basic types. See the keyGetType() and keySetType() methods documentation in the registry(3) man page to understand how to set keys with your own data types.
==

I'm not quite sure I totally agree here. I think it would be *very* usefull to at least be able to store some basic keytypes in a consistent way. UTF8 for strings is a Good Thing (TM). If you can decide on an encoding for the basic string type, why not on an encoding for a small set of other basic types? I think that would only make it easier to get a consistent registry, and would promote integration. Applications that want to use other types can still define them, but an option to use at least basic types like integer, boolean, date/time and perhaps a _few_ others would be very welcome I think. In my experience, these are the types that are most often used in configuration files. Maybe it would even make sence to be able to define a list of values with the same type for a single key.

by tomten (not verified)

I was a fulltime Linux (work/home) user for 3 years until 4 years ago(Suse 5-7/Debian). When Windows 2000/XP came along the whole blue-screen mess was rectified and I now use Windows fulltime, but I regularly install Linux to follow it's progress. At the *system-level* other than printing (CUPS) and dists like Knoppix I must say I'm not impressed. Time has stood still in UNIX-land as the competition at the same time is miles away from there 90s.
One thing that stands out is Linux distros flakiness when it comes to updating configuration files. I still find myself often have to drop to the terminal to fix things.
Just look at Suse, they probably still have a horrid Perl script updating all configuration files behind that flashy GUI after a small software install.
My conclusion is that the (nonexistent) configuration system on Linux takes so much time to keep in shape that they newer have time to do serious quality control.

Unix needs more projects like CUPS. Recognizing that LPR was shit they forged ahead ignoring the pro LPR-dinosaurs and designed a killer system that proved them wrong.
I think Linux registry has the same potential but at a much grander scale and I wish it the best of luck.

by Avi Alkalay (not verified)

Thank you very much for your practical, real-world vision :)

by Nurul (not verified)

How about handling different configurations files (under different circumstances), or comparing changes to the configuration files. I could go on - the problem is not the registry per se but the fact that we through out all those test tools in Unix/Linux that have been developed over the years to handle all these thing and much more.

The registry is conceptually good but let us not try to emulate the Windows crap. You talk about 99% evangelizing - perhaps some of that energy cold be put to better use if we could get more discussion about the issue. I am glad that your team has started us on the process - it is very important but let us go to a place that we can be really proud of instead of trying to mostly emulate an idea that failed to deliver on the Windows platform.

by Avi Alkalay (not verified)

I see your point. But lets try to see it from a business (and not technicall) perspective: MS wanted to deliver a fully and completelly integratable system (programs make the configurations for you, preciselly). They found their way putting all together on a configuration tree that can be accessible by absolutelly any program.

I admit this is not the only success factor for MS, but when I think about the power it provides, I'm pretty sure that this is a key point in Windows success. And I admit Windows is an extremelly successfull platform.

How they did it, the technologies they used, the way it was implemented, etc... these are just operational details. From a successfull business perspective, it does not matter. And if its essence is adopted, good minds will join the effort and will help improving it.

From a technical perspective, yes, we'd like to do a better job then the Windows Registry, but we also learned that too much complexity will make it unaccessible for simple minds, and maybe optimized for only one purpose (which deoptimizes it for other purporses). Oh yes... and also too hard to implement, more likelly to have dependencies, more bugs, etc.

There is a wonderfull document by Doc Searls (one of the Linux Journal colunists), very inspiring, that explains why the Internet is simple because it is a success, and is a success because it is simple. If you have time, I really suggest reading:

http://worldofends.com/
http://worldofends.com/#BM3
http://worldofends.com/#BM4

The entire document is about 15 minutes reading that will change the reader's life. (ok, not that much...)

by Nurul (not verified)

Look don't get me wrong - I thing it is wonderful that you guys have put the effort in solving a significant issue in Unix/Linux. I have been very critical because I strongly believe that the Windows registry has not solved the configuration problem for Windows - it has actually made it a more brittle system. If you have been following advice from windows GURUs they recommend completely reinstalling windows once a year. I agree this is not only the fault of the register, but it is a large part of the problem.

My suggested approach is a hybrid one - keep the current config file scheme. Encouraging developers to use a much small number of differing formats would be a great help, but not necessary. The hybrid approach is that we have the concept of a 'running' config repository that may well be your registry. This 'running' registry is updated when a sub-system is started. The update would typically be done with a script the would be provided (hopefully) with the application. After the running registry has been updated, we also could have a series of scripts for each subsystem that would do some consistency check on the configuration data, this could be a very complete check or even a superficial sanity check.

But the very best aspect of this dynamic config registry is that it would allow other applications to check to the actual configuration information in use and to verify that the necessary precondition that the application requires has been met.

This would not allow the configuration to be changed programatically, but that issue can be tackled later. An arrangement that i favor is to have a script (again provided by the sub-system developers - hopefully) that would create a config from the dynamic data.

When all is said and done, text representation has been the most successful invention of computing. Over time we see reverting back to text representation that were once binary - that really does say something.

AGAIN it is great that you and your group of developers have made this first step but I feel you have not gone far enough.

by superstoned (not verified)

most negative comments here are just because of the name "registry". if the name was different (or the ppl ranting had just *read* what the linux registry is all about) they whouldnt complain.

please read before ranting. because its more like trolling, if you dont understand what you are talking about.

ppl say "I want to be able to read my config files myself" well thats possible with the LINUX registry (no, not in windows, but are we talking about windows or linux registry?)
ppl say "I dont want ONE huge database that can easilly get corrupted" well no problem, the linux registry IS NOT a database, but a bunch of seperate files

etc etc etc so shut up if you dont know what you are talking about, but first see what it *really* is... (more like a common standard for config files so a central program, the linux registry, can access them easilly, notify about changes etc etc and let other programs change things too without difficult parsing etc etc)

(sorry its very irritating to read through all these comments, and see most are just plain stupid)

by Christian Loose (not verified)

Well said! Exactly my thought when I would through those posts.

Christian

by superstoned (not verified)

yeah, poor Avi replied to (almost) all of them, he's just too nice - I whould've said "RTFM" instead...

by Avi Alkalay (not verified)

:-D

by Zooplah (not verified)

Well, I had similar ill feelings when I read about it. But I read the article, and it sounds like it avoids my two problems with Windows registry.

Now, I do think that it's a better implementation than Microsoft's (though the two aren't directly comparable). However, I don't see the big problem with the current configuration (/etc/prog.conf, ~/.progrc ~/.prog/*), but I guess an easy way to access the settings programatically without each program making it's own parser is nice. Let's just hope that it's:
1. Portable (Linux, BSD, Mac, Windows)
2. Accessible, with any programming or shell-scripting language
3. Easier to programmatically access than Windows registry (the norm for Linux is easier API)
4. Includes scripts to carry over your old configuration (apache2reg, for example).

by Graham Daniell (not verified)

The registry is the Achilles Heel of Windows. It is the worst Windows "feature" of them all. It is what makes Windows so hard to deal with and so impossible for ordinary users to understand.

We need a Linux registry like we need a hole in the head.

Graham Daniell

by Andre Somers (not verified)

We don't need the bad implementation used in Windows, but we do need the integration. If you can come up with a better idea, please go ahead. If not, please stop trolling.

by superstoned (not verified)

READ!!!! damn, can't you read? just in the post above yours I said there are so many stupid comments here because no-one actually reads more than the name: linux REGISTRY.

but its not like windows registry, exept for the fact that it tries to solve the same problem. it has none of the problems the windows version has, just READ the damnit article.

jeeez

by Thomas Bettler (not verified)

Apps using Elektra should of course provide own manuals about available settings in the kdb (as 5* manuals do for the config files) as metadata. The manual could also be translated into other languages.

Woudn't it be helpful to specify a machine readable standard for such manuals (so any higher level UI to kdb can provide this metadata. (i.e. like Kconfig does for the linux kernel). Or is this behind the scope of your project and therefore already a higher level project itself?