Interview with Valgrind Author Julian Seward

Valgrind is a CPU simulator which is used by KDE and many other projects to profile and debug our programmes. In the interview below Valgrind author Julian Seward talks to KDE Dot News about why he developed Valgrind, how you can use it and, most importantly, where the name comes from. Julian will be giving a talk on Valgrind next Sunday at FOSDEM.

Julian Seward, Valgrind lead

Please introduce yourself and your Free Software work

I'm the founder and project lead for Valgrind. My background is in compiler technology, having been fascinated by them for a good couple of decades. I've also been interested in issues of software correctness for a long time. Eventually I combined these interests into creating Valgrind, a simulation-based tool which you can use to debug and profile your programs.

Since the original 1.0 release, Valgrind has received a great deal of development effort from many talented people. As a result it is now far more capable, robust and portable than I ever expected.

I'm also the author of bzip2, a data compression program.

As a compiler writer I worked for three years on the Glasgow Haskell Compiler, an open source implementation of the functional language Haskell, working mostly on the back-end code generation, register allocation, and bytecode interpretation. Haskell is a really remarkable programming language. Although I don't currently use Haskell much, the functional-programming style of thinking about problems is something I continue to find valuable, particularly in designing some aspects of Valgrind.

What can a typical programmer do with Valgrind?

Save a lot of time and hassle when debugging and profiling their code, and thereby deliver more stable and faster code to end-users.

Think of running your program on Valgrind as if you were running it on a special, magical computer, which can collect all manner of useful information that no real machine can get for you.

Valgrind now provides a suite of different tools built around a common core. The most important tools are Memcheck, Cachegrind, Massif and Callgrind.

Memcheck is a tool for finding memory management bugs in C, C++ and Fortran programs. Using Memcheck you can detect the following errors, which are hard to track down or even detect by other means:

• reading and writing freed memory
• reading and writing off the start/end of malloc'd blocks
• use of uninitialised values
• passing garbage values to system calls
• leaked memory

The Cachegrind profiler provides detailed low-level profiling. It counts instructions, memory references and most importantly cache misses, on a per-instruction, per-source-line, per-function and per-program basis. As a result of doing that, you can find places where your code is losing time by doing too many cache misses or instructions. Most other profilers cannot find that information for you. In the past few months we have been able to use cachegrind on Valgrind itself, resulting in some cases in a 25-30% speedup by removing cache misses found in places we never imagined they would be.

Massif is a space profiler. It monitors your program's malloc/free (and new, delete, etc) calls to collect information on who allocated what, where, when. It will then draw pretty pictures such as in the manual and provide extensive information to help answer questions about storage use in your code. You can use that information to reduce space use, particularly for allocation intensive C++ code.

Callgrind is a sophisticated profiler by Josef Weidendorfer. It can do gprof-style attribution of costs between callers and callees. It can measure all manner of costs, including function entry/exit counts, as well as the lower level instruction / memory access / cache miss counts that Cachegrind provides. Callgrind will even measure estimated CPU cycles, if you ask nicely. Callgrind has an associated KDE GUI, KCacheGrind, which helps navigate the vast mountains of performance data that Callgrind quickly accumulates.

There are other tools which have come and gone. In particular we have Helgrind, a tool for detecting data races in threaded programs (places where there should be a lock protecting global data, but there is none). For technical reasons Helgrind stopped working about a year ago, but recent work on Valgrind's infrastructure means it should be possible to bring it back to life, which is something that we would like to see, and is requested by many users. Unfortunately no Valgrind developers currently have the time/energy to do so. We would be interested in hearing from folks interested in pushing Helgrind along. You'll need a solid grasp of threading and assembly-level programming, and some understanding of compiler internals.

How does Valgrind work?

It's essentially a simulator for user-space code, designed for flexible instrumentation and data collection. The program is run on a synthetic CPU implemented in software. Your program's instruction stream is unpicked into a simple representation. Instrumentation (debugging or profiling code) is added, and the result is converted back into machine code. All this is done on demand, on the fly, whilst your program is running. (That's why Valgrind is so slow starting programs -- it's busy instrumenting them).

There's also a lot of nasty details behind-the-scenes, all aimed at giving your program the illusion it is running "normally". Most of this nastiness is to do with system calls, signals and threads. It works well enough that you can run most programs unmodified on Valgrind with no special preparation.

Nowadays Valgrind runs on {x86,amd64,ppc32,ppc64}-linux, which means you probably have very little excuse not to use it!

How can a coder start using Valgrind?

You can run any program directly on valgrind, just do "valgrind --tool=<toolname> my_prog --args --for --my --prog". No preparation needed.

For best results, you should recompile with debugging info (-g). If using Memcheck, restrict compiler optimisation if you can (-O1, or ideally -O0 if you have the patience). If profiling, you should optimise as usual, but also use -g.

There are many resources, including on-line documentation on our website. There are also mailing lists for both users and developers.

Valgrind is a really powerful tool suite. Although you can start off easily, like any sophisticated tool, it takes a while to understand what the debugging/profiling tools are telling you, and how to best use them. Expect to spend a couple of evenings playing with Valgrind before you are a Power User :-)

KDevelop with its Valgrind plugin

How well does Valgrind integrate with KDevelop?

I don't know because I've never used KDevelop. I believe KDevelop has Valgrind support, but I don't know any more than that.

Are many projects using Valgrind?

From the feedback we get, it appears that many big-name open source projects use it. We have a page listing literally hundreds of users, and that's just the ones that are happy to be publically listed. A short except:

All the projects on this page use or have used Valgrind, e.g. Firefox, OpenOffice, StarOffice, AbiWord, Opera, KDE, GNOME, Qt, libstdc++, MySQL, PostgreSQL, Perl, Python, PHP, Samba, RenderMan, Nasa Mars Lander software, SAS, The GIMP, Ogg Vorbis, Unreal Tournament, Medal of Honour, RenderMan ...

Do you have examples of cases where Valgrind has made a big difference to a program?

We get a lot of informal feedback indicating that Valgrind is useful. We have done two surveys of the user base, generating more than 300 responses. These also indicate Valgrind is useful and have provided us with much insight into the needs and concerns of our users. See the surveys page for details, including summarised conclusions from the survey.

Another indication we have is a study by Jens-Heiner Rechtien. He ran some tests on OpenOffice.org (pre 2.0) and wrote a short account of the bugs Memcheck found, at. It makes interesting reading. Memcheck found 102 problems in 26 source files. Rechtien comments:

A rough evaluation of the consequences of the detected problems showed that about one third would never show up as a program failure for the user, another third are bugs which have no consequences yet, but might lead to regression later if code is changed and the last third are plain bugs which might crash the application or lead to malfunction anytime.

What is the relationship of Valgrind to KDE?

The KDE folks were Valgrind's first big users ("early adopters"), and a lot of valuable early feedback and development effort came from KDE-land. KDE 3.0 was the first KDE release to be debugged using Valgrind.

There's no specific connection nowadays. For a long time kde.org gave us a web-home (www, cvs), which was great. About a year ago we migrated to our own valgrind.org domain, but we still use the bugzilla at kde.org.

I believe many KDE developers use Valgrind, but we have no direct way to find out for sure.

I've used KDE practically every day since 1.0beta4, which I remember compiling up years ago on a sparc-solaris box. I'm a KDE fan :-)

How did you choose the name Valgrind?

Shaggy dog story (and FAQ!).

The name comes from Nordic mythology. Originally (before release) the project was named Heimdall, after the watchman of the Nordic gods. He could "see a hundred miles by day or night, hear the grass growing, see the wool growing on a sheep's back" (etc). This would have been a great name, but it was already taken by a security package, "Heimdal".

So instead we named a recently-acquired cat Heimdall, and kept searching. For a while the project had no name at all.

Eventually we settled on Valgrind, which is another Nordic name. "Valgrind is the name of the main entrance to Valhalla (the Hall of the Chosen Slain in Asgard). Over this entrance there resides a wolf and over it there is the head of a boar and on it perches a huge eagle, whose eyes can see to the far regions of the nine worlds. Only those judged worthy by the guardians are allowed to pass through Valgrind. All others are refused entrance."

It's not short for "value grinder", although that's not a bad guess.

Useless fact: most people pronounce it "val-grind". I pronounce it "val-grinned", for no particular reason. Maybe someone familiar with Nordic languages and mythology can tell us how the word should be spoken?