Phil Thompson Talks About PyQt

High level languages are increasingly being used in preference to C and C++ in new desktop software. One of these languages best supported in KDE and Qt is Python. To find out about the history and current state of PyQt, KDE Dot News talked to Phil Thompson, author and maintainer of the bindings.

The HP Linux Printing system uses PyQt for its tools

Please introduce yourself and your role in Free Software

Through my company, Riverbank Computing, I develop and maintain a set of related packages. These are SIP, PyQt and QScintilla.

SIP is a Python bindings generator for C and C++ libraries. It started out as a small SWIG (hence the name) and was first released in 1998. At the time I felt that SWIG, as a more general purpose tool, wasn't great at producing Python bindings for C++. So SIP was intended to do a more specialised job much better.

PyQt are the Python bindings for Qt. PyQt v3 supports Qt v1 to v3. PyQt v4 supports Qt v4. The first release was also in 1998, although named PyKDE since originally the two were not split. PyQt is written using SIP. PyQt follows Trolltech's licensing model in that there are GPL and commercial versions.

QScintilla is a port to Qt of the Scintilla code editor widget. It supports Qt v3 and v4 and (of course) includes a set of Python bindings. I initially did the port so that the Eric IDE (written by Detlev Offenbach) would have a decent programmer's editor. Again, it is available under the GPL and a commercial license.

What is PyQt and why should I use it?

PyQt gives the Python programmer access to the full Qt API. You can do almost all the things you can do from Python that you can do from C++.

As the API is the same from C++ as from Python then "why should I use it" comes down to the choice of programming language and isn't specific to PyQt. For me, the advantage of Python over C++, as an application development language, is simply programmer productivity. You can see this with the ports to PyQt of the standard Qt examples. They have the same function and use the same API but the Python versions have 50-60% of the lines of code, and are much easier to read.

Related to productivity is how easy Python is to learn, but still has enough power to satisfy more experienced programmers. Trolltech are discovering this themselves as PyQt allows them to sell Qt into highly technical organisations where the users are industry specialists (chemists, aeronautical engineers) rather than experienced C++ programmers.

And of course PyQt is mature, stable, and has a large user base. The two most common pieces of feedback I get from users is "it just works" and "it's fun".

Why did you start the PyQt bindings?

The usual reason - I had an itch to scratch. I'd been using Tcl and Tk for a few years for developing personal productivity tools and got frustrated with how visually ugly Tk applications were becoming (because everything else was getting prettier). I'd started using KDE v1 and switched to Python because there were some initial KDE bindings built using SWIG. I soon realised that I could do better than that and started again.

The v0.1 was released on November 1st 1998 and used Qt v1.41 and KDE v1.0. Releases have been made about every 3 months ever since.

How are the bindings implemented?

SIP takes a set of specification (.sip) files describing the API and generates the required C++ code. This is then compiled to produce the Python extension modules. A .sip file is basically the class header file with some things removed (because SIP doesn't include a full C++ parser) and some things added (because C++ doesn't always provide enough information about how the API works).

One of the keys to PyQt's success is the attention to detail in the .sip files. I do not believe you can automate the generation of bindings and end up with something that is industrial strength. You can use automation to do 95% of the work, but you must still go through the API adding extra information needed by a particular target language. With Python, for example, it's important that a Python object knows whether or not it should call the corresponding C++ instance's destructor when it is itself garbage collected. Expecting the programmer to manage this themselves is a recipe for core dumps.

In PyQt v3 the .sip files are hand-rolled - they have evolved incrementally from the originals created for Qt v1.41. SIP contains a versioning system and so the v3 .sip files contains a complete history of the Qt API from v1.41 to v3.3.6. Although I don't test it, PyQt v3 should still build against Qt v1.

For PyQt v4 I use an internal tool (written using PyQt of course) called metasip. This is sort of an IDE for SIP. It uses GCC-XML to parse the latest header files and saves the relevant data, as XML, in a metasip project. metasip then does the equivalent of a diff against the previous version of the API and flags up any changes that need to be looked at. Those changes are then made through the GUI and ticked off the TODO list. Generating the .sip files is just a button click. In my subversion repository, PyQt v4 is basically just a 20M XML file. Updating PyQt v4 for a minor release of Qt v4 is about half an hours work.

In terms of how the generated code works then I don't think it's very different from how any other bindings generator works. Python has a very good C API for writing extension modules - it's one of the reasons why so many 3rd party tools have Python bindings. For every C++ class, the SIP generated code creates a corresponding Python class implemented in C.

The Python class implements methods corresponding to the C++ equivalents. As Python doesn't support function overloading the correct signature is determined at run time by checking the types of the arguments passed in.

If a C++ class contains virtual functions then SIP generates a C++ sub-class with code that re-implements those virtuals and checks if there is a corresponding Python re-implementation. If so it is called, otherwise the base implementation is called. The result of the check is cached so it's not a significant overhead. Access to protected functions is also implemented using this generated sub-class.

How does this compare to bindings implemented with SMOKE?

I don't know. I've never used SMOKE or used any bindings based on it.

How complete are the PyQt 3 bindings?

They have been complete for a number of years. New releases of PyQt v3 tend to contain small updates driven by new releases of Qt and infrastructure changes driven by new releases of SIP.

I concluded very early on that I wasn't in a position to decide what parts of the Qt API should be exposed to the Python programmer, so I exposed all of it.

What is the state of bindings for Qt 4?

Complete and stable. PyQt v4.0 was released June 10th after 6 months of development snapshots. PyQt v4.0.1 was released July 15th and is just a minor maintenance release - not a single bug has been reported in the bindings themselves.

What is your opinion of the PyKDE bindings?

Speaking as the original author, I think they are wonderful. Jim Bublitz, the current maintainer, has a much more difficult job than I do with PyQt. KDE is much bigger than Qt, the API is less consistent, distros seem more inclined to mess around with KDE than with Qt, and he has to deal with me improving SIP for me while breaking it for him. Jim is also a much nicer person than I am.

I suspect that he could do with more help than he currently receives when it comes to PyKDE v4. Perhaps those distros that are making more use of PyKDE could step up to the plate (cough, cough).

What contact have you had with Trolltech about your work?

We have had an ongoing, informal relationship for 5 or 6 years. They were kind enough to invite me over to Oslo a few years ago. It's been interesting to observe their development. Originally they were a very technically focused (they still are of course), but without a strong sales/marketing effort. As a company they didn't "get" PyQt because they didn't understand why anybody would want to use anything other than C++ - I suspect that the majority of their programmers still feel that way.

These days however the sales/marketing influence is much greater. They know that people want to use other languages because that's what their customers are telling them. They know that there are large corporates who are buying Qt only because they want to use PyQt.

They are also putting effort into developing the Qt ecosystem: partnership programs with companies like Riverbank that develop complementary technologies.

How is this work funded?

PyQt makes a profit - it is self funding. It's not just sales of licenses, it is also the spin-off work that sometimes arises from a sale. For example I'm spending the rest of the year working with a customer on making greater use of PyQt within their organisation.

How many commercial users do you have?

Over 200 at the moment.

Can you name some exciting specific users of PyQt?

One of the disadvantages of being a one-man-band selling software over the web is that you don't get to know your customers very well. One thing I'm pleased about is that I'm a great film fan and Disney, Dreamworks, Pixar, Industrial Light and Magic, and Sony Pictures are all commercial PyQt users.

How many people have worked to create PyQt?

I'm the only person who works on SIP and the core bindings - apart from patches from others from time to time. For PyQt v4 Torsten Marek wrote pyuic4. Ulli Berning is a huge help in making sure PyQt builds and runs on HP-UX, AIX and Solaris.

Of course a successful project is about much more than people who write code, it's about the people who contribute to the community: Detlev and Jim I've already mentioned, Boudewijn for his original book, the group who ported the PyQt v4 examples, GerardV for PyQwt, DavidB and AndreasP for their quality contributions to the mailing list.

What else does Riverbank Computing offer?

The other area of expertise is embedded Linux systems - porting, device drivers etc. My first job after graduating was as a hardware engineer and I've always liked projects where you can kick the product.

Would you like to see more programmers use higher level languages?

Yes - and I think it is inevitable. In my youth there were debates about whether you should do your CP/M programming in Z80 assembly language or C. C programs took up too many resources (so I was told), but all my C programs had more functionality, fewer bugs and were quicker to write than Z80 programs. I see any debate between the use of C++ and languages like Python (and Ruby etc.) as being exactly the same. At the moment C++ is fine for writing frameworks, but I would prefer not to use if for writing applications.

Are you going to aKademy this year?

No, too busy I'm afraid.