An open question I had while working on Sprinter was how much cost was associated with creating and then adding a QRunnable (aka "a job") to a QThreadPool. If the cost is significant, then the makes sense to try and prevent more jobs from even being created by checking if the job should be created in the first place. This work would normally be done in the job itself as part of its first steps, so this can also be seen as moving work out of the job thread and into the thread that launches the job in order to prevent more jobs.
I dislike optimizing without measuring so I whipped up a quick test of QRunnable and QThreadPool that created and ran increasing batches of jobs. I figured others using QRunnable might be interested in these numbers as well so here is the result of a typical run on my Thnkpad X201 (~3 years old):
============== Batch size: 1000
Run 1 : 15 milliS total @ 15 microS
Run 2 : 10 milliS total @ 10 microS
Run 3 : 10 milliS total @ 10 microS
Run 4 : 6 milliS total @ 6 microS
============== Batch size: 10000
Run 1 : 111 milliS total @ 11.1 microS
Run 2 : 111 milliS total @ 11.1 microS
Run 3 : 111 milliS total @ 11.1 microS
Run 4 : 110 milliS total @ 11 microS
============== Batch size: 100000
Run 1 : 1137 milliS total @ 11.37 microS
Run 2 : 1094 milliS total @ 10.94 microS
Run 3 : 1031 milliS total @ 10.31 microS
Run 4 : 1072 milliS total @ 10.72 microS
============== Batch size: 1000000
Run 1 : 11108 milliS total @ 11.108 microS
Run 2 : 10784 milliS total @ 10.784 microS
Run 3 : 11030 milliS total @ 11.03 microS
Run 4 : 11102 milliS total @ 11.102 microSThat is wonderfully linear performance as the batch size grows, which is exactly what we'd hope for. Each job takes ~10 microseconds to be created, run in a thread from the pool and then deleted. This means that preventing job creation is going to give us a savings of ~10 microseconds per job. Put another way, the cost is ~1 second per 100,000 jobs on my laptop. Given that the overhead scales linearly at least to one million jobs, this kind of overhead simply gets lost in the noise and isn't worth being concerned about.
I tried a few variations how the runs were handled, none of which made any significant difference. For instance, moving the deref of the QAtomicInt counter from the run() method to the destructor of the QRunnable subclass had no real effect, as one would expect. Excluding construction and destruction of the QRunnable objects did not significantly change the per-job numbers, though if you launch a swarm of a million jobs in sequence as in the test one might consider creating a pool of job objects to re-use; that's a pretty specific use case, however. I also tried code that is slightly more analogous to what Sprinter actually does, namely: it creates the jobs and puts them in a QVector for bookkeeping purposes before running them. QVector's performance is also such that this had no real impact at all on the numbers. Even looping once to insert into and again to run the jobs from the QVector wasn't significant.
In Sprinter checking whether or not a job should be created is currently fast (~0.5micros). This is about to gain some additional complexity for more aggressive pre-match screening of runners. This will slow that process down, though I don't yet know what the exact overhead will be at this point. The code is not yet written, after all. I expect a few microseconds at least, but doubt it will be significant. Even if kept outside the jobs, this code would run in the main worker thread and not block the UI. Still, given the low cost of a QRunnable, I will be moving this check into the QRunnable job as this simplifies the code in question and allows Sprinter to parallelize these checks.
Racing close to Kate/KTextEditor on the way to Qt5/KF5 fields , the port of Okteta to Qt5/KF5 now crossed the lane and moved out of “KDE4″ terrain as well, by no longer depending on kde4support.
Of course also with zero compiler warnings during build (with build.kde.org settings) and all existing tests succeeding (to follow the environment-friendly standards set by Kate’s developers ), as can be now also nicely seen on Okteta’s framework builds on build.kde.org (Thanks to the ever-awesome Ben for settings things up).
As you might remember, Alexander started the port last summer and has ever since been in the feedback cycle of KF5 and Okteta as well as other KF5 porting work (Note that the Qt5/KF5 port of Okteta has meanwhile moved to a branch named
frameworks, for consistency with the other repos).
Shifting priorities I finally managed to join his efforts (jumping on the moving train, eh), just right in time to “steal” him the milestone commit “Remove KDE4Support, no longer needed”, pardon Alex , at least crediting you here.
So what is up next? Of course giving lots of feedback to the KDE Frameworks 5, as there is quite some work left to do. From simple things like incomplete CamelCase forwarding headers to platform integration issues, e.g. the filedialog ignoring the QFileDialog::FileMode set and also blocking user input after cancelling or QFontDatabase::systemFont(QFontDatabase::FixedFont) not delivering a font with fixed width.
And then thinking about adding the Okteta widgets libraries to KDE Frameworks 5. The KHexEdit Interfaces were not integrated into KF5, because the original reason for having them instead of proper libs in kdelibs was to not bloat kdelibs for just a few possible users of a hex edit widget. That reason is gone with KF5. Just, I am not yet sure about the API, and KF5 would require a stable one. So in case you are interested in Qt5/KF5 hex edit widgets, ping me and tell me your requirements (find my email address in Okteta’s file headers or the About dialog).
More, my playground project Kasten finally needs to get pushed forward and possibly soon into a repo of its own. More core/UI splitting, QML variants etc are now to do, the core/widget splitting in KF5 help here a lot. Then proper async behaviour and more. Long TODO list there.
And from Qt5/KF5 terrain besides the obvious Plasma Workspaces also Sailfish OS, even Ubuntu Touch can be seen on the horizon, so program variants for those might be interesting. And whatever other platform can be reached from there. (Yes, you just do not yet know you might need a hex editor there )
So much for the current dreams
The roadmap draft sees as first Qt5/KF5-based release of Okteta the version 0.14 (released “when it is ready”).
Means, there will be another, final Qt4/kdelibs4-based version of Okteta before, 0.13, to be released as part of KDE Apps 4.13 this summer, possibly having 1-2 small features added.
Lots to do. But by using KDevelop everything seems easy possible
New rekonq release (2.4.2) ready just 1 day after another (2.4.1). Ehi, don’t get used to this
The motivation for this fast update is an idea came out from a bugzilla discussion for an old request: the clear urlbar button.
You may know rekonq uses urlbar right corner to notify things like “bookmarks”, “rss”, “ads blocked” and so on. So basically things related to the site you are actually visiting. This completely excludes KLineEdit clear urlbar button implementation. And, in my opinion, excludes also the logic for such a button there.
The idea has been to add it as a normal action, excluded by default from the main toolbar, but easy to add just right clicking there and choosing “configure main toolbar”. So, the usual way.
But… what happened in the road from 2.4.0 to 2.4.1??
There is another important report from the bug system we implemented this time: global zoom settings. You can choose a global setting for the default zoom in rekonq and just see all pages with it. Moreover, the action “default zoom” (and the shortcut CTRL + 0) restores to it (instead of 100%), letting easy testing new zoom. This job helped also fixing a strange bug reported about rekonq losing some time (non default) zoom settings for a site.
And… is there more? Well.. yes! The usual periodic bugfixing and code improvements: This time I worked around urlbar suggestions, to have them faster and more fluid. Some small improvements also around favicons. A big documentation update from the doc team, a patch to fix compilation on window and a couple of minor non crashing bugs fixed.
Last thing, a change in the default tab width and in the resizing mechanism, discussed with a nice guy reporting his ideas. That’s it! Rekonq 2.4.2 for the masses.
PS: In the while, I have now a working fresh KDE frameworks installation here. Rekonq moving starts this afternoon…
Filed under: international, rekonq
This is a follow-up to Aaron’s blog Frameworks 5: Timing is everything, put into the perspective of Kate and the KTextEditor interfaces. In short, Aaron suggests application developers the following:
When Frameworks 5 has a firm release schedule, then it makes sense to start coordinating application releases targets with that and syncing up development cycles.
I agree with this entirely, provided it’s really about applications. In the context of Kate, this is not the case, since the term ‘Kate’ usually also refers to the ‘KTextEditor’ interfaces along with its implementation ‘Kate Part’.
In essence, the KTextEditor interfaces together with Kate Part provide a text editor in a library that can be used by applications, or more to the point, KTextEditor+Kate Part build a framework.
In fact, about a week ago, the KTextEditor interfaces and Kate Part were split out of the kate.git repository. So from now on, kate.git contains the Kate and KWrite applications. The ktexteditor.git module contains the KTextEditor interfaces and Kate Part.
ktexteditor.git is is a tier3 framework and meanwhile officially part of the frameworks initiative: if you want so, KDE now provides 58 frameworks instead of the previously announced 57!Why timing is everything
Now what about timing? Started on November 28th 2013, Christoph pushed the first compiling version of Kate into the frameworks branch. This was very good timing, since the split of kdelibs into all the frameworks was more or less just finished at that time. Or to put it the other way round: Christoph started porting Kate pretty much as early as the KF5 initialive allowed us – cool!
Around that time by coincidence, Michal started to work on Kate in the master branch and committed lots of cool features. However, since Kate was already rudimentary ported to Qt5 and KF5, the code base diverged more and more, so merging got more complicated. Therefore, Michal started to work in Kate’s frameworks branch. And within only a week (!), Kate, KWrite, the KTextEditor interfaces and Kate Part build without a single warning, meaning that the code was completely free of Qt4 and kde4support – wow!
Again, a month before would have been too early – so this was perfect timing. The result was a pretty stable and mostly bug-free Kate in mid of December.
Well, not completely free of porting bugs. For instance, file saving did not work correctly, ending up in data corruption. It turned out that this was an issue in the framework karchive, which David Faure was able to reproduce and fix. This is good timing in the sense that without Kate’s usage of karchive, the bug would have probably ended up in the technical preview TP1. But it didn’t – yay!
Last but not least, the Kate and KDevelop developers planned already months ago to have a joint Kate/KDevelop sprint from 18th to 26th of January, 2014. Given the current state of Kate and KTextEditor, this again is perfect timing, since most of the KTextEditor cleanup are already done. So the Kate developers can focus on the needs of e.g. KDevelop, on fine-tuning and implementing interfaces, etc. So in about two weeks, when our developer sprint will have ended, Kate will most probably shine as it never did before. And this is the case already now, even before a firm release date of Plasma 2 exists.
This is great news for Kate and all applications that use the KTextEditor interfaces, as Kate/KTextEditor already now reached a maturity that they never had before.
And this is so awesome! I cannot think of better timing Thanks to all who made this happen!
Hm... I've been weeding out my linguistics library to donate everything that has to do with Indo-European, Greek or Celtic to Menna, and everything that has to do with the history of the English to Rebecca. There's still about a forty feet of bookshelves left.
And then I found my old Classical Chinese readers and Pulleyblank's. Now I've forgotten most of the characters, and I learned those by writing them down on index cards and manually shuffling them (I threw those out ten years ago). Back then, you'd need special ISA board plugged into your PC to show Chinese characters on the monitor...
But in this day and age, something better should be available. So... Is there a open source, preferably KDE, application that assists learning hanzi, preferably integrated with a tablet for dealing with writing characters in strokes? I know of commercial websites that do that, some open source java applications that are not that nice to use -- and a KDE app for learning Japanese characters and phrases, which are just too different to be useful.
Bonus points for traditional characters instead of abbreviated ones.
it has been a long time since my last post. A lot had happened and my interests shifted quite a bit as well. Firstly, I am not contributing with free software actively, like in the good and fun days tinkering with Amarok about a year and a half ago. The point is, desktop development, in general, does not makes me tick anymore. Especially when that involves front-end development. KDE is a great community and the knowledge I had acquired by contributing, for two years, certainly surpasses ordinary desktop development assumptions. It certainly entails good software development practices, agile methodologies, build systems, tests, revision control, community building, software licensing and the like.
But then things changed. I got an internship at The Networking and Telecommunications Research Group (GPRT) to give some dynamism to my undergraduation. My desire was to find something more closer to the bare metal. And that was what I found. The opportunity to toy with a new and challenging area, commonly called the Internet of Things (IoT). Or, as IBM like to puts it, the Internet of Everything. Actually, the Internet of Things is, as every buzzword, drawing a lot of attention from tech guys and gals around the world. With new startups appearing here and there with the promise to solve the several obstacles hindering IoT's envisioned future to become a reality.
Coding for wireless sensor network (WSN) nodes with specs such as 16Mhz - 16 bits Texas Instruments' MSP430 MCUs, 8KB of RAM and 92KB of ROM, among other tasks were part of my daily routine at GPRT. And believe me, embedded device programming can be very challenging. It was not uncommon, at the very beginning of my internship, to find out that, after compiling, your program did not fit in the scarce node's ROM. Several months revolving around WSNs, routing protocols, addressability, wireless interference, energy usage and the whole new set of problems that appears when you are dealing with such wireless low-power devices. Zigbee, IEEE 802.15.4, Contiki, TinyOS are some of the commonplaces when dealing with this kind of technology.
Afterwards, my focus changed to developing middlewares to coordinate those wireless resources-limited devices. Leveraging on ZeroMQ, that amazing free software socket library that acts as a concurrency framework, it was pretty easy to develop a distributed middleware that could scale horizontally. Because everybody knows, scaling vertically is too old school. It is much better to just buy commodity hardware, like Google and every sane company dealing with high processing requirements, instead of wasting tons of dollars to put all your highly expensive servers on steroids.
Scalable systems are a must for IoT, as every WSN node has to somehow report their sensor's measurements. Usually in a timely manner. There are alternatives to that, but I will not enter into the merits of them. To give a feel of envisioned WSN proportions, the academic literature mentions numbers close to three thousand or more nodes in a single network. And standards like IPV6 for Low-Power and Lossy-Networks (6LowPan) and Rooting over Low-Power and Lossy-Networks (RPL), where developed to address these scenarios. Obviously, the field is recent enough for me to doubt that anyone uses WSNs that big in production. Curiously though, one of our industry partners has a network of similar proportions, but it still uses wires. Have you ever wondered how hard is to manage all that wires as the shop floor is reassembled. Their case, however, is very specific. Since shop floors tend to suffer from severe interference in the wireless spectrum, mainly due to soldering and the presence of metal everywhere. Despite this kind of harsh environment, IoT adoption is most a matter of how cheap are those nodes and how we overcome the existent technical difficulties o the wireless world. Cause, in the end of the day, industry traction around new technologies is just a consequence of their economic effectiveness.
And believe me, tons of data flowing can be really precious when decision making is at stake. Curiously, decision making is actually the biggest paradox limiting IoT adoption by ordinary people. While data can help a professional to make a decision based on available data about a phenomenon, being that common in several disciplines like engineering, economics, administration and so on, that is not actually what most people expect from a home automation system for instance. Technology already provided us with too many gadgets to tame and I suspect that no one wants more.
Companies like SmartThings and NinjaBlocks, which are in the home automation niche market, face this kind of problem. Requiring users some input in an "if this than that" format, in order to adjust the system for the end users needs. Therefore, end users have to know in advance what it would be interesting for the system to do if something happens. Should the lights be turned off if nobody is inside the house? Should this? Should that even if x happens? Managing all that can be pretty tedious and it is, in my humble opinion, the biggest challenge that these IoT era home automation systems have to copy with, in order to really succeed in the long run.
Despite that, I have been also exploring the realms of Big Data. From NoSQL graph-based databases (Titan for the win o/) to map-reduce architectures for our new research project at the internship.
In the end, my next post will cover details about my personal project to analyse the data from the Brazilians' 2012 Municipal Election with Python, that initiative aims to provide me the means to master the art of data analysis. And, I hope that it can be somehow informative to both people interested in the data analysis process and Brazilians politics.
If you’ve been following development in KDE recently, you might have the feeling that either it’s not changing much or we’re doing things that are not having an immediate impact on our daily life.
This is changing soon, the KDE developers have definitely been busy. We’ve been preparing the technical layer where we will base our future technologies. That’s Qt 5 and the KDE Frameworks 5, which has been discussed and documented largely.
Anyway, I didn’t want to discuss about the past today, but instead point you out that everything is in place for you to start porting the project of your liking and start taking advantage of Qt5 and KF5.
So if you’re interested, just take a peek at the projects that are already ported (Kate, KDE Workspace, KAlgebra, KGeography and probably others I’m not aware of, you can check if the repository has a frameworks branch) and give it a try.
If you have questions, remember you can use #kde-devel in irc.freenode.org, kde-frameworks-devel mailing list.
Interesting times coming, do you want to be part of it?
You can find the release schedule for KDE Applications and Platform 4.13 at http://techbase.kde.org/Schedules/KDE4/4.13_Release_Schedule
This schedule consolidates the changes made for 4.12 since it seems to have worked well.
Also remember that 4.13 will be applications and kdelibs only; KDE Workspace (aka Plasma) will be getting 4.11.x LTS releases until August 2015
At first, I would like to thank you, becase we’ve got quite a lot of comments during our usability studies and it was really helpful. You could see a prototype in Björn’s blog post and now, when it’s almost finished, I would like to show you our result to get again some feedback. I hope you like it more than previous versions.
The last missing functionality are options for opening the editor and turning on/off wireless etc. We are still not sure where to put them, so there is still a space for your ideas. And again, we really need a design hero, because as you can see in the video, our icons needs some polishing, i.e the vpn icon is barely visible and the ethernet icon with question mark is quite ugly, that’s a proof that developers are not good designers.
Since the KDE SC 4.12 release a month ago, it’s about time to look at the changes of Kate in 4.12:
- Vi input mode: complete macro support
- Heavily improved vi input mode
- Multi-column text editing (thanks to Andrey Matveyakin)
- MiniMap: align to the top (instead of vertically centered)
- Projects plugin: modified files now show a modified icon
- Improved C++/Boost Style indenter
- in total, 21 issues (including wishes and bugs) were fixed
Kate will get more polishing in the next 4.x releases, for instance, in KDE SC 4.13 Kate optionally supports animated bracket matching.
However, as already mentioned in Kate in 4.11, the main efforts are put into making Kate on Qt5 and Frameworks 5 rock solid. Already now Kate, KWrite and Kate Part are fully ported, i.e. all are free of KDE4support libraries. A preview (already one month old!) can be found here.
Besides that, Kate and KDevelop again join forces and there is a developer meeting from 18th to 25th of January 2014. So expect quite some blogs about Kate on 5 then!
Kexi reports have for a long time supported a powerful scripting interface. Forms however, while supporting a wide range of data-enabled widgets, only have support for simple macros.
Forms and reports are both plugins in Kexi. Kexi plugins are based around views....a plugin exposes a Design View and a Data View (in the normal case). Reports and Form design views are based on quite extensive wysiwyg designers which create XML representations used by the Data view to display the data in the object.
After another development cycle, this time of 2,5 years, the libssh Team is proud to announce version 0.6.0 of libssh.
The most important functionality which has been added is a new callback-based server API. Also we added ECDSA support and a new algorithm called firstname.lastname@example.org for key exchange to have something better than the NIST curves. OpenSSH also uses email@example.com as the default for key exchange. For ECDSA there is a complete new API for public key management available. Also a big improvement is Kerberos support which has been tested by Red Hat engineers with FreeIPA and gssproxy.
In the first episode of Luminosity of 2014, I'll be visiting the following topics as well as taking your questions as we go:
- Do people need protection from themselves? More and more Free software developers are building their software with the idea of protecting users (or as I like to think of them: people) from themselves and, by extension, others. This started with improved SSL bars in web browsers, but has gone much farther than that with a mix of simplified (some would say "dumbed down") interfaces, security innovations (e.g. containers) and limiting of use cases that are deemed "dangerous". We'll discuss the gamut of approaches being taken and which may hold more promise than others.
- Hardware for hackers and innovators: Given that I've been working a lot this past year on the Improv device, there's been ample opportunity to look at the field of hardware available to Free software developers, users, innovators and entrepreneurs. I'll try to sum up what I've learned and take questions from the audience.
- Q&A: If you have a burning question to ask, do so in the comments here or on G+ and I'll do my best to get to it in the show. Or you can ask live on irc ...
Qt and KDE will be present at FOSDEM, the largest open-source event in Europe. One more year, we will be sharing the Desktops DevRoom with Gnome, Unity, Enlightenment, LXQt and Hawaii (a Qt Quick desktop environment). We recently published the schedule for the devroom, which will be also available in the printed booklet available at the front desk.
For the 2014 edition, the FOSDEM organization wants to achieve 100% recording of presentations. That means every presentation, in every room (devroom, lightning talk, main conference, etc) must be recorded. That’s hundreds of talks. While the FOSDEM and devrooms organization teams comprise a lot of people, we are far too busy already with the organizative stuff and cannot spend time doing the actual recordings.
Good thing is, you can help!
Do you want to join the FOSDEM Video Team and receive the t-shirt? We are now looking for volunteer cameramen (and camerawomen, of course ).
FOSDEM will provide you with equipment and training, you only need to start recording, focus, make sure nobody gets between the camera and the speaker/stage, etc. You do NOT need to record the whole track, even one talk recording would help. More details on what will be required from you are available in this e-mail from Wouter.
Please contact me (pgquiles at elpauer dot org) if you are interested in recording one or more presentations from the Desktops DevRoom.
I see this question occasionally in #kde, #kubuntu, and #amarok. Tonight I saw the first answer that seems to shed light on it.
Cousin_luigi said in #kde tonight: "I expect the volume to be at the same level I left it." Which is perfectly reasonable! And yet, we often find something else. Axtroz had the answer:
Cousin_luigi, because some systems run "alsactl restore" on startup which restores the volume state saved with "alsactl store" and that tunes the volume for a particular soundcard. Since Pulseaudio and Alsa are working together via plugins, pulse follows. Check your init scripts, or raise the volume to an apropriate level and run alsactl store as root.
I don't have this problem, so I didn't test the solution. However, here it is as a public service. Thanks to Cousin_luigi for asking, and Axtroz for answering.
Converging Form FactorsOn the Frameworks, one can soon expect to see releases of KDE's Plasma Workspaces. A Technology Preview of Plasma 2 has already been released and this ambitious project has not lost any of its goals. Today, I noted that ZDNet's Steven J. Vaughan-Nichols wrote about what he expects from Ubuntu in 2014. There, he quotes Jono Bacon talking about formfactor convergence.
Separating The Men From The BoysI would argue that neither Microsoft, GNOME, nor Ubuntu/Canonical are even half as ambitious as the KDE Community in the area of convergence. They are all merely catching up to the state of KDE technology in 2010. In that year, the KDE community released Plasma Netbook, a plasma-based shell optimized for another form factor: the netbook. With far more advanced convergence than anybody today has yet shown: Plasma Netbook and Plasma Desktop share well over 90% of their code, as opposed to not even sharing toolkit or display shell (Ubuntu) and having a completely separate desktop (Microsoft Windows).On Plasma, widgets can dynamically adjust to the constraints of their environment, be it on a panel, free-form on the desktop, full-screen, in a window or in a tiled environment. And yes, the different form-factor optimized shells be switched on-the-fly. No separate login or account, no loss of functionality, no separate applications for each shell, nothing like that. It just works.
I understand what Microsoft is doing - trying to build a single user interface for vastly different devices. And I guess we've all seen how it does not work - Apple is smarter, in that regard. Underlying technology can of course be re-used but you simply can not make a UI which works equally well on a 75 DPI 24" screen with mouse & keyboard, on a 455 dpi touch phone, on a 300 DPI touch tablet and a 64" television with Kinect or something like that...
Instead, the Plasma team has build a technology which separates presentation from logic, allowing you to build UI's which adapt dynamically to the needs of the form factor.
Moving ForwardThis technology will be brought to a new level with the release of Plasma Workspaces 2, where your workspace will be able to smoothly morph into a different form factor without even a hickup. So, when it comes to the convergence of formfactors, KDE is lightyears ahead of what the competition is even aiming for.
Did I say something about the power of innovating in the open? That's what I'm talking about.
Today, the KDE Community has released KDE Frameworks 5. I've written a guest blog on linux.com about why I think that that matters for Free Software.
Why it mattersIn short, I argue that the most used Free Software toolkit stands to benefit tremendously from the knowledge, experience and code of the dominant Free Software desktop project.
The value of a community like KDE can hardly be overstated. The open and inspiring place it provides naturally is something companies spend billions to build (and usually fail). The KDE community has already contributed a lot to Qt and with Frameworks 5 this stands to continue at a fast pace.
So, as I conclude in the article:
A complete, mature and openly developed toolkit on and for open source platforms is important for Free and Open Source, for current hardware configurations, the new generation of mobile devices and other consumer electronics. KDE Frameworks is an important contribution to innovative and compelling open platforms.
And it is coming soonAnd Free Software won't have to wait long. As Sebastian Kügler noted, most of the work is done. Two modules, KArchive and Threadweaver are already good to go and as Mirko Boehm blogged, these are very mature and usable already. He noted that no API breaks are coming for Threadweaver and as far as I know the same is true for KArchive, making these two ready for use right now.
This allows developers to save development and maintenance work. You can be assured that this code is mature, tested and build based on real-world needs, developed in an open governance process and following clear code- and API style guidelines.
And that is a big deal ;-)
Today, the KDE Community released a tech preview of the upcoming KDE 5 Frameworks, the new, modularised incarnation of what was previously distributed simply as the KDE libraries. The new frameworks are drop-in extensions to Qt applications, with minimal and well-documented dependencies for easier deployment. The tech preview contains two frameworks that are marked as mature, namely KArchive and ThreadWeaver. The updated ThreadWeaver was my major piece of library coding work in 2013, and was finished just in time for the release. Even though it is a tech preview, it is stable, and no major (or even significant but minor) changes in the current API are expected until the final release. Programmers are already encouraged to use it, and provide feedback and bug reports.
ThreadWeaver is a concurrent execution scheduler written in C++. Available for all target platforms of the Qt framework, including desktop, mobile and embedded environments, ThreadWeaver delivers concurrent execution of tasks, load balancing with regard to user-defined criteria, multiple independent queues, processing graph modelling, aggregate jobs and other comprehensive features. As all other KDE frameworks, ThreadWeaver is Free Software. Its only dependency is Qt, which makes it a tier 1 framework in KDE’s lingo.
A number of the new features of ThreadWeaver were announced at Akademy 2013. Jobs, the unit of concurrent execution in ThreadWeaver, are now managed by the queue using shared pointers, meaning that auto-delete behaviour is implicit and controlled by the user. Helper templates are available to queue stack or member variables, so allocation of jobs can be static or dynamic. Functors or lambda functions can be used to construct jobs. Job aggregates like collections and sequences now execute their own run() method before queueing their elements, so that aggregates can generate their own elements. Success and queueing state of jobs are now integrated into a single status. Jobs can signal the result of execution by setting a status, but also using exceptions, simplifying error reporting in more complex job classes. Jobs can be decorated, and no more inherit QObject by default. Decorators can be used to add signals, change priorities or modify just about any behaviour of jobs independently of the actual job class used. The construction of the global queue can now be customised using a queue factory. The QueueStream API greatly simplifies queueing jobs with a familiar iostream-like C++ syntax.
ThreadWeaver follows the Unix idiom of doing one thing, and doing it right. Similar to how small Unix programs can be combined to create an practically infinite space of computing solutions, ThreadWeaver offers itself to programmers as an add-on module with minimal dependencies. Including it extends an application with concurrent scheduling capability. But the same Unix idiom is also applied in a second sense. Within ThreadWeaver, a few basic concepts – jobs and their aggregates, queues and policies – are implemented that again provide simple building blocks that can be combined creatively, offering a vast space of potential solutions within the scope of the application.
The history of ThreadWeaver goes back to KDE 3. The idea of implementing a thread pool based execution scheduler that manages dependencies between jobs was implemented as a proof of concept using Qt 3. However it turned out to be difficult to implement and use because of the lack of thread-safe reference counting of the implicitly shared classes at the time. These fundamental problems have been solved with the release of Qt 4. Additionally, the introduction of cross-thread signal-slot connections further simplified the communication between jobs and the application’s user interface. The first production ready version of ThreadWeaver was released as part of KDELibs with KDE 4.0. For KDE Frameworks 5, it was almost completely re-written to simplify memory management of jobs, make use of new Qt 5 features like atomic variables, and in part to reflect new language constructs in C++11 like lambda functions. ThreadWeaver comes with an extensive set of unit tests that all pass in the tech preview (hear, hear).
In the following weeks and months, the framework will be polished and debugged based on user feedback. Also, a series of posts here on this blog will introduce individual ThreadWeaver concepts and features in depth, mostly based on example programs, including contrasting it to thread handling in Qt using QThread or Qt Concurrent. ThreadWeaver is very close to production quality, having been tested continuously in the last couple of months. There may still be smaller, source compatible changes to the framework. We ask interested programmers out there to provide feedback and bug reports to make ThreadWeaver what it should be — a worry-free, easy to use and powerful add-on to Qt that programmers enjoy using. Have fun!
[Image by Shannan Sinclair, thanks: http://www.flickr.com/photos/originalbliss/2897019812 ]
Filed under: Coding, CreativeDestruction, English, FLOSS, KDE, OSS, Qt, ThreadWeaver Tagged: Akademy, Creative Destruction, FLOSS, free software communities, KDE, kde community, Open Governance, ThreadWeaver