Feed aggregator
Nordic KDE [Global Themes (Plasma 5)]
Nordic KDE [Global Themes (Plasma 5)]
How YOU Help With Quality
In today’s other blog post, I mentioned how we’ve been getting a huge number of bug reports since the Mega-Release. If you’re not in software engineering, this probably seems like a bad thing. “Oh no, why so many bugs? Didn’t you test your software properly!?”
Since most people are not involved in software engineering, this perspective is common and understandable. So I’d like to shed a light on the assumptions behind it, and talk about the challenges involved in improving software quality, which is a passion of mine.
Don’t kill the messengerSee, bug reports are a “don’t kill the messenger” type of thing. Bugs are there whether they get reported or not, and getting them reported is important so you have a more accurate picture of how your software is actually being used by real people in the real world.
In our KDE world, the alternative to “lots of bug reports” isn’t “few bug reports because there are few bugs” but rather “few bug reports because the software isn’t actually being used much so no one is finding the bugs.” What matters more is the severity of the actionable bug reports.
What bug-free software looks likeThat sounds so defeatist! Surely it must actually be possible to have bug-free software, right?
Yes. But to achieve it, you have to test literally every possible thing the software can do to make sure it performs correctly in the environment in which it’s doing it. If the software can do infinite things in infinite environments, then testing also becomes infinite and therefore impossible, so combinations get missed and bugs sneak through. Bug-free software must aggressively limit the scope and variety of those environments to just the ones that can be tested. What does that look like in practice?
- Limit what the software can do in the first place. Remove as many features, options, and settings as possible without compromising the product’s necessary core functionality.
- Limit how the user can modify and extend the software after release. No user-created themes, widgets, scripts–nothing! Every modification to what the software can do or how it looks must go through a the same QA team that QAd the software in its original state. 1st-party modifications only.
- Limit the versions of upstream 3rd-party libraries, dependencies, and kernels that the software is allowed to use. Lock those versions and test everything the software can do on those specific versions.
- Limit how downstream 3rd-party user-facing software (i.e. apps) can interface with the system. Lock it down in a sandbox as much as you can so any misbehavior can’t affect the rest of the system.
- Limit the hardware that the software stack is allowed to run on. Test everything the software can do only on that hardware.
Does this sound very much like how KDE software is developed, distributed, and used? I’d say it’s more like how Apple builds products (and note that Apple products still have bugs, but I digress)! By contrast: KDE develops lots of features and options; we’re liberal with supported library, dependency, and kernel versions; and we don’t prevent you from installing our software on any random device you can get your hands on.
You can see the challenge right away! The foundation of quality is a set of restrictions that we don’t want to impose on ourselves and our users. You folks reading this probably don’t want us to impose them on you, either.
Quality from chaosSo is it just impossible to ensure quality in a permissive environment? No, but it’s harder. That’s right: we in the KDE free software world set for ourselves a fundamentally more difficult task than the big corporations with their billions of dollars of resources. Given this, I think we’ve done a pretty darn impressive job with the Mega-Release. And judging by initial impressions out there, it seems like many others agree too!
So how did we do it?
We lengthened our beta period to 3 months and relied on bug reports from people using the software in their own personal environments.
Yes you, loyal reader! We wanted to hear how our software was working on your 12-year-old Netbook. We wanted to hear how it worked when plugged into two TVs and a rotated monitor, all through a KVM switch. We wanted to hear how it coped with the most bizarre-looking 3rd-party themes. By using our flexible and non-limited software in your diverse ways on your diverse hardware, you’re testing it and finding all the bugs that we lack the resources to find ourselves.
Does this sort of QA work sound like something you don’t want to do? That’s 100% fine. But then you need for someone else to be the QA team for you. There are two options:
- Buy a computer with KDE software pre-installed; there are a lot of them now! Then it’s the vendor’s responsibility to have done adequate QA on their own products. Is it buggy anyway? Complain to them or find a better vendor!
- If you’re going to install it yourself, limit yourself to common hardware, default software settings, and operating systems that are relatively conservative in their update schedules. Then the QA has been provided by others who who already used your exact setup and reported all the bugs affecting it.
But what if you do want to help out with making the software better for others, but you’re not a programmer? Congratulations, you’re a real-life superhero.
We’ve already talked about reporting bugs. It’s also important to do a good job with your bug reports so they’re actionable! I’ll encourage folks to read through our documentation about this. Low-quality bug reports don’t just waste our time, they waste yours as well!
But where do all those 150-200 bug reports per day that I mentioned actually go? There’s a flip side which is that someone needs to do something with every single one of them. The more bug reports we get (which, again, is good!) the more we need people to help triaging them.
Because the truth is, most bug reports don’t begin life being actionable for developers. They may be missing key information; they may be mistaking a feature for a bug; they may be describing an issue in someone else’s software; they may be about an issue that was already fixed in a version of the software that the reporter doesn’t have; they may be describing a real issue but in an unclear and confusing way; and so on.
The job of bug triagers is to make each of these bug reports actionable. Ask for missing information! Move them to the right products! Set the version and severity appropriately! Mark already reported bugs as duplicates of the existing report! Mark obvious upstream or downstream issues accordingly and direct people to the places where they can report the bugs to the responsible developers! Try to reproduce the issue yourself and mark it as confirmed if you can! And so on. It isn’t terribly glamorous work, so there aren’t very many people lining up to be volunteer bug triagers, unlike developers. But it’s very important. And so every person who helps out adds resources to what’s currently a very small team, making a massive difference in the process.
If you’ve been looking for a way to help out KDE in a way that doesn’t require programming or a consistent time commitment, this is it. Triage a few bugs here, a few bugs there. Chip in when you can. If 30 people each triaged three bugs a day (this would take under 10 minutes, on average), we’d be in an amazing position.
So get started today! I’m available to help in the #kde-bugs Matrix room.
Still don’t wanna? Donate to KDE e.V. so we can eventually hire our own professional bug triage and QA team!
Working Build With KDE Frameworks 6
With KDE’s Frameworks 6 being released recently, I’ve been working on getting Tellico to compile with it. It didn’t actually take too much work since I’ve been gradually porting away from any deprecated functions in Qt5.
There’s plenty to do to make sure everything is fully functional and has the correct appearance. But I’m hopeful to have a release soon. At the moment, the master branch compiles with either KF5/Qt5 or KF6/Qt6.
