Planet KDE

Let’s go for my web review for the week 2024-04.

Where have all the flowers gone? | daverupert.com

Tags: tech, web, blog, culture

Good continuation of “where have all the websites gone?”. They’re still here but we changed, all the more reason for curating.

https://daverupert.com/2024/01/where-have-all-the-websites-gone/

The Open Source Sustainability Crisis

Tags: tech, foss, sustainability

Yes, there’s something to do in this space. More funding is necessary, some form of platform might help… but it definitely won’t be enough.

https://openpath.chadwhitacre.com/2024/the-open-source-sustainability-crisis/

Platform Tilt: Documenting the Uneven Playing Field for an Independent Browser Like Firefox - Open Policy & Advocacy

Tags: tech, firefox, web, browser, google, apple, microsoft

Nice call from Mozilla to make this public. This way it is very obvious where the blockers are on some platforms.

https://blog.mozilla.org/netpolicy/2024/01/19/platform-tilt/

Victory! Ring Announces It Will No Longer Facilitate Police Requests for Footage from Users | Electronic Frontier Foundation

Tags: tech, surveillance

Still a long way to go but definitely a move in the right direction.

https://www.eff.org/deeplinks/2024/01/ring-announces-it-will-no-longer-facilitate-police-requests-footage-users

Fairly Trained launches certification for generative AI models that respect creators’ rights

Tags: tech, ai, machine-learning, gpt, copyright, licensing

This is an interesting move, we’ll see if this certification gets any traction.

https://www.fairlytrained.org/blog/fairly-trained-launches-certification-for-generative-ai-models-that-respect-creators-rights

Nightshade: Protecting Copyright

Tags: tech, ai, machine-learning, copyright

The tooling to protect against the copyright theft of image generator models training is making progress. This will clearly turn into an arm race.

https://nightshade.cs.uchicago.edu/whatis.html

Reading QR codes without a computer!

Tags: tech, barcode, qrcode

Another great way to understand how QR codes work.

https://qr.blinry.org/

Journey to the Centre of the JVM — Daniel Spiewak

Tags: tech, java, multithreading, memory

Very interesting talk. It gives a good idea of some properties of the JVM memory model. It also shows how the CPU architecture properties can leak all the way through changing some of the behavior of JVM code. Too bad this particular thing seems badly documented on the JDK side.

https://www.youtube.com/watch?v=EFkpmFt61Jo

That’s it! I’m making my own C++ package manager…

Tags: tech, c++, dependencies, buildsystems

Clearly I can understand the feeling. That’s yet another attempt at a solution for this… it’s young, will it get traction? Has the benefit of being kind of simple, too simplistic maybe?

https://david-delassus.medium.com/thats-it-i-m-making-my-own-c-package-manager-555eecbf7d2e

Re: The Case for Rust (in the base system)

Tags: tech, rust, c++, safety, system

Very interesting contribution to the FreeBSD hackers mailing list. Gives quite a good background about Rust, C++ and safety. Debunks a few claims you can easily hear in many places as if they were common knowledge.

https://lists.freebsd.org/archives/freebsd-hackers/2024-January/002876.html

On‐demand JSON: A better way to parse documents? - Keiser - Software: Practice and Experience - Wiley Online Library

Tags: tech, json, parsing, performance, c++

Very interesting approach to JSON parsing. Comes with a very thorough performance analysis.

https://onlinelibrary.wiley.com/doi/10.1002/spe.3313

Handling external API errors: A resumable approach

Tags: tech, api, services, consistency

On the difficulties of dealing with third party APIs. How to handle failures and reach eventual consistency? A few good solutions and patterns are proposed here.

https://thoughtbot.com/blog/handling-errors-when-working-with-external-apis

The most important goal in designing software is understandability | nicole@web

Tags: tech engineering, craftsmanship, quality

Understandability is indeed a very important goal. There are easy ways to improve it in a system.

https://ntietz.com/blog/the-most-important-goal-in-designing-software-is-understandability/

TDD Revisited - Ian Cooper - NDC Porto 2023 - YouTube

Tags: tech, tdd, tests

Nice talks, debunks very well quite a bit of the fallacies around people wrongly practicing TDD. I never realized how the root cause of those fallacies was the misusing of the “unit tests” term instead of “developers test”. This was indeed the wrong term, knew it, but first time I realize how profound the effects were.

https://www.youtube.com/watch?v=IN9lftH0cJc

Trunk Based Development

Tags: tech, version-control

A nice knowledge base about what is probably my favorite branching model. Goes in the variations you can have, the trade-offs and the other techniques you need to bring in for it to work well.

https://trunkbaseddevelopment.com/

On “owning” software - avdi.codes

Tags: tech, economics, cost, licensing, services, foss

Good exploration on how the total cost of ownership is spread depending on how is licensed the software you use and where you get your support from. I think there’s one point a bit too glanced over in the analysis of the cost for the proprietary SaaS case: what’s the cost of fixing a bug that affect your team? You might be a tiny fish in a large pond, good luck getting attention from support in this case.

https://avdi.codes/on-owning-software/

Lessons learned: 1,000 days of distributed at Atlassian

Tags: tech, atlassian, remote-working

Interesting report about distributed and remote work at Atlassian. They really did their homework. I recommend reading the whole report, they came up with a few original ideas.

https://www.atlassian.com/blog/distributed-work/distributed-work-report

Cancel your meetings if you can live with the outcome - Andy Grunwald

Tags: tech, meetings

Interesting approach to handling meetings. Start thinking about what happens if you’re not there.

https://andygrunwald.com/blog/cancel-your-meetings-if-you-can-live-with-the-outcome/

Bye for now!

KDE’s 6th Megarelease is the is-it-tacky-is-it-awesome name we came up with for the combined release of KDE Frameworks 6, Plasma 6 and KDE Gear’s bundle of apps and libraries. It’s out in a month’s time and it’s the porting of all our libraries and many of our apps to Qt 6. In principle this makes no difference to end users but we still like to make a song and dance about it and there will be new features and old cruft removed which allows for accelarated new features to come shortly. But first it needs testing. So download KDE neon Testing Edition which is build with the Git branches of the soon to be released products and install it either on hardware if you can or on something like Virtualbox (mind on Virtualbox you need to turn on “Enable 3D Accelaration” in Display settings because it uses Wayland, you should also turn on “EFI Special OSes only” if only to feel special).

Many thanks to Carlos and the others who have worked hard to get everything here.

In the beginning, there was C.

That sentence actually could serve as the introduction to a multitude of blog posts, all of which would come to the conclusion “legacy programming conventions are terrible, but realistically we can’t throw everything out and start over from scratch”. However, today we will merely be looking at two ways C has contributed to making language interoperability difficult.

extern "C", but for structs

In the first installment of this series, I mentioned that one blocker to language interoperability is struct layout. Specifically, different programming languages may organize data in structs in different ways. How can we overcome that on our way to language interoperability?

Layout differences are mostly differences of alignment, which means that data is just located at different offsets from the beginning of the struct. The problem is that there is not necessarily a way to use keywords like align to completely represent a different language’s layout algorithm.

Thankfully, there is a solution. In our example used previously, we were using Rust and C++ together. It turns out that Rust can use the #[repr(C)] representation to override struct layouting to follow what C does. Given that C++ uses the same layouting as C, that means that the following code compiles and runs:

// file: cppmodule.cpp #include <iostream> #include <cstdint> struct Foo { int32_t foo; int32_t bar; bool baz; }; void foobar(Foo foo) { std::cout << "foo: " << foo.foo << ", bar: " << foo.bar << ", baz: " << foo.baz << '\n'; } extern { #[link_name = "_Z6foobar3Foo"] pub fn foobar(foo: Foo); } #[repr(C)] pub struct Foo { pub foo: i32, pub bar: i32, pub baz: bool, } fn main() { let f = Foo{foo: 0, bar: 42, baz: true}; unsafe { foobar(f); } }

My proof-of-concept project polyglot automatically wraps C++ structs with #[repr(C)] (and also does so for enums).

The one major downside of this approach is that it requires you to mark structs that you created in your Rust code with #[repr(C)]. In an ideal world, there would be a way to leave your Rust code as is; however, there is currently no solution that I am aware of that does not require #[repr(C)].

Arrays, strings, and buffer overflows

Now that we’ve covered structs in general, we can look at the next bit of C behavior that turned out to be problematic: handling a list of items.

In C, a list of items is represented by an array. An array that has n elements of type T in it really is just a block of memory with a size n * sizeof(T). This means that all you have to do to find the kth object in the array is take the address of the array and add k * sizeof(T). This seemed like a fine idea back in the early days of programming, but eventually people realized there was a problem: it’s easy to accidentally access the seventh element of an array that only has five elements, and if you write something to the seventh element, congratulations, you just corrupted your program’s memory! It’s even more common to perform an out-of-bounds write when dealing with strings (which, after all, is probably the most used type of array). This flaw has led to countless security vulnerabilities, including the famous Heartbleed bug, (you can see a good explanation of of how Heartbleed works at xkcd 1354).

Eventually, people started deciding to fix this. In languages like Java, D, and pretty much any other language invented in the last 25 years or so, strings (and arrays) are handled more dynamically: reading from or writing to a string at an invalid location will generally throw an exception; staying in bounds is made easy by the addition of a length or size property, and strings and arrays in many modern languages can be resized in place. Meanwhile, C++, in order to add safer strings while remaining C-compatible, opted to build a class std::string that is used for strings in general (unless you use a framework like Qt that has its own string type).

All of these new string types are nice, but they present a problem for interoperability: how do you pass a string from C++ to Rust (our example languages) and back again?

Wrap all the things!

The answer, unsurprisingly, is “more wrappers”. While I have not built real-life working examples of wrappers for string types, what follows is an example of how seamless string conversion could be achieved.

We start with a C++ function that returns an std::string:

// file: links.cpp #include <string> std::string getLink() { return "https://kdab.com"; }

We’ll also go ahead and create our Rust consumer:

// file: main.rs mod links; fn main() { println!("{} is the best website!", links::getLink()); }

Normally, we would just create a Rust shim around getLink() like so:

// wrapper file: links.rs extern { #[link_name = "_Z7getLinkB5cxx11v"] pub fn getLink() -> String; // ??? }

However, this doesn’t work because Rust’s String is different from C++’s std::string. To fix this, we need another layer of wrapping. Let’s add another C++ file:

// wrapper file: links_stringwrapping.cpp #include "links.h" // assuming we made a header file for links.cpp above #include <cstring> const char *getLink_return_cstyle_string() { // we need to call strdup to avoid returning a temporary object return strdup(getLink().c_str()); }

Now we have a C-style string. Let’s try consuming it from Rust. We’ll make a new version of links.rs:

// wrapper file: links.rs #![crate_type = "staticlib"] use std::ffi::CStr; use std::os::raw::c_char; use std::alloc::{dealloc, Layout}; extern { #[link_name = "_Z28getLink_return_cstyle_stringv"] fn getLink_return_cstyle_string() -> *const c_char; } pub fn getLink() -> String { let cpp_string = unsafe { getLink_return_cstyle_string() }; let rust_string = unsafe { CStr::from_ptr(cpp_string) } .to_str() .expect("This had better work...") .to_string(); // Note that since we strdup'ed the temporary string in C++, we have to manually free it here! unsafe { dealloc(cpp_string as *mut u8, Layout::new::()); } return rust_string; }

With these additions, the code now compiles and runs. This all looks very convoluted, but here’s how the program works now:

1. Rust’s main() calls links::getLink().
2. links::getLink() calls getLink_return_cstyle_string(), expecting a C-style string in return.
3. getLink_return_cstyle_string() calls the actual getLink() function, converts the returned std::string into a const char *, and returns the const char *.
4. Now that links::getLink() has a C-style string, it converts it into a Rust CString wrapper, which is then converted to an actual String.
5. The String is returned to main().

There are a few things to take note of here:

1. This process would be relatively easy to reverse so we could pass a String to a C++ function that expects an std::string or even a const char *.
2. Rust strings are a bit more complicated because we have to convert from a C-style string to CString to String, but this is the basic process that will need to be used for any automatic string type conversions.
3. This basic process could also be used to convert types like std::vector.

Is this ugly? Yes. Does it suffer from performance issues due to all the string conversions? Yes. But I think this is the most user-friendly way to achieve compatible strings because it allows each language to keep using its native string type without requiring any ugly decorations or wrappers in the user code. All conversions are done in the wrappers.

Implementation

Based on the concepts here, I’ve written a (non-optimal) implementation of type proxying in polyglot that supports proxying std::string objects to either Rust or D. In fact, I’ve taken it a bit further and implemented type proxying for function arguments as well. You can see an example project, along with its generated wrappers, here.

Next up

Interoperability requires lots of wrappers, and as I’ve mentioned, polyglot can’t generate wrappers for anything more complex than some basic functions, structs, classes, and enums. In the next installment of this series, we’ll explore some viable binding generation tools that exist today.

The post Mixing C++ and Rust for Fun and Profit: Part 2 appeared first on KDAB.