this post was submitted on 29 Aug 2024

204 points (99.5% liked)

Linux

48287 readers

619 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
No misinformation
No NSFW content
No hate speech, bigotry, etc

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago

MODERATORS

AgreeableLandscape@lemmy.ml

nooter692@lemmy.ml

MarcellusDrum@lemmy.ml

cypherpunks@lemmy.ml

cyclohexane@lemmy.ml

d3Xt3r@lemmy.nz

204

Debian Orphans Bcachefs-Tools: "Impossible To Maintain In Debian Stable" (www.phoronix.com)

submitted 2 months ago by pnutzh4x0r@lemmy.ndlug.org to c/linux@lemmy.ml

45 comments fedilink hide all child comments

Even before the Bcachefs file-system driver was accepted into the mainline kernel, Debian for the past five years has offered a "bcachefs-tools" package to provide the user-space programs to this copy-on-write file-system. It was simple at first when it was simple C code but since the Bcachefs tools transitioned to Rust, it's become an unmaintainable mess for stable-minded distribution vendors. As such the bcachefs-tools package has now been orphaned by Debian.

From John Carter's blog, Orphaning bcachefs-tools in Debian:

"So, back in April the Rust dependencies for bcachefs-tools in Debian didn’t at all match the build requirements. I got some help from the Rust team who says that the common practice is to relax the dependencies of Rust software so that it builds in Debian. So errno, which needed the exact version 0.2, was relaxed so that it could build with version 0.4 in Debian, udev 0.7 was relaxed for 0.8 in Debian, memoffset from 0.8.5 to 0.6.5, paste from 1.0.11 to 1.08 and bindgen from 0.69.9 to 0.66.

I found this a bit disturbing, but it seems that some Rust people have lots of confidence that if something builds, it will run fine. And at least it did build, and the resulting binaries did work, although I’m personally still not very comfortable or confident about this approach (perhaps that might change as I learn more about Rust).

With that in mind, at this point you may wonder how any distribution could sanely package this. The problem is that they can’t. Fedora and other distributions with stable releases take a similar approach to what we’ve done in Debian, while distributions with much more relaxed policies (like Arch) include all the dependencies as they are vendored upstream."

...

With this in mind (not even considering some hostile emails that I recently received from the upstream developer or his public rants on lkml and reddit), I decided to remove bcachefs-tools from Debian completely. Although after discussing this with another DD, I was convinced to orphan it instead, which I have now done.

top 45 comments

sorted by: hot top controversial new old

[–] just_another_person@lemmy.world 101 points 2 months ago (1 children)

It's a huge tire fire at this point. This issue isn't Rust, per-se, but the dev is just being an asshole here. Submitting something that is generally problematic and yelling about how it will EVENTUALLY be good is a good way to get your shit tossed out.

He just lost a good amount of favor with the general community.

[–] Laser@feddit.org 10 points 2 months ago (1 children)

Submitting something that is generally problematic and yelling about how it will EVENTUALLY be good is a good way to get your shit tossed out.

What are you hinting at regarding this specific news?

[–] Ghoelian@lemmy.dbzer0.com 16 points 2 months ago* (last edited 2 months ago) (1 children)

This entire thread:

https://lore.kernel.org/lkml/sctzes5z3s2zoadzldrpw3yfycauc4kpcsbpidjkrew5hkz7yf@eejp6nunfpin/

tl;dr: bcachefs dev sent in a massive pull request, linus thinks it's too big and touches too much other code for the current state of the release cycle, dev says his filesystem is the future and should just be merged

[–] Laser@feddit.org 11 points 2 months ago (1 children)

Which is a completely different issue than what the post is about, hence my question

[+] just_another_person@lemmy.world -11 points 2 months ago (1 children)

Wha?

This is exactly what the entire thread is about. Did you not read it?

[–] Laser@feddit.org 16 points 2 months ago (2 children)

The OP is about packaging issues with userspace utilities due to version pinning in Rust. It's an issue with Rust in general. Kent is not obligated to lock dependencies in any particular fashion. He could loosen the dependencies, but there is no obligation, and Debian has no obligation to package it.

This is different from the thread you linked in which the bcachefs kernel code and the submission process is discussed, and on which there was a thread here as well in the last days. But your criticism, as valid as it is, only applies there, not in a thread about tooling packaging issue.

[–] just_another_person@lemmy.world -2 points 2 months ago (1 children)

You must have missed the last half of the post then. Especially the last two paragraphs.

[–] Laser@feddit.org 6 points 2 months ago (1 children)

The only hint at the other topic I see is this:

(not even considering some hostile emails that I recently received from the upstream developer or his public rants on lkml and reddit)

I guess this is about https://www.reddit.com/r/bcachefs/comments/1em2vzf/psa_avoid_debian/, and while I think the title is too broad, the actual message is

If you're running bcachefs, you'll want to be on a more modern distro - or building bcachefs-tools yourself.

I don't consider Kent's reasoning (also further down the thread) a rant - it might not be the most diplomatic, but he's not the only one who has problems with Debian's processes. The xscreensaver developer is another one for similar reasons.

I think, in fairness, bcachefs and Debian currently aren't a good fit. bcachefs is also in the kernel so users can rest it and report, but it wasn't meant to be stable; it's meant to not lose data unrecoverably.

Anyhow, while I think that he's also not the easiest person on the LKML, I don't consider him ranting there; and with the author's and my judgement differing in these points, I'm led to believe that we might also disagree on what qualifies as hostile.

Lastly, while I'm not a big fan of how Rust packaging works, it ensures that the program is built exactly the same on the developer's and other machines (for users and distributors); it is somewhat ironic to see Debian complain about it, since they do understand the importance of reproducibility.

You must have missed the last half of the post then. Especially the last two paragraphs.

There's isn't much more to that issue than that sentence, while all other paragraphs cover the packaging. It's tangential at best.

[–] thingsiplay@beehaw.org 51 points 2 months ago

First Linus, now Debian team. Rough times for bcachefs fans.

[–] superkret@feddit.org 19 points 2 months ago (3 children)

ELI5: What the fuck is going on with Rust in Linux?

[–] that_leaflet@lemmy.world 20 points 2 months ago (2 children)

On the kernel side, there are disagreements between long term C maintainers (who may not know Rust or may actively dislike it) and the new Rust community trying to build in Rust support. To make the Rust parts work, there needs to be good communication and cooperation between them to ensure that the Rust stuff doesn't break.

On the Debian side, they have strict policies that conflict with how Rust development works. Rust has a dependency system called Cargo which hosts dependencies for Rust projects. This is different from C, C++ where there really isn't a centralized build system or dependency hoster, you actually install a lot of dependencies for these languages from your distro's repos. So if your Rust app is built against up to date libraries in Cargo, it's going to be difficult to package those apps in Debian when they ship stable, out of date libraries since Debian's policies don't like the idea of using outside dependencies from Cargo.

[–] p03locke@lemmy.dbzer0.com 21 points 2 months ago

So if your Rust app is built against up to date libraries in Cargo, it’s going to be difficult to package those apps in Debian when they ship stable, out of date libraries since Debian’s policies don’t like the idea of using outside dependencies from Cargo.

As they should. You don't just auto-update every package to bleeding edge in a stable OS, and security goes out the window when you're trusting a third-party's third-party to monitor for dependency chain attacks (which they aren't). This is how we get Crowdstrike global outages and Node.JS bitcoin miner injections.

If some Rust tool is a critical part of the toolchain, they better be testing this shit against a wide array of dependency versions, and plan for a much older baseline. If not, then they don't get to play ball with the big Linux distros.

Debian is 100% in the right here, and I hope they continue hammering their standards into people.

[–] Wooki@lemmy.world 1 points 2 months ago* (last edited 2 months ago)

Big, old man vitriol was a sad show of ignorance of Rust.

https://m.youtube.com/watch?t=1529&v=WiPp9YEBV0Q&feature=youtu.be

[–] qqq@lemmy.world 12 points 2 months ago* (last edited 2 months ago)

This doesn't seem to be a Rust problem, but a modern development trend appearing in a Rust tool shipped with Cargo. The issue appears to be the way things are versioned and (reading between the lines maybe?) vendoring and/or lockfiles. Lockfiles exist in a lot of modern languages and package managers: Go has go.sum, Rust has Cargo which has Cargo.lock, Python has pip which gives a few different ways to pin versions, JavaScript has npm and yarn with lock files. I'm sure there are tons of others. I'm actually surprised this doesn't happen all the time with newer projects. Maybe it does actually and this instance just gains traction because people get to say "look Rust bad Debian doesn't like it".

This seems like a big issue if you want your code to be packaged by Debian, and it doesn't seem easy to resolve if you also want to use the modern packaging tools. I'm not actually sure how they resolve this? There are real benefits to pinning versions, but there are also real benefits to Debian's model (of controlling all the dependencies themselves, to some extent Debian is a lockfile implemented on the OS level). Seems like a tough problem and seems like it'll end up with a lot of newer tools just not being available in Debian (by that I mean just not packaged by Debian, they'll likely all run fine on Debian).

[–] GolfNovemberUniform@lemmy.ml 3 points 2 months ago

Hype.

[–] CasualTee@beehaw.org 18 points 2 months ago* (last edited 2 months ago) (2 children)

the common practice is to relax the dependencies

I found this a bit disturbing

I find that funny that, since this is rust, this is now an issue.

I have not dwelved in packaging in a long while, but I remember that this was already the case for C programs. You need to link against libfoo? It better work with the one the distribution ship with. What do you mean you have not tested all distributions? You better have some tests to catch those elusive ABI/API breakage. And then, you have to rely on user reported errors to figure out that there is an issue.

On one hand, the package maintainer tend to take full ownership and will investigate issues that look like integration issue themselves. On the other hand, your program is in a buggy or non-working state until that's sorted.

And the usual solutions are frown upon. Vendoring the dependencies or static linking? Are you crazy? You're not the one paying for bandwidth and storage. Which is a valid concern, but that just mean we reached a stalemate.

Which is now being broken by

slower moving C/C++ projects (though the newer C++ standards did some waves a few years back) which means that even Debian is likely to have a "recent" enough version of your dependencies.
flatpack and the likes, which are vendoring everything and the kitchen sink
newer languages that static link by default (and some distributions being OK with it)

In other words, we never figured out a proper solution for C projects that will link with a different minor than the one the developer tested.

Well, /rant I guess. The point I'm raising does not seem to be the only one, and maybe far from the main one, for which bcachefs-tools is now orphaned. But I've seen very dubious arguments to try and push back against rust adoption. I feel like people have forgotten where we came from. And while there is no reason to go back per say, any new language that integrate this deep into the system will face similar challenges.

[–] 2xsaiko@discuss.tchncs.de 8 points 2 months ago* (last edited 2 months ago)

Well, it's now an issue with Rust since Cargo makes it a pain in the ass to do. It's one of the big things that makes me very reluctant to write any sort of system tools in Rust despite being a big fan of the language itself.

[–] merthyr1831@lemmy.ml 2 points 2 months ago

Because it's Rust it's now "rust bad" but Debian and other distros have been fucky with dependency management for YEARS. That's why we're moving to flatpak and other containerised apps!

Once again, the wider Linux dev community is trying to openly kneecap the first attempt in decades to bring Linux and its ecosystem up to a vaguely modern standard.

[–] solrize@lemmy.world 15 points 2 months ago (4 children)

Oh man, that isn't a good luck for Rust. What do those tools do and how much code is there anyway? If they are userspace tools, what is the benefit of writing them in Rust?

[–] kbal@fedia.io 42 points 2 months ago (1 children)

Rust is a good language. There is no reason not to use it for userspace tools if you manage dependencies with sufficient care. As with most other currently fashionable languages they make it easy to not do that.

[–] InverseParallax@lemmy.world 12 points 2 months ago

It's a great language, and I even like their deployment/packaging system.

But oh my god it assumes everything follows its rules, and does NOT play well with others.

We need a rust-based distribution, there can be only one.

[–] unknowing8343@discuss.tchncs.de 31 points 2 months ago* (last edited 2 months ago) (4 children)

Everything is better in Rust. Faster, safer... And also the developer experience is amazing with cargo.

The problem here is not Rust, it's the humans, it seems.

The dependencies are set manually, of course, and the dev was enforcing something too strict, it seems, and that is causing headaches.

But, as the debian dude has learned... Rust programs will 99.999 % work if they can be compiled.

[–] gencha@lemm.ee 7 points 2 months ago

5 nines imply a downtime of 6 minutes a year, or every 100,000th operation failing. That's not great for a file system. I assume you picked the number arbitrarily, but still think about it.

[–] atzanteol@sh.itjust.works 6 points 2 months ago (2 children)

Rust programs will 99.999 % work if they can be compiled.

It's the same in C. Most programs don't test against the exact versions of most C libraries either. I'm not sure why he's so amazed at this.

[–] MrPoopyButthole@lemmy.world 22 points 2 months ago (1 children)

Debian is the most stable distro and downstream loads of distros rely on Debian being clean. This dev has to be strict if they want to maintain the status quo. Rather let the user DL this as a standalone package and still use it, instead of it being included by default with the possibility of breaking.

And another thing. Version pinning should be normalized. I just can't bend my mind around code which has to be refactored every 12 - 24 months because dependencies were not version pinned and a new thing broke an old thing. Unless this code is your baby and you stare at every day, constantly moving forward, you should write code that lasts.

[–] atzanteol@sh.itjust.works 6 points 2 months ago

That's fair. Debian does need to be extra careful given their position in the OS chain.

[–] unknowing8343@discuss.tchncs.de 3 points 2 months ago (1 children)

The thing is that, in C the API could be slightly different and you could get terrible crashes, for example because certain variables were freed at different times, etc.

In Rust that is literally impossible to happen unless you (very extremely rarely) need to do something unsafe, which is explicitly marked as such and will never surprise you with an unexpected crash.

Everything is so strongly typed that if it compiles... It will run without unexpected crashes. That's the difference with C code, and that's why Rust is said to be safe. Memory leaks, etc, are virtually impossible.

[–] atzanteol@sh.itjust.works 0 points 2 months ago* (last edited 2 months ago) (1 children)

The thing is that, in C the API could be slightly different and you could get terrible crashes, for example because certain variables were freed at different times, etc. In Rust that is literally impossible to happen unless you (very extremely rarely) need to do something unsafe, which is explicitly marked as such and will never surprise you with an unexpected crash.

What? That's utter BS. Maybe the kernel devs aren't wrong about the "rust religion". Not every bug in C is a memory bug.

We're talking about a future version having regressions or different-than-expected behavior from what your application was built and tested on. I guarantee you that can happen with rust.

[–] dgriffith@aussie.zone 1 points 2 months ago* (last edited 2 months ago) (1 children)

If library devs do versioning correctly, and you pin to major versions like "1.*" instead of just the "anything goes" of "*", this should not happen.

Your unit tests should catch regressions, if you have enough unit tests. And of course you do, because we're all operating in the dream world of, "I am great and everyone else is shit".

[–] atzanteol@sh.itjust.works 3 points 2 months ago

This is true of all languages.

[–] p03locke@lemmy.dbzer0.com 5 points 2 months ago (1 children)

But, as the debian dude has learned… Rust programs will 99.999 % work if they can be compiled.

That's a dumb statement. Every tool needs unit tests. All of them!

If grep complied, but always returned nothing for every file and filter, then it's still not "working". But, hey, it compiled!

[–] LeFantome@programming.dev 3 points 2 months ago

You are not wrong of course but it does not really refute what they are saying.

Many people have had the experience with Rust that, if it builds, the behaviour is probably correct. That does not prevent logic errors but those are not kinds of bugs that relate to dependencies.

These kinds of dependency shenanigans would be totally unsafe in C but Rust seems to handle them just fine.

[–] someacnt_@lemmy.world 1 points 2 months ago

Why is the rust executable not statically compiled? I thought this was the norm.

[–] p03locke@lemmy.dbzer0.com 4 points 2 months ago* (last edited 2 months ago)

Dev being an asshole and not accept Linus' patching advice = Rust is bad?

[–] merthyr1831@lemmy.ml 0 points 2 months ago (1 children)

This isn't Rust's fault lmao, this is distro maintainers trying to fuck with dependencies on software which has been proven to be a horrible way of managing software distribution for years.

When it's a problem with other languages, we don't pin the blame on them. However, because Linux and its developer community is being dragged by its heels to accept ANYTHING more modern than C99 and mailing lists, the typical suspects are using any opportunity to slow progress.

The same shit has happened/is happening with Wayland. The same shit will happen when the next new technology offers a way for Linux to improve itself. A few jackasses who haven't had to learn anything new for a lifetime are gonna always be upset that new Devs might nip at their heels.

[–] solrize@lemmy.world 2 points 2 months ago

If whatever they are doing has been working for stuff written in languages other than Rust, we have to ask what makes Rust special. Rust is a low level language, so its dependencies if anything should be simpler than most, with just a minimal shim between its runtime and the C world. Why does any production software have a version <= X constraint in any of its dependencies anyway? I can understand version >= X, but the other way implies that the API's are unstable and you're going to get tons of copies stuff around. I remember seeing that in Ruby at a time when Python was relatively free of it, but now Python has it too. Microsoft at least understood in the 1990s that you can't go around breaking stuff like that.

No it's not all C99. I'm using Calibre (written in Python), Pandoc (written in Haskell), GCC (written in C, C++, and Ada), and who knows what else. All of these are complex applications with many dependencies. Eclipse (written in Java) is also in Debian though I don't use it. Bcachefs though is apparently just special.

Joe Armstrong (inventor of Erlang) said of OOP, "you wanted a banana but what you got was a gorilla holding the banana, and the entire jungle". Rust begins to sound like that too. It might not be inherent in the language, but it looks like the way the community thinks.

I also still don't understand why the Bcachefs userspace stuff is written in Rust. I can understand about the kernel part, but the concept of a low level language is manual resource management that a HLL handles for you automatically. Writing the userspace in a LLL seems like more pain for unclear gain. Are there intense performance or memory constraints or what?

Actually I see now that kernel part of Bcachefs is also considered unstable, so maybe the whole thing is not yet ready for production.

[–] boredsquirrel@slrpnk.net 2 points 2 months ago* (last edited 2 months ago) (1 children)

Fedora has this package? I didnt find it, strange

[–] bsergay@discuss.online 7 points 2 months ago (1 children)

Fedora absolutely does have that package.

[–] boredsquirrel@slrpnk.net 1 points 2 months ago (1 children)

Strange, thanks that is nice! The development of bcachefs seems to not be that problematic on rolling or semi-rolling distros.

That guy might just have bad time managent, but filesystem based encryption is really cool.

[–] bsergay@discuss.online 1 points 2 months ago

filesystem based encryption is really cool.

Can't agree more.

[–] merthyr1831@lemmy.ml -1 points 2 months ago

TLDR Debian and the traditional Linux package management system is antiquated and insecure, but somehow this is the fault of one of the many programming languages that is designed around the sensible expectations of being able to manage your own dependencies.

[–] markstos@lemmy.world -2 points 2 months ago

Maybe start over with whole kernel OS in Rust?

https://drewdevault.com/2024/08/30/2024-08-30-Rust-in-Linux-revisited.html