Atemu

joined 4 years ago
MODERATOR OF
[–] Atemu@lemmy.ml 5 points 1 week ago* (last edited 1 week ago)

It only works if the hardware doesn't lie about write barriers. If it says it's written some sectors, btrfs assumes that reading any of those sectors will return the written data rather than the data that was there before. What's important here isn't that the data will forever stay in-tact but ordering. Once a metadata generation has been written to disk, btrfs waits on the write barrier and only updates the superblock (the final metadata "root") afterwards.

If the system loses power while the metadata generation is being written, all is well because the superblock still points at the old generation as the write barrier hasn't passed yet. On the next boot, btrfs will simply continue with the previous generation referenced in the superblock which is fully committed.
If the hardware lied about the write barrier before the superblock update though (i.e. for performance reasons) and has only written e.g. half of the sectors containing the metadata generation but did write the superblock, that would be an inconsistent state which btrfs cannot trivially recover from.

If that promise is broken, there's nothing btrfs (or ZFS for that matter) can do. Software cannot reliably protect against this failure mode.
You could mitigate it by waiting some amount of time which would reduce (but not eliminate) the risk of the data before the barrier not being written yet but that would also make every commit take that much longer which would kill performance.

It can reliably protect against power loss (bugs not withstanding) but only if the hardware doesn't lie about some basic guarantees.

[–] Atemu@lemmy.ml 3 points 1 week ago* (last edited 1 week ago) (1 children)

I am well aware of how CoW works. What I wrote does not stand in conflict with it.

Perhaps I wasn't clear enough in what I said though:

Each metadata operation ("commit" I think it's called) has a generation number; it first builds this generation (efficiently in a non-damaging way via CoW) and then atomically switches to it. The next generation is built with an incremented generation number and atomically switched again.
That's my understanding of how btrfs generally operates.

When things go awry, some sector that holds some of the newest generation may be corrupt but it might be that a relatively recent generation does not contain this data and is therefore unaffected.

What I'm suggesting is that you should be able to roll back to such a generation at the cost of the changes which happened in between in order to restore a usable filesystem. For this to be feasible, btrfs would need to take greater care not to overwrite recent generation data though which is what I meant by making it "more log-structured".

I don't know whether any of this is realistically doable though; my knowledge of btrfs isn't enough to ascertain this.

[–] Atemu@lemmy.ml 1 points 1 week ago

The complex part of Guix comes from it being a inmutable distribution based on the ideas from NixOS

That's not the most significant factor in what makes it hard/different. There are immutable distros that come with less complexity and are arguably more immutable than NixOS or guix.

What actually sets it apart and can make these harder to use is:

  1. They're declarative rather than imperative. You describe the desired end-result rather than providing (or manually executing) the steps to construct it.
  2. There is not a single global dependency dependency state upon parts of which any given package depends. Dependencies are explicit, direct and encapsulated on a level that's as fine as you'd like instead (down to the per-file level).
[–] Atemu@lemmy.ml 24 points 1 week ago (9 children)

could be due to some bad block or read failure from the hdd (it is supposedly brand new, but i guess it could be busted)

I'd suspect the controller or cable first.

shit like this never happens to me on extfs, even if the hdd is literally dying

You say that as if it's a good thing. If you HDD is "literally dying", you want the filesystem to fail safe to make you (and applications) aware and not continue as if nothing happened. extfs doesn't fail here because it cannot even detect that something is wrong.

btrfs has its own share of bugs but, in theory, this is actually a feature.

i have an ext4 and an ufs partition in the same hdd without any issues.

Not any issue that you know of. For all extfs (and, by extension, you) knows, the disk/cable/controller/whatever could have mangled your most precious files and it would be none the wiser; happily passing mangled data to applications.

You have backups of course (right?), so that's not an issue you might say but if the filesystem isn't integer, that can permeate to your backups because the backup tool reading those files is none the wiser too; it relies on the filesystem to return the correct data. If you don't manually verify each and every file on a higher level (e.g. manual inspection or hashing) and prune old backups, this has potential for actual data loss.

If your hardware isn't handling the storage of data as it should, you want to know.

even if we suppose this is the hardware's fault and not btrfs's, should a file system be a little bit more resilient than that? at this rate, i feel like a cosmic ray could set off a btrfs corruption.

While the behaviour upon encountering an issue is in theory correct, btrfs is quite fragile. Hardware issues shouldn't happen but when they happen, you're quite doomed because btrfs doesn't have the option to continue despite the integrity of a part of it being compromised.
btrfs-restore disables btrfs' integrity; emulating extfs's failure mode but it's only for extracting files from the raw disks, not for continuing to use it as a filesystem.

I don't know enough about btrfs to know whether this is feasible but perhaps it could be made a bit more log-structured such that old data is overwritten first which would allow you to simply roll back the filesystem state to a wide range of previous generations, of which some are hopefully not corrupted. You'd then discard the newer generations which would allow you to keep using the filesystem.
You'd risk losing data that was written since that generation of course but that's often a much lesser evil. This isn't applicable to all kinds of corruption because older generations can become corrupted retroactively of course but at least a good amount of them I suspect.

[–] Atemu@lemmy.ml 6 points 1 week ago* (last edited 1 week ago)

Linus didn't only call out people posting flame replies, but also folks interested in a serious discussion on that topic, who also contributed to the kernel before (see PeterCxy's blog: https://typeblog.net/55833/getting-called-paid-actor-by-linus-torvalds).

Here's what he actually said:

I'm also not going to start discussing legal issues with random internet people who I seriously suspect are paid actors and/or have been riled up by them.

That's ambiguous at best.

I find it more likely that it's targetted at the greater crowd replying to that thread rather than this person specifically due to it's general tone. If Linus thinks you are full of shit, you can be that he will call you out specifically rather than wording it like this.

AFAIK we still don’t know the exact reasons for the removal, which is just intransparent.

We do: https://www.phoronix.com/news/Linux-Compliance-Requirements.

[–] Atemu@lemmy.ml 8 points 1 week ago (2 children)

where he called several kernel developers paid actors

He didn't. He (rightly) called out people who have never contributed to the kernel and posted flame replies to the initial announcement on the mailing list.

This isn't some wild conspiracy theory either, this is a thing Russia actually does: cause disarray and split "western" communities; divide and conquer. It's an their explicit goal of the Kremlin to do this; there's credible accounts of that.

If you want a community that's resistant to such influences, you ought to call this sort of thing out whenever you see it.

[–] Atemu@lemmy.ml 23 points 1 week ago (1 children)

AND have backups.

[–] Atemu@lemmy.ml 4 points 2 weeks ago (4 children)

NixOS because it's the only usable stab at sustainable system configuration.

[–] Atemu@lemmy.ml 1 points 2 weeks ago

I recommend borgbackup.

[–] Atemu@lemmy.ml 4 points 2 weeks ago

It has since taken away Gentoo’s raison d’être a bit in my head.

I wouldn't say so. We currently don't hold a candle to USE-flags. Many packages are already configurable but there's no standard on anything w.r.t. that.

There's no technical reason we couldn't have such a standard but it hasn't happened yet.

[–] Atemu@lemmy.ml 2 points 2 weeks ago

That's Nix, not NixOS.

I also wouldn't be too sure on that "explicit" part for Docker. It's somewhat isolated, sure, but everything but explicit: you can download arbitrary data from wherever you like.

[–] Atemu@lemmy.ml 0 points 2 weeks ago

That isn't going to help them one bit if they have a SteamVR HMD.

 

Due to the lack of innovation, there wasn't really a community where discussion and news about alternative search engines would fit, so I created one.

I was introduced to Kagi by an IRL friend of mine and was initially veeery sceptical of a paid search engine. You can probably relate. Fast forward a few months, I gave it a trial and was very pleasantly surprised. So much so that I'm now a subscriber and use it on all my devices.

What's your experience like?

 

cross-posted from: https://lemmy.ml/post/11150038

I'm trying out Actual and have imported my bank's (Sparkasse) data for my checking account via CSV. In the CSV import, I obviously had to set the correct fields and was a bit confused because Actual only has the "Payee" field while my CSVs have IBAN, BIC and a free text name (i.e. "Employer GmbH".)

IBAN is preferable because it's a unique ID while the free text name can be empty or possibly even change(?). (Don't know how that works.)
OTOH, the free text name is preferable because I (as a human) can use it to infer the actual payee while the IBANs are just a bunch of numbers.

Is it possible to use IBAN aswell as the free text name or have a mapping between IBAN and a display name?

How do you handle that?

 

I assume many of you host a DMS such as Paperless and use it to organise the dead trees you still receive in the snail mail for some reason in the year of the lord 2023.

How do you encode your scans? JPEG is pretty meh for text even at better quantisation levels ("dirty" artefacts everywhere) and PNGs are quite large. More modern formats don't go into a PDF, which means multiple pages aren't possible (at least not in Paperless).

Discussion on GH: https://github.com/paperless-ngx/paperless-ngx/discussions/3756

view more: ‹ prev next ›