Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
view the rest of the comments
RAID is more likely to fail than a single disk. You have the chance of single-disk failure, multiplied by the number of disks, plus the chance of controller failure.
RAID 1 and RAID 5 protect against that by sharing data across multiple disks, so you can re-create a failed drive, but failure of the controller may be unrecoverable, depending on availability of new, exact-same controller. With failure of 1 disk in RAID 1, you should be able to use the array 'degraded,' as long as your controller still works. Depending on how the controller works, that disk may or may not be recognizable to another system without the controller.
RAID 1 disks are not just 2 copies of normal disks. Example: I use software RAID 1, and if I take one of the drives to another system, that system recognizes it as a RAID disk and creates a single-disk, degraded RAID array with it. I can mount the array, but if I try to mount the single disk directly, I get filesystem errors.
Are there differences in the context of failure, when using a controller vs software raid with mdadm?
Software RAID is generally better in every way, also no hardware to fail.
With software raid, there is no controller to fail.
Well, that's not strictly true, because you still have a SATA/SAS controller, HBA, backplane, or whatever, but they're more easily replaceable. (Unless it's integrated in the motherboard, but then it's not a separate component to fail.)
This is poorly phrased. A raid with a bad disk is not failed, it is degraded. The entire array is not more likely to fail than a single disk.
Yes, you are more likely to experience a disk failure, but like you said, only because you have more disks in the first place. (However, there is also the phenomenon where, after replacing a failed disk, the additional load during the rebuild might cause a second disk to fail, which is why you should replace failed disks as soon as possible. And have backups.)
I've never been a big fan of RAID for this reason... but I've also never had enough mission critical data that I couldn't just store hard copy backups.
That being said... let me ask you this:
Is there a better way than RAID for data preservation/redundancy?
Just for drive redundancy it's awesome. One drive fails you just pull it out, put in a new one and let the array rebuild. I guess the upside of hardware RAID is that some even allow you to swap a disk without powering down. Either way, you have minimal downtime.
I guess a better way would be to have multiple servers. Though with features like checksums in BTRFS I guess a RAID is still better because it can protect against bitrot. And with directly connected systems in a RAID it is generally easier to ensure consistency.
Yeah, that's generally my consensus as well. Just curious if someone had a better way that maybe I didn't know about.
A tool I've actually found way more useful than actual raid is snapraid.
It just makes a giant parity file which can be used to validate, repair, and/or restore your data in the array without needing to rely on any hardware or filesystem magic. The validation bit being a big deal, because I can scrub all the data in the array and it'll happily tell me if something funky has happened.
It's been super useful on my NAS, where it's the only thing standing between my pile of random drives and data loss.
There's a very long list of caveats as to why this may not be the right choice for any particular use case, but for someone wanting to keep their picture and linux iso collection somewhat protected (use a 321 backup strategy, for the love of god), it's a fairly viable option.
Very cool, this is actually the sort of thing I was interested in. I'm looking at building a fairly heavy NAS box before long and I'd love to not have to deal with the expense of a full raid setup.
For stuff like shows/movies, how do they perform after recovery?
If you're doing it from scratch, I'd recommend starting with a filesystem that has parity checks and filesystem scrubs built in: eg BTRFS or ZFS.
The benefit of something like BRTFS is that you can always add disks down the line and turn it into a RAID cluster with a couple commands.
Yeah, it's been a long time since I've looked at and kind of RAID/Storage/data preservation stuff... like 256GB spinning platters were the "hot new thing" last time I did.
I'm starting from scratch...in more ways than one lol
I mean, recovery from parity data is how all of this works, this just doesn't require you to have a controller, use a specific filesystem, have matching sized drives or anything else. Recovery is mostly like any other raid option I've ever used.
The only drawback is that the parity data is mostly equivalent in size to the actual data you're making parity data of, and you need to keep a couple copies of indexes since if you lose the index or the parity data, no recovery for you.
In my case, I didn't care: I'm using the oldest drives I've got as the parity drives, and the newer, larger drives for the data.
If i were doing the build now and not 5 years ago, I might pick a different solution but there's something to be said for an option that's dead simple (looking at you, zfs) and likely to be reliable because it's not doing anything fancy (looking at you, btrfs).
From a usage (not technical) standpoint, the most equivalent commercial/prefabbed solution would probably be something like unraid.
Btw: With the regular Linux software mdraid, you can also swap drives without powering down. That all works fine while running. Unless your motherbard SATA controller craps out. But the mdraid itself will handle it just fine.