There are a bunch of options available. I think the exact layout depends on the exact use-case. From GlusterFS, Ceph, to (S3 compatible) block storage, to straightforward NFS, to database replication, that's all for different use-cases like VM failover to decoupling storage from a service, to something like a Jellyfin sharing the media library with another service, to horizontal scaling of services... I don't think there is a single answer to all of that.
this post was submitted on 25 Jul 2025
15 points (100.0% liked)
Selfhosted
49824 readers
808 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 2 years ago
MODERATORS
Thanks. I will take a closer look into GlusterFS and Ceph.
The use case would be a file storage for anything (text, documents, images, audio and video files). I'd like to share this data among multiple instances and don't want to store that data multiple times - it is bad for my bank account and I don't want to keep track of the various redundant file sets. So data and service decoupling.
Service scaling isn't a requirement. It's more about different services (some as containers, some as VMs) which should work on the same files, sometimes concurrently.
That jellyfin/arr approach works well and is easy to set up, if all containers access the same docker volume. But it doesn't when VMs (KVM) or other containers (lxc) come into play. So I can't use it in this context.
Failover is nice to have. But there is more to it than just the data replication between hosts. It's not a priority to me right now.
Database replication isn't required.
GlusterFS is (was) really cool, but I would not set up a new instance. It used to have significant support and development from RedHat, but they decided to halt their work on it, and focus on Ceph.
GlusterFS is getting a few slow updates from some alternate developers, but I would only count on that being fixes for current installations.
Just be warned that those two are relatively complicated pieces of tech. And they're meant to set up a distributed storage network including things like replication and load-balancing. Clusters with failover to a different datacenter and such. If you just want access to the same storage on one server from different instances, that's likely way to complicated for you. (And more complexity generally means more maintenance and more failure modes.)
Moot point. I do not really need the distributed storage part for my scenario. Not right now.
Maybe I start with NFS and explore gluster as soon as storage distribution is needed. Looks like it could be a drop-in eplacement for NFSv3. Since it doesn't access the block devices directly, I still could use the respective fs' tool set (I.e. ext4 or btrfs) for maintenance tasks.
VirtioFS. You can share from the host to any number of VMs with that. LibVirtd is good. Even has a nice GUI in virt-manager.
Here is what incus supports. If you have multiple hypervisor hosts then you are talking about remote storage.
https://linuxcontainers.org/incus/docs/main/explanation/storage/
What device are you going to put this shared storage on?
Thanks for asking. I left that detail out. An SSD which is attached to the virtualization host via SATA. I plan to use either a LVM2 volume group or a BTRFS with subvolumes to provide the storage pool to Incus/LXC.
Just use NFS then.