This is exactly what I'm about to do (later this week when I visit their house)
I've been using syncthing for years, but any tips for the encryption?
I was going to use SendOnly at my end to ensure that the data at the other end is an exact mirror, but in that case, how would the restore work if it's all encrypted?
It varies of course, but most of my torrents are movies and linux ISOs (for real)
I seed any Movies I leech at a 2:1 ratio... most are leeched from Europe, but I've had them from Canada, South America, Asia, but weirdly not many from North America.
I like to give back more to the Linux community, so I'm constantly seeding Arch & Mint ISOs (as that's just what I'm using... maybe something Raspberry-ish) - they go everywhere.
I had a weird instance once where the same Chinese IP address was constantly re-downloading the same ISO. Could've been a VPN endpoint, but after I'd shared something like 40:1 there, I started using GeoIP to block it and similar regions I was uncomfortable with... so the world's becoming smaller for me.
So, I'm a little behind the times here... should I be moving away from torrent search sites to this new fangled spot stuff?
To be fair, the link's just to git comments, so the headline captures the main point.
+1 for this.
You need to see all the data flowing through a sensor to be able to map it, so a router / firewall is often the central point.
I run it as an addon for pfSense and it'll show me all sorts of info.
If you setup the GeoIP you can see which countries your network's connecting too... interesting for torrents...
There's BeyondCompare and Meld if you want a GUI, but, if I understand this correctly, rmlint and fdupes might be helpful here
I've done similar in the past - I prefer commandline for this...
What I'd do is create a "final destination" folder on the 4TB drive and then other working folders for each hdd / cd / dvd that you're working through
Ie
/mnt/4TB/finaldestination /mnt/4TB/source1 /mnt/4TB/source2 ...
Obviously finaldestination is empty to start with so it could just be a direct copy of your first hdd - so make that the largest drive.
(I'm saying copy here, presuming you want to keep the old drives for now, just in case you accidentally delete the wrong stuff on the 4TB drive)
Maybe clean up any obvious stuff
Remove that first drive
Mount the next and copy the data to /mnt/4TB/source2
Now use rmlint or fdupes and do a dry-run between source2 and finaldestination and get a feel whether they're similar or not, so then you'll know whether to just move it all to finaldestination or maybe then use the gui tools.
You might completely empty /mnt4TB/source2, or it might still have something in, depends on how you feel it's going.
Repeat for the rest, working on smaller & smaller drives, comparing with the finaldestination first and then moving the data.
Slow? Yep. Satisfying that you know there's only 1 version there? Yep.
Then do a backup 😉
My choice is Arch Linux purely because it's bleeding edge
I've no idea if Arch actually has newer drivers than Debian / Fedora, but if they are you'll (usually) get better support from the developers of whatever application / package - or in your case - drivers that you're facing.
It's more involved than "just" installing Debian, etc... but reading through the Arch Linux wiki as you install will (should) ensure you've got the correct drivers setup and you'll know why they're working.
So... it'll be more effort, but you might get "better" results.
Crowdsec will block external, public, IPs
Fail2Ban will block login attempts (ie from anywhere)
I have a similar setup with pfSense, pfBlockerNG, HAProxy, etc, but I keep F2B running on my DMZ server in case something is ever compromised as it'll block / slow down anyone trying to move around the network.
Maybe, I'm self hosting several trees, so... 🤷♂️
Depends on the variety of course... my Willows need cutting down, whereas the acorns I planted before the willows are still tiny oaks in pots 🙂
(Sorry, couldn't resist)
Just a friendly reminder that RAID is not a backup...
Just consider if something accidentally overwrites some / all your files. This is a perfectly legit action and the checksums will happily match that new data, but your file(s) are gone...