Linux

There's BeyondCompare and Meld if you want a GUI, but, if I understand this correctly, rmlint and fdupes might be helpful here

I've done similar in the past - I prefer commandline for this...

What I'd do is create a "final destination" folder on the 4TB drive and then other working folders for each hdd / cd / dvd that you're working through

Ie

/mnt/4TB/finaldestination /mnt/4TB/source1 /mnt/4TB/source2 ...

Obviously finaldestination is empty to start with so it could just be a direct copy of your first hdd - so make that the largest drive.

(I'm saying copy here, presuming you want to keep the old drives for now, just in case you accidentally delete the wrong stuff on the 4TB drive)

Maybe clean up any obvious stuff

Remove that first drive

Mount the next and copy the data to /mnt/4TB/source2

Now use rmlint or fdupes and do a dry-run between source2 and finaldestination and get a feel whether they're similar or not, so then you'll know whether to just move it all to finaldestination or maybe then use the gui tools.

You might completely empty /mnt4TB/source2, or it might still have something in, depends on how you feel it's going.

Repeat for the rest, working on smaller & smaller drives, comparing with the finaldestination first and then moving the data.

Slow? Yep. Satisfying that you know there's only 1 version there? Yep.

Then do a backup 😉