I mean, I'd say that all instances copying media by default, to be stored forever, is kind of unnecessary. (And as far as I'm aware Mastodon is the only one configured like this by default anyway)

The largest instances? Sure. I'd say they have an obligation to not DoS smaller instances by simply hotlinking or proxying without any kind of cache. But smaller ones can get away with short lived middleware-level caches, and single user ones can often get away with hotlinking (oh boo hoo your firewalled IPv4 behind enough CGNATs to block any incoming connections got exposed)

One idea I’ve seen floated around is to have some sort of cooperative CDN for instances. Let’s say four or five relatively kindred instances, make a commitment to last and pool their resources to maintain a joint CDN from from which they’ll get their “media federation” from. This would reduce costs and issues a lot, since by the very nature of the fediverse, if everyone builds their own caches most of those caches are going to be hosting most of the same content. Basically: deduplication, but the poor man’s version.

https://jortage.com/ already exists, and the code behind is open.

[–] lambalicious@lemmy.sdf.org 1 points 3 weeks ago* (last edited 3 weeks ago)

Interesting! Didn't know that was what it was for. I always thought it was merely a storage backend.

Any metrics on how many instances are using it and how much deduplication is it doing? EDIT: I see the numbers on their page, I was wondering more about people or instances using their own copy of it, since it's open source.