Selfhosted

59939 readers

304 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

584

Based on this graph, and this graph alone, guess at what time I completely blocked OpenAI crawlers (jlai.lu)

submitted 4 months ago by hylobates@jlai.lu to c/selfhosted@lemmy.world

80 comments fedilink hide all child comments

I really hope they die soon, this is unbearable…

you are viewing a single comment's thread
view the rest of the comments

[–] quick_snail@feddit.nl 4 points 4 months ago (2 children)

Y'all need to learn to cache things, shiit

[–] poVoq@slrpnk.net 5 points 4 months ago (1 children)

This is not how things work on the modern web. Did you just wake up from a 20 year coma?

[–] FreedomAdvocate@lemmy.net.au 1 points 4 months ago* (last edited 3 months ago) (1 children)

Whenever you’re browsing even a semi popular website these days there’s probably a 98% chance you’re hitting a cloudflare cached version of it. Have you been asleep the last 10 years?

[–] poVoq@slrpnk.net 1 points 3 months ago (1 children)

For static sites, yes. To actually protect dynamic sites against AI crawlers, Cloudflare has to do much more than just caching.

And besides that, Cloudflare is a huge single point of failure and highly privacy invasive.

[–] FreedomAdvocate@lemmy.net.au 1 points 3 months ago

Dynamic sites still get cached.

Cloudflare definitely is a huge single point of failure, and it is a huge problem imo - but what can we do? Their product is so widely used because of how comprehensive, good, and necessary it is.

[–] hylobates@jlai.lu 1 points 4 months ago (1 children)

There is cache but for some reason this does not affect every page I serve currently

[–] quick_snail@feddit.nl 3 points 4 months ago

Fix the cache.