this post was submitted on 22 Jun 2026
38 points (91.3% liked)

Selfhosted

60074 readers
715 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam.

  3. Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.

  4. Don't duplicate the full text of your blog or git here. Just post the link for folks to click.

  5. Submission headline should match the article title.

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
ayyy@sh.itjust.works
curbstickle@anarchist.nexus
curbstickle_lw@lemmy.world
38
Are ISPs responsible for bots having residential IPs or is this a user problem? (lemmy.world)
submitted 18 hours ago by Maroon@lemmy.world to c/selfhosted@lemmy.world
33 comments fedilink hide all child comments
 

I'm trying to understand the bot problem in the internet and finding more ways to defend myself. One thing that I can't seem to understand is why most bots, scrapers and crawlers seem to have residential IPs.

  • Is it that ISPs are being paid by tech-bros to assign them these IPs?
  • Is it that residential devices have been hacked /contain malware that does this?
  • Is it trivial for companies to assign themselves residential IPs?
  • Paid volunteers are doing this for AI companies?

Or is there is some other reason for this?

Obviously this is a problem because one can rotate / cycle through residential IPs and if I aggressively block each offender in my logs permanently, then the next person assigned this IP who may be a legitimate user will be unable to access my site.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] irmadlad@lemmy.world 7 points 16 hours ago (1 children)

I have blocked much of the Digital Ocean IP space

Over the years, I have blocked the entire probably Digital Ocean spectrum, a ton of Linode, clients.your-server.de, a long string of them. It used to make me paranoid I was setting up something wrong. I'd fire up a brand new, untouched server, and boom! Here they'd come. Then I thought, maybe it was a few Docker containers pushing analytics to a collector. Upon blocking them tho, nothing seemed to malfunction. So instead of trying mitigate each and every offending IP, I switched to a deny all until something complains posture, and block by ip ranges. I can still see them trying to gain access, but as long as I can keep them out on the edge, everything is golden.

Some ISPs basically advertise to criminals about their ability to evade take down orders and unwillingness to work with law enforcement.

There are Russian ISPs that advertise VPS services and such by the hour. I'm sure there are other ISPs in other countries that do the same. Nothing good could be coming out of that. When you consider that the internet gobbles through 14,000,000+/- petabytes per 24 hours, and 40%+/- is bot traffic, the picture gets a bit clearer, because that's a shit load of bots, and they are sophisticated bots at that.

[โ€“] Cyber@feddit.uk 3 points 12 hours ago

Yeah, a long time ago I started blocking outbound traffic to all but our laptops and it's interesting how many things work perfectly well, but continually try to get out... the SolarPv inverter is VERY persistent to get to China due to an overly helpful installer setting it up on the wifi.

I now have a firewall alias to only allow certain devices out and then just update that alias, do whatever (usually updates) and then revert the alias.