this post was submitted on 03 Sep 2024
1579 points (97.8% liked)
Technology
59569 readers
3825 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Google (used to) scrapes the specific details authorized by robots.txt and uses it to make your content visible.
OpenAI scrapes everything it can technically see, ignoring robots.txt and feeds i to a black box and regurgitates it claiming it’s something new, that it deserves to be paid for.
Quite different actually.
So if OpenAI complies with Robots.txt files then there's no issue right?
Because then they're identical. Open AI spent a bunch of money building a powerful system they feed those results to, as did Google.
Actually Google tries their hardest NOT to point you to content. They scrape the data from sites and display it directly in the search results so that you don't need to visit any site except Google. Their new AI answers that they are pushing on users are just another step in that direction.
Which is why Google is no longer my default browser. I'd be quite happy if it reverted Back to don't be evil or just ceased ro exist