this post was submitted on 26 Feb 2025
935 points (96.3% liked)
Technology
63547 readers
2398 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
While that’s true, the thing that stuck out to me is not even that the AI was mislead by itself finding AI slop, or even somebody falsely asserting something. I googled something with a particular yea or no answer. “Does X technology use Y protocol”. The AI came back with “Yes it does, and here’s how it uses it”, and upon visiting the reference page for that answer, it was documentation for that technology where it explained very clearly that x technology does NOT use Y protocol, and then went into detail on why it doesn’t. So even when everything lines up and the answer is clear and unambiguous, the AI can give you an entirely fabricated answer.
What's really awful is that it seems like they've trained these LLMs to be "helpful", which means to say "yes" as much as possible. But, that's the case even when the true answer is "no".
I was searching for something recently. Most people with similar searches were trying to do X, I was trying to do Y which was different in subtle but important differences. There are tons of resources out there showing how to do X, but none showing how to do Y. The "AI" answer gave me directions for doing Y by showing me the procedure for doing X, with certain parts changed so that they match Y instead. It doesn't work like that.
Like, imagine a recipe that not just uses sugar but that relies on key properties of sugar to work, something like caramel. Search for "how do I make caramel with stevia instead of sugar" and the AI gives you the recipe for making caramel with sugar, just with "stevia" replacing every mention of "sugar" in the original recipe. Absolutely useless, right? The correct answer would be "You can't do that, the properties are just too different." But, an LLM knows nothing, so it is happy just to substitute words in a recipe and be "helpful".