this post was submitted on 21 Feb 2024
289 points (95.0% liked)
Technology
59569 readers
3431 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
To be honest this is the kind of outcome I expected.
Garbage in, garbage out. Making the system more complex doesn't solve that problem.
The development of LLMs is possibly becoming self defeating, because the training data is being filled not just with human garbage, but also AI garbage from previous, cruder LLMs.
We may well end up with a machine learning equivalent of Kessler syndrome, with our pool of available knowledge eventually becoming too full of junk to progress.
I mean, surely the solution to that would be to use curated/vetted training data? Or at the very least, data from before LLMs became commonplace?
The funny thing is, children are similar. They just learn whatever you put in front of them. We have whole systems for educating children for decades of their lives.
With AI we literally just plopped them in front of the Internet, with no guidelines on what to learn. AI researchers say "it's a black box! We don't know why it's doing this!" You fed it everything you could and gave it few rules on what to do. You are the reason why it's nuts.
Humans come hardwired to be a certain way, do certain things. Maybe they need to start AI off like that, some basic programs that guide learning. "Learn everything" isn't working.
That's a good point. For real brains, size and intelligence are not linked. An elephant brain has 3 times the amount of neurons as a human brain, but a human brain is more intelligent. There is more to intelligence than just the amount of neutrons, real or virtual, so making larger and larger AI models may not be the right direction.
True. Maybe they just need more error correction. Like spend more energy questioning whether what you say is true. Right now LLMs seems to just vomit out whatever they thought up, with no consideration of whether it makes sense.
They're like an annoying friend who just can't shut up.
They aren’t thinking though. They’re making connection with the trained data that they’ve processed.
This is really clear when they are asked to write code worth to vague a prompt.
Maybe feeding them through primary school curriculum (including essays and tests) would be helpful, but I don’t think the language models really sort knowledge yet.