Technology

59605 readers

3403 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

For Data-Guzzling AI Companies, the Internet Is Too Small (tech.slashdot.org)

submitted 7 months ago by 0nekoneko7@lemmy.world to c/technology@lemmy.world

12 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] elshandra@lemmy.world 13 points 7 months ago* (last edited 7 months ago) (2 children)

Idk, I find this hard to believe. I would think the challenge is more access to the information (gates, bandwidth), a speedy vault to store that information, and improving their models.

When you think about what's available on the internet, how much of human knowledge and propaganda is out there. With enough/deus ex tech, there's no way ai shouldn't be able to learn most of anything with the knowledge available, and the right trainers.

[–] General_Effort@lemmy.world 2 points 7 months ago

Yes, it's BS, like most of the AI takes here.

The kernel of truth is scaling laws:

[T]he Chinchilla scaling law for training Transformer language models suggests that when given an increased budget (in FLOPs), to achieve compute-optimal, the number of model parameters (N) and the number of tokens for training the model (D) should scale in approximately equal proportions.

[–] isles@lemmy.world 1 points 7 months ago (1 children)

and propaganda

Well, that's the rub, right? Garbage in, garbage out. For an LLM, the value is predicting the next token, but we've seen how racist current datasets can be. If you filter it, there's not as much lot of high quality data left.

So yes, we have a remarkable amount of (often wrong) information to pull from.

[–] elshandra@lemmy.world 2 points 7 months ago

Mhm, I wonder when we'll have the resources to build one that can tell the truth from other lies. I suppose you have to learn to crawl before you learn to walk, but these things still having trouble rolling over.