Technology

59963 readers

3330 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 2 years ago

MODERATORS

133

Mapping the Mind of a Large Language Model (www.anthropic.com)

submitted 7 months ago by kromem@lemmy.world to c/technology@lemmy.world

21 comments fedilink hide all child comments

I often see a lot of people with outdated understanding of modern LLMs.

This is probably the best interpretability research to date, by the leading interpretability research team.

It's worth a read if you want a peek behind the curtain on modern models.

you are viewing a single comment's thread
view the rest of the comments

[–] misk@sopuli.xyz 3 points 6 months ago* (last edited 6 months ago) (1 children)

Most people don't know what Bayesian statistics are so you could say most people don't really get how machine learning works in general anyway. It's not misleading though as it perfectly sets expectations on what you're getting as output. It's much more healthy to general understanding of AI than anthropomorphizing very inflexible and limited models achieved thanks to technology that is seemingly in a plateau.

[–] General_Effort@lemmy.world 3 points 6 months ago (1 children)

I would not expect almost human-like conversation on being told that is just statistics. I'd expect something like the old Markov chain jobs. What kind of knowledge leads you to have higher expectations?

Also, how does Bayesian statistics enter into this?

[–] misk@sopuli.xyz 1 points 6 months ago (1 children)

ELIZA from 1966 was enough to convince people that computer program they were talking to was human. People are now being sold on getting answers to their questions via natural language prompts and those answers are pretty much plausibly sounding sentences that happen to be right sometimes due to probability calculations.

Bayesian statistics is very different from what's being taught up until high school (at least here) and is foundational to earlier machine learning applications like spam filters. It's hard to imagine understanding what LLMs do without basics.

[–] General_Effort@lemmy.world 4 points 6 months ago

Those aren't the basics, though. That's how saying it's statistics is misleading. A Bayesian network is not a neural network.