this post was submitted on 09 Mar 2024
99 points (89.6% liked)

Technology

59534 readers
3195 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago
MODERATORS
L3s@lemmy.world
L3s@fry.gs
L4s@fry.gs
L4s@lemmy.world
enu@lemmy.world
99
How Quickly Do Large Language Models Learn Unexpected Skills? (nautil.us)
submitted 8 months ago by dominiquec@lemmy.world to c/technology@lemmy.world
19 comments fedilink hide all child comments
 

So-called "emergent" behavior in LLMs may not be the breakthrough that researchers think.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] UraniumBlazer@lemm.ee 52 points 8 months ago (3 children)

TLDR: Let's say you want to teach an LLM a new skill. You give them training data pertaining to that skill. Currently, researchers believe that this skill development shows up suddenly in a breakthrough fashion. They think so because they measure this skill using some methods. The skill levels remain very low until they unpredictably jump up like crazy. This is the "breakthrough".

BUT, the paper that this article references points at flaws in the methods of measuring skills. This paper suggests that breakthrough behavior doesn't really exist and skill development is actually quite predictable.

Also, uhhh I'm not AI (I see that TLDR bot lurking everywhere, which is what made me specify this).

[โ€“] Venator@lemmy.nz 16 points 8 months ago (1 children)

Also, uhhh I'm not AI

An AI would say that... ๐Ÿ˜‚

[โ€“] gravitas_deficiency@sh.itjust.works 4 points 8 months ago

Clearly, the AI is learning deception

[โ€“] inspxtr@lemmy.world 8 points 8 months ago (2 children)

re: your last point, AFAIK, the TLDR bot is also not AI or LLM; it uses more classical NLP methods for summarization.

[โ€“] far_university1990@feddit.de 3 points 8 months ago

https://github.com/RikudouSage/LemmyAutoTldrBot readme say summarization is in summarizer.py which use sumy, specifically LSA which documented here

[โ€“] dirtySourdough@lemmy.world 1 points 8 months ago

Natural language processing falls under AI though, and so do large language models (see chapters 23 and 24 of Russell and Norvig, 2021 http://aima.cs.berkeley.edu/).

[โ€“] dust_accelerator@discuss.tchncs.de 7 points 8 months ago (1 children)

Also, uhhh I'm not AI

That's exactly what an AI would say that got an emergent skill to lie

๐Ÿคฅ

[โ€“] XTL@sopuli.xyz 4 points 8 months ago

Or a model that picked up on a pattern of sources saying that.