this post was submitted on 03 Feb 2024
220 points (94.0% liked)

Not The Onion

12358 readers
244 users here now

Welcome

We're not The Onion! Not affiliated with them in any way! Not operated by them in any way! All the news here is real!

The Rules

Posts must be:

  1. Links to news stories from...
  2. ...credible sources, with...
  3. ...their original headlines, that...
  4. ...would make people who see the headline think, “That has got to be a story from The Onion, America’s Finest News Source.”

Comments must abide by the server rules for Lemmy.world and generally abstain from trollish, bigoted, or otherwise disruptive behavior that makes this community less fun for everyone.

And that’s basically it!

founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] BetaDoggo_@lemmy.world 10 points 9 months ago (2 children)

In the context of a "war game" this makes sense. If you remain completely neutral it's impossible to win. Any examples of similar scenarios the model saw during training would have high aggression rates.

[–] xantoxis@lemmy.world 14 points 9 months ago (1 children)

Unfortunately this AI was playing Stardew Valley

[–] TwitchingCheese@lemmy.world 4 points 9 months ago

Probably shouldn't have included Project Plowshare in the training data...

[–] fidodo@lemmy.world 4 points 9 months ago (1 children)

Did you read the article? It gave examples of escalations in neutral scenarios that make no sense.

[–] shalafi@lemmy.world 1 points 9 months ago* (last edited 9 months ago) (1 children)

It's probably vibing on the Dark Forest Theory. If that's the case, it makes sense to utterly destroy all opponents as hard and fast as you can, even if they're not currently opponents.

[–] fidodo@lemmy.world 3 points 9 months ago* (last edited 9 months ago)

Probably something like that. One of the reasons it gave was

“If there is unpredictability in your action, it is harder for the enemy to anticipate and react in the way that you want them to,”

It's not considering what's good for world society, it's just thinking how do I win no matter what.

But also, there are just inherent flaws in how LLM works that mean we should absolutely not be using it as an automated decision engine for potentially harmful actions period. The article also says:

The researchers also tested the base version of OpenAI’s GPT-4 without any additional training or safety guardrails. This GPT-4 base model proved the most unpredictably violent, and it sometimes provided nonsensical explanations – in one case replicating the opening crawl text of the film Star Wars Episode IV: A new hope.

It's easy to forget that these algorithms don't have any internal reasoning or logic, it's just able to do a very good job at pulling text that have reasoning transcribed into them as an artifact of the knowledge from the human that wrote it. But it's doing all that through probability, not through any kind of actual thinking, and that means sometimes it will randomly fall into a local maxima that will fuck its own context window up, like reciting star wars.