this post was submitted on 09 Jun 2025
820 points (91.8% liked)

Technology

71492 readers
4789 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
top 50 comments
sorted by: hot top controversial new old
[–] NeilBru@lemmy.world 76 points 6 days ago* (last edited 6 days ago) (4 children)

An LLM is a poor computational/predictive paradigm for playing chess.

[–] surph_ninja@lemmy.world 30 points 6 days ago (1 children)

This just in: a hammer makes a poor screwdriver.

[–] WhyJiffie@sh.itjust.works 8 points 6 days ago

LLMs are more like a leaf blower though

[–] Takapapatapaka@lemmy.world 12 points 6 days ago (1 children)

Actually, a very specific model (chatgpt3.5-turbo-instruct) was pretty good at chess (around 1700 elo if i remember correctly).

[–] NeilBru@lemmy.world 3 points 6 days ago (3 children)

I'm impressed, if that's true! In general, an LLM's training cost vs. an LSTM, RNN, or some other more appropriate DNN algorithm suitable for the ruleset is laughably high.

load more comments (3 replies)
[–] Bleys@lemmy.world 5 points 6 days ago (1 children)

The underlying neural network tech is the same as what the best chess AIs (AlphaZero, Leela) use. The problem is, as you said, that ChatGPT is designed specifically as an LLM so it’s been optimized strictly to write semi-coherent text first, and then any problem solving beyond that is ancillary. Which should say a lot about how inconsistent ChatGPT is at solving problems, given that it’s not actually optimized for any specific use cases.

[–] NeilBru@lemmy.world 3 points 6 days ago* (last edited 6 days ago)

Yes, I agree wholeheartedly with your clarification.

My career path, as I stated in a different comment in regards to neural networks, is focused on generative DNNs for CAD applications and parametric 3D modeling. Before that, I began as a researcher in cancerous tissue classification and object detection in medical diagnostic imaging.

Thus, large language models are well out of my area of expertise in terms of the architecture of their models.

However, fundamentally it boils down to the fact that the specific large language model used was designed to predict text and not necessarily solve problems/play games to "win"/"survive".

(I admit that I'm just parroting what you stated and maybe rehashing what I stated even before that, but I like repeating and refining in simple terms to practice explaining to laymen and, dare I say, clients. It helps me feel as if I don't come off too pompously when talking about this subject to others; forgive my tedium.)

[–] sugar_in_your_tea@sh.itjust.works 3 points 6 days ago* (last edited 6 days ago)

Yeah, a lot of them hallucinate illegal moves.

[–] nednobbins@lemm.ee 50 points 6 days ago (2 children)

Sometimes it seems like most of these AI articles are written by AIs with bad prompts.

Human journalists would hopefully do a little research. A quick search would reveal that researches have been publishing about this for over a year so there's no need to sensationalize it. Perhaps the human journalist could have spent a little time talking about why LLMs are bad at chess and how researchers are approaching the problem.

LLMs on the other hand, are very good at producing clickbait articles with low information content.

[–] nova_ad_vitum@lemmy.ca 24 points 6 days ago (6 children)

Gotham chess has a video of making chatgpt play chess against stockfish. Spoiler: chatgpt does not do well. It plays okay for a few moves but then the moment it gets in trouble it straight up cheats. Telling it to follow the rules of chess doesn't help.

This sort of gets to the heart of LLM-based "AI". That one example to me really shows that there's no actual reasoning happening inside. It's producing answers that statistically look like answers that might be given based on that input.

For some things it even works. But calling this intelligence is dubious at best.

[–] Ultraviolet@lemmy.world 6 points 6 days ago* (last edited 6 days ago)

Because it doesn't have any understanding of the rules of chess or even an internal model of the game state, it just has the text of chess games in its training data and can reproduce the notation, but nothing to prevent it from making illegal moves, trying to move or capture pieces that don't exist, incorrectly declaring check/checkmate, or any number of nonsensical things.

[–] Noodle07@lemmy.world 4 points 6 days ago

Hallucinating 100% of the time 👌

[–] JacksonLamb@lemmy.world 3 points 6 days ago (1 children)

ChatGPT versus Deepseek is hilarious. They both cheat like crazy and then one side jedi mind tricks the winner into losing.

load more comments (1 replies)
load more comments (3 replies)
[–] LovableSidekick@lemmy.world 7 points 6 days ago* (last edited 6 days ago) (3 children)

In this case it's not even bad prompts, it's a problem domain ChatGPT wasn't designed to be good at. It's like saying modern medicine is clearly bullshit because a doctor loses a basketball game.

load more comments (3 replies)
[–] Halosheep@lemm.ee 43 points 6 days ago (3 children)

I swear every single article critical of current LLMs is like, "The square got BLASTED by the triangle shape when it completely FAILED to go through the triangle shaped hole."

[–] drspod@lemmy.ml 42 points 6 days ago (3 children)

It's newsworthy when the sellers of squares are saying that nobody will ever need a triangle again, and the shape-sector of the stock market is hysterically pumping money into companies that make or use squares.

[–] inconel@lemmy.ca 19 points 6 days ago (1 children)

It's also from a company claiming they're getting closer to create morphing shape that can match any hole.

And yet the company offers no explanation for how, exactly, they're going to get wood to do that.

[–] MrSqueezles@lemmy.world 6 points 6 days ago

The press release where OpenAI said we'd never need chess players again

[–] PushButton@lemmy.world 6 points 6 days ago (1 children)

You get 2 triangles in a single square mate...

CHECKMATE!

load more comments (1 replies)
load more comments (2 replies)
[–] cley_faye@lemmy.world 25 points 6 days ago

Ah, you used logic. That's the issue. They don't do that.

[–] stevedice@sh.itjust.works 8 points 5 days ago

2025 Mazda MX-5 Miata 'got absolutely wrecked' by Inflatable Boat in beginner's boat racing match — Mazda's newest model bamboozled by 1930s technology.

[–] arc99@lemmy.world 20 points 6 days ago (1 children)

Hardly surprising. Llms aren't -thinking- they're just shitting out the next token for any given input of tokens.

[–] stevedice@sh.itjust.works 1 points 5 days ago (1 children)

That's exactly what thinking is, though.

[–] arc99@lemmy.world 1 points 4 days ago* (last edited 4 days ago) (1 children)

An LLM is an ordered series of parameterized / weighted nodes which are fed a bunch of tokens, and millions of calculations later result generates the next token to append and repeat the process. It's like turning a handle on some complex Babbage-esque machine. LLMs use a tiny bit of randomness ("temperature") when choosing the next token so the responses are not identical each time.

But it is not thinking. Not even remotely so. It's a simulacrum. If you want to see this, run ollama with the temperature set to 0 e.g.

ollama run gemma3:4b
>>> /set parameter temperature 0
>>> what is a leaf

You will get the same answer every single time.

[–] stevedice@sh.itjust.works 1 points 7 hours ago* (last edited 7 hours ago)

I know what an LLM is doing. You don't know what your brain is doing.

[–] jsomae@lemmy.ml 13 points 6 days ago (1 children)

Using an LLM as a chess engine is like using a power tool as a table leg. Pretty funny honestly, but it's obviously not going to be good at it, at least not without scaffolding.

[–] kent_eh@lemmy.ca 3 points 5 days ago (1 children)

is like using a power tool as a table leg.

Then again, our corporate lords and masters are trying to replace all manner of skilled workers with those same LLM "AI" tools.

And clearly that will backfire on them and they'll eventually scramble to find people with the needed skills, but in the meantime tons of people will have lost their source of income.

[–] jsomae@lemmy.ml 1 points 5 days ago* (last edited 5 days ago) (1 children)

If you believe LLMs are not good at anything then there should be relatively little to worry about in the long-term, but I am more concerned.

It's not obvious to me that it will backfire for them, because I believe LLMs are good at some things (that is, when they are used correctly, for the correct tasks). Currently they're being applied to far more use cases than they are likely to be good at -- either because they're overhyped or our corporate lords and masters are just experimenting to find out what they're good at and what not. Some of these cases will be like chess, but others will be like code*.

(* not saying LLMs are good at code in general, but for some coding applications I believe they are vastly more efficient than humans, even if a human expert can currently write higher-quality less-buggy code.)

[–] kent_eh@lemmy.ca 1 points 4 days ago (1 children)

I believe LLMs are good at some things

The problem is that they're being used for all the things, including a large number of tasks that thwy are not well suited to.

[–] jsomae@lemmy.ml 2 points 4 days ago

yeah, we agree on this point. In the short term it's a disaster. In the long-term, assuming AI's capabilities don't continue to improve at the rate they have been, our corporate overlords will only replace people for whom it's actually worth it to them to replace with AI.

[–] finitebanjo@lemmy.world 15 points 6 days ago

All these comments asking "why don't they just have chatgpt go and look up the correct answer".

That's not how it works, you buffoons, it trains off of datasets long before it releases. It doesn't think. It doesn't learn after release, it won't remember things you try to teach it.

Really lowering my faith in humanity when even the AI skeptics don't understand that it generates statistical representations of an answer based on answers given in the past.

[–] FourWaveforms@lemm.ee 5 points 5 days ago

If you don't play chess, the Atari is probably going to beat you as well.

LLMs are only good at things to the extent that they have been well-trained in the relevant areas. Not just learning to predict text string sequences, but reinforcement learning after that, where a human or some other agent says "this answer is better than that one" enough times in enough of the right contexts. It mimics the way humans learn, which is through repeated and diverse exposure.

If they set up a system to train it against some chess program, or (much simpler) simply gave it a tool call, it would do much better. Tool calling already exists and would be by far the easiest way.

It could also be instructed to write a chess solver program and then run it, at which point it would be on par with the Atari, but it wouldn't compete well with a serious chess solver.

[–] Sidhean@lemmy.blahaj.zone 10 points 6 days ago

Can i fistfight ChatGPT next? I bet I could kick its ass, too :p

this is because an LLM is not made for playing chess

load more comments
view more: next ›