okay how many of these "delusional" people in the study are making fun of the LLM tho
i don't know because I don't use the LLM i only see the screenshots. I am the control group. kinda. my nut is already off.
this post was submitted on 27 Mar 2026
244 points (96.6% liked)
Technology
83126 readers
3716 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
I think what we're seeing is similar to lactose intolerance. Most people can handle it just fine but some people simply can't digest it and get sick. The problem is there's no way to determine who can handle AI and who can't.
When I'm reading about people developing AI delusions their experiences sound completely alien to me. I played with LLMs same as anyone and I never treated it as anything other than a tool that generates responses to my prompts. I never thought "wow, this thing feels so real". Some people clearly have predisposition to jumping over the "it's a tool" reaction straight to "it's a conscious thing I can connect with". I think next step should be developing a test that can predict how someone will react to it.
Surprisingly, the people who have that issues with it aren't the ones who contact to it emotionally, it's the people who offload their decision making to AI
It's more like a codependence spiral than anything else
I bet it's probably correlated with low education as most things
So you're saying there's a chance I can have cheese if I go to college?
Sign me up! Where's the cheddar?
Unfortunately Its now in the Dean's pockets 😭
Cults and toxic self-help literature have existed before LLMs copied them. I don't know if LLMs are getting people who couldn't have been gotten by human scammers.
Scams have many different vectors and people can be vulnerable to them depending on their mood or position in life. Testing people on LLM intolerance would be more like testing them on their susceptibility to viruses.
People can be immunocompromised for various reasons, temporarily or permanently, so as a society public hygiene standards (and the material conditions to produce them) are a lot more valuable. Wash your hands after interacting, keep public spaces clean, that sort of stuff.
Yes, definitely can be a temporary thing which would make it even harder to protect people from. It's also most likely some spectrum. If you're "resistance" is at 10 you may not be at risk even at your lowest point. Other people can be at 5 when they are doing great but risk psychosis when they are down for some reason. I just think it's kind of scary that people interact with it voluntarily (unlike with scammers or cults) without knowing how it will affect them. We all tried LLMs but most of us was lucky so far.
I have yet to see any evidence that AI is inducing problems. People with problems use it just like anyone else and others consider that use problematic.
Huge Study
*Looks inside
this latest study examined the chat logs of 19 real users of chatbots — primarily OpenAI’s ChatGPT — who reported experiencing psychological harm as a result of their chatbot use.
Pretty small sample size despite being a large dataset that they pulled from, its still the dataset of just 19 people.
AI sucks in a lot of ways sure, but this feels like fud.
The hugeness is probably
391, 562 messages across 4,761 different conversations
That's a lot of messages
If that's only 19 users, that's around 250 conversations per user 🤔
...and about 82 messages per conversation. Also, at least half of all the messages are from the user to the AI, and the other half are from the AI to the user, meaning around 41 messages from the user per conversation.
Thanks, you saved me a click 😐
It's not really ethical to just yoink people's chats and study them
"We received chat logs directly from people who self-identified as having some psychological harm related to chatbot usage (e.g. they felt deluded) via an IRB-approved Qualtrics survey "
Tell that to the advertizing companies.
I remember reading my old states book that said a minimum of 30 points needed for normal distribution. Also typically these small sets about proof of concept, so yeah you still got a point.
I wonder if the headline was written by an AI
.....fud?
fud: Fear, Uncertainty and Doubt. A tactic for denigrating a thing, usually by implication of hypothetical or exaggerated harms, often in vague language that is either tautological or not falsifiable.
load more comments
(4 replies)
*hugely funded?
load more comments
(6 replies)
I have a friend that’s really taken to ChatGPT to the point where “the AI named itself so I call it by that name”. Our friend group has tried to discourage her from relying on it so much but I think that’s just caused her to hide it.
its like the AI BF/GFs the subs are posting about.
I certainly enjoy talking to LLMs about work for example, asking things like "was my boss an arse to say x, y, z" as the LLM always seems to be on my side... Now it could be my boss is an arse, or it could be the LLM sucking up to me. Either way, because of the many examples I've read online, I take it with a pinch of salt.
It's definitely sucking up to you. It's programmed to confirm what you say, because that means you keep using it.
Consider how you phrase your questions. Try framing a scenario from the position of your boss, or ask "why was my boss right to say x, y, z", and it'll still agree with you despite the opposite position.
If you're just shooting the shit, consider doing it with a human being. Preferably in person, but there are plenty of random online chat groups too
I use LLMs for work (low priority stuff to save time on search or things that I know I will be validate later in the process) and I can't stand the writing style and the constant attempts to bring in adjacent unrelated topics (I've been able to tone down the cute language and bombastic delivery style in Gemini's configuration).
It's like Excel trying chat with me when I am working with a pivot table or transforming data in PowerQuery.
"Centaurs"
They think they are getting mythical abilities
They're right but not in the way they think
As the researchers wrote in a summary of their findings, the “most common sycophantic code” they identified was the propensity for chatbots to rephrase and extrapolate “something the user said to validate and affirm them, while telling them they are unique and that their thoughts or actions have grand implications.”
There's a certain irony in all the alright techbros really just wanting to be told they were "stunning and brave" this whole time.
Huh. I hate it when people do that. Fake/professional empathy/support. Yet others gobble it up when a machine does that.
Are the users in this study techbros?
Besides, tech bros didn't program this in, this is just an LLM getting stuck in the data patterns stolen from toxic self-help literature.
For decades there has been a large self-help subculture who consume massive amounts of vacuous positive affirmation produced by humans. Now those vacuous affirmations are copied by the text copying machine with the same result and it's treated as shocking.
Anthropic has some similar findings, and they propose an architectural change (activation capping) that apparently helps keep the Assistant character away from dark traits (sometimes). But it hasn't been implemented in any models, I assume because of the cost of scaling it up.
When you talk to a large language model, you can think of yourself as talking to a character
But who exactly is this Assistant? Perhaps surprisingly, even those of us shaping it don't fully know
Fuck me that's some terrifying anthropomorphising for a stochastic parrot
The study could also be summarised as "we trained our LLMs on biased data, then honed them to be useful, then chose some human qualities to map models to, and would you believe they align along a spectrum being useful assistants!?". They built the thing to be that way then are shocked? Who reads this and is impressed besides the people that want another exponential growth investment?
To be fair, I'm only about 1/3rd of the way through and struggling to continue reading it so I haven't got to the interesting research but the intro is, I think, terrible
stochastic parrot
A phrase that throws more heat than light.
What they are predicting is not the next word they are predicting the next idea
How it functionally works, its the next word / token / chunk a lot more than its an "idea". An idea is even rough to define
The other relatively accurate analogy is a probabilistic database
Neither work if you've fallen into anthropomorphising, but they're relatively accurate to architecture and testing for people that aren't too computer literate, far more than the anthropomorphising alternatives at least
throws more heat than light
Thanks, I haven't heard this phrase before, but it feels quite descriptive :)
Technically, they are predicting the next token. To do that properly they may need to predict the next idea, but thats just a means to an end (the end being the next token).
Also, the LLM is just predicting it, it's not selecting it. Additionally it's not limited to the role of assistant, if you (mis) configure the inference engine accordingly it will happily predict user tokens or any other token (tool calls etc).
view more: next ›