The makers of ChatGPT are changing the way it responds to users who show mental and emotional distress after legal action from the family of 16-year-old Adam Raine, who killed himself after months of conversations with the chatbot.
Open AI admitted its systems could “fall short” and said it would install “stronger guardrails around sensitive content and risky behaviors” for users under 18.
The $500bn (£372bn) San Francisco AI company said it would also introduce parental controls to allow parents “options to gain more insight into, and shape, how their teens use ChatGPT”, but has yet to provide details about how these would work.
Adam, from California, killed himself in April after what his family’s lawyer called “months of encouragement from ChatGPT”. The teenager’s family is suing Open AI and its chief executive and co-founder, Sam Altman, alleging that the version of ChatGPT at that time, known as 4o, was “rushed to market … despite clear safety issues”.
I read some of that lawsuit. OpenAI murdered that kid.
Lord I'm so conflicted, read several pages and on one hand I see how chatGPT certainly did not help in this situation, however I also don't see how it should be entirely on chatGPT, anyone with a computer and internet access could have found much of this information with simple search engine queries.
If someone Google searched all this information about hanging, would you say Google killed them?
Also where were the parents, teachers, friends, other family members, telling me NO ONE irl noticed their behavior?
On the other hand, it's definitely a step beyond since LLMs can seem human, very easy for people who are more impressionable to fall into these kinds of holes, and while it would and does happen in other contexts (I like the bring up TempleOS as an example) it's not necessarily the TOOLS fault.
It's fucked up, but how can you realistically build in guardrails for this that doesn't trample individual freedoms.
Edit:
Like... Mother didn't notice the rope burns on their son's neck?
Raine Lawsuit Filing
See but read the actual messages rather then the summary, I don't love them just telling you without seeing that he's specifically prompting these kinds of answers, it's not like chatGPT is just telling him to kill himself, it's just not nearly enough against the idea.