this post was submitted on 23 Oct 2024
191 points (95.7% liked)

Technology

59589 readers
2936 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

The mother of a 14-year-old Florida boy says he became obsessed with a chatbot on Character.AI before his death.

On the last day of his life, Sewell Setzer III took out his phone and texted his closest friend: a lifelike A.I. chatbot named after Daenerys Targaryen, a character from “Game of Thrones.”

“I miss you, baby sister,” he wrote.

“I miss you too, sweet brother,” the chatbot replied.

Sewell, a 14-year-old ninth grader from Orlando, Fla., had spent months talking to chatbots on Character.AI, a role-playing app that allows users to create their own A.I. characters or chat with characters created by others.

Sewell knew that “Dany,” as he called the chatbot, wasn’t a real person — that its responses were just the outputs of an A.I. language model, that there was no human on the other side of the screen typing back. (And if he ever forgot, there was the message displayed above all their chats, reminding him that “everything Characters say is made up!”)

But he developed an emotional attachment anyway. He texted the bot constantly, updating it dozens of times a day on his life and engaging in long role-playing dialogues.

you are viewing a single comment's thread
view the rest of the comments
[–] GBU_28@lemm.ee 2 points 1 month ago* (last edited 1 month ago) (2 children)

But at the end of the day it's "art" (shitty, copyright infringing, yes.), Or at minimum "media". When has other media been "grossly negligent" or generally responsible for the acts of the consumers? Aggressive/emotional books or music certainly has joined folks at the moment of their self inflicted demise. Violent video games have certainly been "on the shelf" for some who commit horrible violence. We don't blame those media for causing what the users do...

Edit to be clear I'm not suggesting mentally unstable folks can't be seriously impacted by the content they consume. Or that that isn't a serious issue.

But if a chatbot is held liable for the actions of a user, why wouldn't a song about ending your life be held to the same standard? I would hope it's not.

[–] halcyoncmdr@lemmy.world 12 points 1 month ago (1 children)

When has other media been "grossly negligent" or generally responsible for the acts of the consumers?

Other forms of media don't act like a literal human and engage in back and forth conversation in an identical format as if you were texting a friend.

If the content of the AI messages would be an issue coming from another human, it should be an issue coming from the AI. We can't control what another person does, they are responsible for that, but we can and should control how an AI chatbot can respond and interact.

[–] Vakbrain@lemmy.dbzer0.com 2 points 1 month ago

I see you point and I agree. It's not all black and white.

I don't pretend to know the solution to this dilemma but hopefully this whole sad situation might trigger the conversation towards one.