this post was submitted on 29 Mar 2024
341 points (93.4% liked)

Technology

59605 readers
3434 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

A Telegram user who advertises their services on Twitter will create an AI-generated pornographic image of anyone in the world for as little as $10 if users send them pictures of that person. Like many other Telegram communities and users producing nonconsensual AI-generated sexual images, this user creates fake nude images of celebrities, including images of minors in swimsuits, but is particularly notable because it plainly and openly shows one of the most severe harms of generative AI tools: easily creating nonconsensual pornography of ordinary people.

you are viewing a single comment's thread
view the rest of the comments
[–] RobotToaster@mander.xyz 122 points 8 months ago (3 children)

This is only going to get easier. The djinn is out of the bottle.

[–] goldteeth@lemmy.dbzer0.com 78 points 8 months ago* (last edited 8 months ago) (1 children)

"Djinn", specifically, being the correct word choice. We're way past fun-loving blue cartoon Robin Williams genies granting wishes, doing impressions of Jack Nicholson and getting into madcap hijinks. We're back into fuckin'... shapeshifting cobras woven of fire and dust by the archdevil Iblis, hiding in caves and slithering out into the desert at night to tempt mortal men to sin. That mythologically-accurate shit.

[–] BrokenGlepnir@lemmy.world 14 points 8 months ago (1 children)

Have you ever seen the wishmaster movies?

[–] db2@lemmy.world 8 points 8 months ago

Make. Your. Wishes.

[–] conciselyverbose@sh.itjust.works 42 points 8 months ago (3 children)

Doesn't mean distribution should be legal.

People are going to do what they're going to do, and the existence of this isn't an argument to put spyware on everyone's computer to catch it or whatever crazy extreme you can take it to.

But distributing nudes of someone without their consent, real or fake, should be treated as the clear sexual harassment it is, and result in meaningful criminal prosecution.

[–] treadful@lemmy.zip 16 points 8 months ago

Almost always it makes more sense to ban the action, not the tool. Especially for tools with such generalized use cases.

[–] ITGuyLevi@programming.dev 6 points 8 months ago

While I agree in spirit, any law surrounding it would need to be very clearly worded, with certain exceptions carved out. Which I'm sure wouldn't happen.

I could easily see people thinking something was of them, when in reality it was of someone else.

[–] cley_faye@lemmy.world 3 points 7 months ago (1 children)

I'm not familiar with the US laws, but… isn't it already some form of crime or something to distribute nude of someone without their consent? This should not change whether AI is involved or not.

[–] T156@lemmy.world 4 points 7 months ago* (last edited 7 months ago)

It might depend on whether fabricating them wholesale would be considered a nude or not. Legally, it could be considered a different person if you're making it, since the "nude" is someone else, and you're putting their face on top, or it's a complete fabrication made by a computer.

Unclear if it would still count if it was someone else and they were lying about it being the victim, for example, pretending a headless mirror-nude was sent by the victim, when it was sent by someone else.

[–] roscoe@lemmy.dbzer0.com 11 points 8 months ago (2 children)

As soon as anyone can do this on their own machine with no third parties involved all laws and other measures being discussed will be moot.

We can punish nonconsensual sharing but that's about it.

[–] CeeBee@lemmy.world 37 points 8 months ago (1 children)

As soon as anyone can do this on their own machine with no third parties involved

We've been there for a while now

[–] roscoe@lemmy.dbzer0.com 6 points 8 months ago (1 children)

Some people can, I wouldn't even know where to start. And is the photo/video generator completely on home machines without any processing being done remotely already?

I'm thinking about a future where simple tools are available where anyone could just drop in a photo or two and get anything up to a VR porn video.

[–] CeeBee@lemmy.world 25 points 8 months ago (1 children)

And is the photo/video generator completely on home machines without any processing being done remotely already?

Yes

[–] roscoe@lemmy.dbzer0.com 13 points 8 months ago (1 children)

Well...shit. It seems like any new laws are already too little too late then.

[–] JDPoZ@lemmy.world 18 points 8 months ago* (last edited 7 months ago) (1 children)

Stable Diffusion has been easily locally installed and runnable on any decent GPU for 2 years at this point.

Combine that with Civitai.com for easy to download and run models of almost anything you can imagine - IP, celebrity, concepts, etc… and the possibilities have been endless.

In fact, with completely free apps like Draw Things on iOS, which allows you to run it on YOUR PHONE locally - where you can download models, tweak, customize, hand it images directly from your mobile device’s library… making this stuff is now trivial on the go.

[–] T156@lemmy.world 2 points 7 months ago

Tensor processors/AI accelerators have also been a thing on new hardware for a while. Mobile devices have them, Intel/Apple include them with their processors, and it's not uncommon to find them on newer graphics cards.

That would just make it easier compared to needing quite a powerful computer for that kind of task.

[–] neptune@dmv.social 10 points 8 months ago (1 children)

I can paint as many nude images of Rihanna as I want.

[–] yildolw@lemmy.world 6 points 8 months ago

You may be sued for damages if you sell those nude paintings of Rihanna at a large enough scale that Rihanna notices