this post was submitted on 22 Feb 2025
45 points (90.9% liked)

Technology

63082 readers
3599 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
45
Who needs a sneaker bot when AI can hallucinate a win for you? - EQL Blog (www.eql.com)
submitted 15 hours ago by cyrano@lemmy.dbzer0.com to c/technology@lemmy.world
6 comments fedilink hide all child comments
 

By this point, we’d narrowed down the affected users to a single email client - Yahoo Mail, which is where we got suspicious. Had Yahoo Mail introduced any features lately that might be causing this…?

As it turns out, yes, yes they had. A quick Google search revealed that a few months ago Yahoo jumped on the AI craze with the launch of ”AI-generated, one-line email summaries”.

At this point, the penny dropped. Just like Apple AI generating fake news summaries, Yahoo AI was hallucinating the fake winner messages, presumably as a result of training their model on our old emails. Worse, they were putting an untrustworthy AI summary in the exact place that users expect to see an email subject, with no mention of it being AI-generated 🤯

top 6 comments
sorted by: hot top controversial new old
[–] muntedcrocodile@lemm.ee 6 points 11 hours ago (1 children)

I have no issue with AI summaries but they need to be clearly marked as AI and not replace the goddamn subject line.

[–] Mbourgon@lemmy.world 1 points 50 minutes ago

iOS does a decent job of showing it - on the normal list of messages, you get the sender, the first 45 letter or so of the subject, and then their “summarize” icon followed by two smaller-font lines (about 75 characters) of the summarized body

[–] nahostdeutschland@feddit.org 12 points 13 hours ago (1 children)

Just imagine how much energy it must cost to provide this garbage to every mail going through Yahoos servers.

[–] Forester@pawb.social 1 points 1 hour ago* (last edited 1 hour ago)

You do not want to know how much energy it takes to run all of Ymail's spam cannons. Remember they also own and operate AOL.

[–] andallthat@lemmy.world 12 points 14 hours ago

I think that using large language models to summarize email (especially marketing), news, social media posts or any type of content that uses a lot of formulaic writing is going to generate lots of errors.

The way I understand large language models, they create chains of words statistically, based on "what is this most likely to say based on my training material"?

In marketing emails, the same boilerplate language is used to say very different things. "You have been selected" emails have similar wording to "sorry this time you have not won but...". Same cheery "thanks for being such a wonderful sucker" tone and 99% similar verbiage except for a crucial "NOT" here and there.

[–] davidgro@lemmy.world 7 points 14 hours ago

That's insane that they apparently replace the subject line. Completely irresponsible.

Should also be labeled as AI regardless. (Or ideally disabled, especially after incidents like this)