this post was submitted on 01 Jun 2026
1019 points (99.7% liked)
Technology
85080 readers
4002 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Why would the LLM tool have access to send recovery emails to non account verified emails at all?
That’s insane.
Recently I had to cancel an order. The support for the company was an LLM bot. I accidentally mistyped a number in the order id. It accepted it anyways refunded every order on my account that includes the product I wished to cancel.
I tired to get to a human to correct the mistake and couldn't their phone number is an LLM bot their only chat is an LLM bot.
It use to not be. But now I'm sitting here the order in my hand cause the bot didn't cancel it. But like 30 orders from the last few years have all been refunded to me.
I tried to reach em a few more times but couldn't and it's been like a month. I just have like 2 grand usd that I shouldn't and no way to give it back.
So that's fun.
Would sincerely love to know the name of the company. You know, to avoid them. Yup. I’m sure that’s the reason.
I wonder how long you need to keep that money aside before you can spend it?
That's amazing
Maybe I ought to be taking more advantage of this era of rampant incompetence
I hope you saved what you could from that exchange, as well the attempts to contact them. If they ever notice, their AI mistake will become your problem to deal with, (and the kind of news story to end up on a Steve Lehto video).
If that happened to me, I'd have a chat with my bank, "please help me return this money to where it came from, it was payed in error. They have no way to contact a human and I don't want them to accuse me of fraud down the line".
Who else is going to have access to it when you keep laying off all the people?
Because AI bros are incredibly deluded about both the capability of AI, and by extension their own capabilities using AI>
Because one of the biggest companies on the planet that has issues with account takeovers clearly has no internal red team working on this stuff.
I guarantee they do have a red team that most likely flagged this as an obvious and severe risk. It was ignored by suits experiencing AI psychosis.
I don’t know, more and more of those teams these days are being headed up by the same folks. Most on the ground, in the weeds know what not to do but the ivory tower keeps building more and more floors without ever updating the foundation.
"one of"
should’ve asked it to delete the database instead, why else would it have that level of permissions.
Heh. Watched an old episode of Scorpion yesterday. The one with the armed hostage-takers who just had the one demand to the social media data mining company, to delete all the data they've mined. I amused myself a lot, by uttering "I like these guys".
Oh, man, I hope someone tries this.
Little Tommy Drop Tables.
Drop Table suddenly becomes the newest baby name fad.
This isn't even a hack, it's just poorly written endpoints.
Would you consider phreaking equivalent to hacking? This is AI phreaking.
Kinda.
If you designed a publicly addressable system since 1985 and didn’t design it for security then you’re asking for it.
TIL about phreaking
Then this is also probably new to you
The Anarchist Cookbook
I need to set aside some time to read that although I'm not an anarchist myself.
It was largely overblown due to it getting banned. It was also published in the height of the Vietnam War, when the big evil communists were coming to brainwash your children into eating each other. It has a lot of blatantly incorrect info, which could be outright “blow up in your face” dangerous to anyone attempting the things in it. It’s not all wrong, but certain recipes have incorrect info that could easily lead to accidents.
Also fair warning, the UK will give people hard prison time simply for owning it. So maybe keep that shit onion-encrypted if you’re in the UK.
Thanks for the warning on the blowing up! Well, I'm certainly not in authoritarian UK.
What is '69 ?
I linked to the Wikipedia article, not the handbook inself. And more for the (obsolete) phreaking content than the (highly dangerous) explosive content.
It's not phreaking. Social engineering.
So, I'm currently developing a chatbot for my company. If an LLM needs to do something, a developer must build a tool. It just so happens that this tool that was built did not take traditional security into account. Really it should only be using the tools already built for users, but it seems the Jr. Devs that have been replacing seniors do not have the sensibilities yet.
The entity being manipulated is not human so I would not classify it as social engineering, even if similar techniques are used (help me my grandmother needs info).
It's because they move fast and break things. They think that makes them cool.
Let's mix these chemicals and see what happens. No funds for lab coats or protective glasses. We got a bottom line to feed.
Hold on, do you expect Facebook to pay a human to deal with the inventory? Come on now.
It's not insane. It's advanced!
I tried this and couldn't get it to work. Disappointed.