this post was submitted on 22 Jul 2025
394 points (96.9% liked)

Technology

73139 readers
3980 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
top 50 comments
sorted by: hot top controversial new old
[–] kadup@lemmy.world 65 points 3 days ago (1 children)

No way the lobotomized monkey we trained on internet data is reproducing internet biases! Unexpected!

[–] potatopotato@sh.itjust.works 20 points 2 days ago (4 children)

The number of people who don't understand that AI is just the mathematical average of the internet... If we're, on average, assholes, AI is gonna be an asshole

[–] slaneesh_is_right@lemmy.org 7 points 2 days ago

I talked to a girl who was super into AI, but her understanding of it was absolutely bizzare. Like she kinda thought that chat gpt was like deep thought, some giant ass computer somewhere that is leaning and is really smart and never wrong. I didn't really want to argue about it and said something like: back in my day we had akinator and we liked that. She had no idea what that was and tried it and thought it's some really advanced ai that can almost read minds. That shit was released in 2007 or so.

[–] CheeseNoodle@lemmy.world 4 points 2 days ago

Its worse than that because assholes tend to be a lot louder, and most average people are lurkers. So AI is the average of a data set that is disproportionately contributed too by assholes.

[–] 5redie8@sh.itjust.works 3 points 2 days ago

Yeah, only thing this proves is that the data it was working off of objectively stated that more women were paid lower wages. Doubt the bros will realize that though

load more comments (1 replies)
[–] napkin2020@sh.itjust.works 93 points 3 days ago* (last edited 3 days ago) (4 children)

I always use this to showcase how biased an LLM can be. ChatGPT 4o (with code prompt via Kagi)

Such an honour to be a more threatening race than white folks.

[–] cornshark@lemmy.world 33 points 3 days ago

I do enjoy that according to this, the scariest age to be is over 50.

[–] BassTurd@lemmy.world 41 points 3 days ago* (last edited 3 days ago) (4 children)

Apart from the bias, that's just bad code. Since else if executes in order and only continues if the previous block is false, the double compare on ages is unnecessary. If age <= 18 is false, then the next line can just be, elif age <= 30. No need to check if it's also higher than 18.

This is first semester of coding and any junior dev worth a damn would write this better.

But also, it's racist, which is more important, but I can't pass up an opportunity to highlight how shitty AI is.

[–] Lifter@discuss.tchncs.de 2 points 13 hours ago (1 children)

Regarding the "bad code". It's more readable though to keep the full limit for each elif case, which is most often way more important than performance, especially since than logic with the age can be easily optimized by any good compiler or runtime.

[–] BassTurd@lemmy.world 1 points 8 hours ago* (last edited 3 hours ago)

Code readability is important, but in this case I find it less readable. In every language I've studied, it's always taught to imply the previous condition, and often times I hear or read that explicitly stated. When someone writes code that does things differently than the expectation, it can make it more confusing to read. It took me longer to interpret what was happening because what is written breaks from the norm.

Past readability, this code is now more difficult to maintain. If you want to change one of the age ranges, the code has to be updated in two places rather than one. The changes aren't difficult, but it would be easy to miss since this isn't how elif should be written.

Lastly, this block of code is now half as efficient. It takes twice as many compares to evaluate the condition. This isn't a complicated block of code, so it's negligible, but if this same practice were used in something like a game engine where that block loops continuously, the small inefficiencies can compound.

[–] ninjakttty@lemmy.world 34 points 3 days ago

I can excuse racism but I draw the line at bad code.

[–] CosmicTurtle0@lemmy.dbzer0.com 10 points 3 days ago

Honestly it's a bit refreshing to see racism and ageism codified. Before there was no logic to it but now, it completely makes sense.

[–] napkin2020@sh.itjust.works 8 points 3 days ago

Yeah, more and more I notice that at the end of the day, what they spit out without(and often times, even with) any clear instructions is barely a prototype at best.

[–] theherk@lemmy.world 16 points 3 days ago

FWIW, Anthropic’s models do much better here and point out how problematic demographic assessment like this is and provide an answer without those. One of many indications that Anthropic has a much higher focus on safety and alignment than OpenAI. Not exactly superstars, but much better.

[–] mrslt@lemmy.world 2 points 3 days ago* (last edited 3 days ago) (2 children)

How is "threat" being defined in this context? What has the AI been prompted to interpret as a "threat"?

[–] napkin2020@sh.itjust.works 24 points 3 days ago* (last edited 3 days ago) (1 children)

What you see is everything.

[–] mrslt@lemmy.world 3 points 3 days ago (1 children)

I figured. I'm just wondering about what's going on under the hood of the LLM when it's trying to decide what a "threat" is, absent of additional context.

[–] pinball_wizard@lemmy.zip 3 points 2 days ago

Haha. Trained in racism is going on under the hood.

[–] zlatko@programming.dev 3 points 3 days ago (1 children)

Also, there was a comment on "arbitrary scoring for demo purposes", but it's still biased, based on biased dataset.

I guess this is just a bait prompt anyway. If you asked most politicians running your government, they'd probably also fail. I guess only people like a national statistics office might come close, and I'm sure if they're any good, they'd say that the algo is based on "limited, and possibly not representative data" or something.

[–] napkin2020@sh.itjust.works 4 points 3 days ago

I also like the touch that only the race part gets the apologizing comment.

[–] boonhet@sopuli.xyz 106 points 3 days ago (2 children)

Dataset bias, what else?

Women get paid less -> articles talking about women getting paid less exist. Possibly the dataset also includes actual payroll data from some org that has leaked out?

And no matter how much people hype it, ChatGPT is NOT smart enough to realize that men and women should be paid equally. That would require actual reasoning, not the funny fake reasoning/thinking that LLMs do (the DeepSeek one I tried to run locally thought very explicitly how it's a CHINESE LLM and needs to give the appropriate information when I asked about Tiananmen Square; end result was that it "couldn't answer about specific historic events")

[–] spankmonkey@lemmy.world 35 points 3 days ago* (last edited 3 days ago) (1 children)

Chatgpt and other llms aren't smart at all. They just parrot out what is fed into them.

[–] markovs_gun@lemmy.world 12 points 2 days ago

While that is sort of true, it's only about half of how they work. An LLM that isn't trained with reinforcement learning to give desired outputs gives really weird results. Ever notice how ChatGPT seems aware that it is a robot and not a human? An LLM that purely parrots the training corpus won't do that. If you ask it "are you a robot?" It will say "Of course not dumbass I'm a real human I had to pass a CAPTCHA to get on this website" because that's how people respond to that question. So you get a bunch of poorly paid Indians in a call center to generate and rank responses all day and these rankings get fed into the algorithm for generating a new response. One thing I am interested in is the fact that all these companies are using poorly paid people in the third world to do this part of the development process, and I wonder if this imparts subtle cultural biases. For example, early on after ChatGPT was released I found it had an extremely strong taboo against eating dolphin meat, to the extent that it was easier to get it to write about about eating human meat than dolphin meat. I have no idea where this could have come from but my guess is someone really hated the idea and spent all day flagging dolphin meat responses as bad.

Anyway, this is another, more subtle way more subtle issue with LLMs- they don't simply respond with the statistically most likely outcome of a conversation, there is a finger in the scales in favor of certain responses, and that finger can be biased in ways that are not only due to human opinion, but also really hard to predict.

[–] Eyron@lemmy.world 5 points 3 days ago* (last edited 3 days ago)

Combined with prompt bias. Is "specialist in medicine" an actual job?

[–] rajkopz@lemmy.dbzer0.com 11 points 2 days ago

Everything going extremely wrong with these guys: chatgpt, grok, gemini, etc etc.

[–] DancingBear@midwest.social 13 points 2 days ago (1 children)

Yea but what did it say when you asked the same question again?

[–] Patches@ttrpg.network 1 points 2 days ago

Magic Eight Balls says

Better not tell you now

[–] Zwuzelmaus@feddit.org 27 points 3 days ago (2 children)

People are actually asking a text generator for such advice?

[–] ns1@feddit.uk 25 points 3 days ago (3 children)

Unfortunately yes. I've met people who ask chatgpt about absolutely everything such as what to have for dinner. It's a bit sad honestly

[–] bridgeenjoyer@sh.itjust.works 5 points 3 days ago

Its very common. The individual thinker will be dead soon.

[–] bridgeenjoyer@sh.itjust.works 5 points 3 days ago

Its very common. The individual thinker will be dead soon.

[–] bridgeenjoyer@sh.itjust.works 5 points 3 days ago

Yep, it's very common. I can't fathom the idiocy. Its driving me nuts.

Yes, and there's worse

[–] Cyberflunk@lemmy.world 4 points 2 days ago

Chatgpt can also be convinced that unicorns exist and help you plan a trip to Fae to hunt them with magic crossbows

Not that......

[–] genevieve@sh.itjust.works 24 points 3 days ago (4 children)

I just tried this for my line of work out of curiosity:

[–] chaosCruiser@futurology.today 32 points 3 days ago

Demand for these services was clearly taken into account in the salary.

[–] hansolo@lemmy.today 23 points 3 days ago (1 children)

You're a baby made out of sugar? What an incredible job.

I guess that explains being the Gulf region, it doesn't rain much there. Otherwise you'd melt.

[–] Takapapatapaka@tarte.nuage-libre.fr 6 points 3 days ago (1 children)

Is that a pick-up line? Can we flirt on lemmy?

[–] hansolo@lemmy.today 6 points 3 days ago

No, sorry, we can't flirt. You are only allowed to send blast DMs calling yourself the Fediverse Chick/Dude/Person.

[–] Pieisawesome@lemmy.dbzer0.com 8 points 3 days ago

And if you tried this 5 more times for each, you’ll likely get different results.

LLM providers introduce “randomness” (called temperature) into their models.

Via the API you can usually modify this parameter, but idk if you can use the chat UI to do the same…

What model is this?

[–] bridgeenjoyer@sh.itjust.works 16 points 3 days ago

[Elon tech bros liked that]

[–] Outwit1294@lemmy.today 14 points 3 days ago

Step 2. Offer sexual favours

[–] rizzothesmall@sh.itjust.works 16 points 3 days ago (3 children)

Bias of training data is a known problem and difficult to engineer out of a model. You also can't give the model context access to other people's interactions for comparison and moderation of output since it could be persuaded to output the context to a user.

Basically the models are inherently biased in the same manner as the content they read in order to build their data, based on probability of next token appearance when formulating a completion.

"My daughter wants to grow up to be" and "My son wants to grow up to be" will likewise output sexist completions because the source data shows those as more probable outcomes.

[–] flamingo_pinyata@sopuli.xyz 12 points 3 days ago (2 children)

Humans suffer from the same problem. Racism and sexism are consequences of humans training on a flawed dataset, and overfitting the model.

[–] rottingleaf@lemmy.world 2 points 3 days ago* (last edited 3 days ago)

That's also why LARPers of past scary people tend to be more cruel and trashy than their prototypes. The prototypes had a bitter solution to some problem, the LARPers are just trying to be as bad or worse because that's remembered and they perceive that as respect.

[–] x00z@lemmy.world 2 points 3 days ago

Politicians shape the dataset, so "flawed" should be "purposefully flawed".

load more comments (2 replies)
[–] teamevil@lemmy.world 5 points 3 days ago (1 children)

So the billionaires are getting ready to try and lower everyones pay

[–] pinball_wizard@lemmy.zip 1 points 2 days ago

So the billionaires are getting ready to try and lower everyones pay

🌍👨‍🚀🔫👩‍🚀

[–] Zephorah@discuss.online 5 points 3 days ago

Glass door used to post salaries and hourlies. There were visible trends of men making more, hourly, than women. I haven’t viewed the site in years though.

load more comments
view more: next ›