There's this extremely cringe "museum" that OpenAI effectively paid for where they have all these AI exhibits, and one of them involves a phone you can pick up and talk to an AI generated Mr Rogers. This was done without the knowledge or consent of Fred Roger's widow or family. They took his voice and his words, contorted and strung them up with software, and made them dance.
The man that spent decades teaching and entertaining children with puppets had now been turned into one, without his consent.
The women behind this place goes around trying to sell AI to museum professionals in the form of seminars and such. She had the audacity to say "When I'm feeling down, I just pick up the phone, and let Mr Rogers cheer me up." to a room full of museum professionals whose entire job is to honestly interpret and represent history and the dead, and the never, ever, put words in their mouth.
She got chewed apart in the QandA. It was glorious.
Have you ever called a place and got the annoying automated answering voice? Have you ever sent an email to someone and got a boilerplate response? How did that make you feel?
Words aren't math. They are how humans communicate. When you read/hear them, knowing they didn't come from a human, they're hollow.
It's the facade of communication, because you're not actually communicating to a human being. You're using voice commands to control a computer. Asking an AI a question and getting a response is functionally no different than entering 2+2 in a calculator and getting 4 when you hit =.
If we get to a point humanity no longer recognizes or cares about that difference, we'll be in an extremely dark place.