Because it's not. The base architecture of how it works is by probabilistic word suggestion. That isn't thought.

We have a concept of self. We understand our place. We can interpret and respond to entirely new situations. LLMs routinely fail that. They regularly fall into local minima that keep it on the wrong path, and I've personally seen them just... Get lost in the weeds and swing back and forth based on what you tell it.

Give it a protein sequence and tell it to calculate the pI. Then tell it it's wrong. "Oh my bad yes you're right it's {whatever you said it was}."

Tell it you lied and that the number you said was wrong, and it turns up saying "Yes, you're correct, the pI is {original value}" - that is objectively false.

That is not the behavior of something that thinks. That's the behavior of a simple probability model updating priors and weighting things differently by the most recent information you gave it.

LLMs are soulless, brainless, thoughtless word generators. And they have some uses.