LLMs can be good at openings. Not because it is thinking through the rules or planning strategies, but because opening moves are likely in most general training data from various sources. It's copying the most probable reaction to your move, based on lots of documentation. This can of course break down when you stray from a typical play style, as it has less to choose from in the options of probability, and only a few moves in there won't be any more since there's a huge number of possible moves.
I.e., there's no calculations involved. When you play a LLM at chess, you're playing a list of common moves in history.
An even simpler example would be to tell the LLM that its last move was illegal. Even knowing the rules you just told it, it will agree and take it back. This comes from being trained to give satisfying replies to a human prompt.
The twist - AI was originally trained on human input, so it's human even with 99% bots, from a certain point of view.