That's my point. OP doesn't know the maths, has probably never implemented any sort of ML, and is smugly confident that people pointing out the flaws in a system generating one token at a time are just parroting some line.
These tools are excellent at manipulating text (factoring in the biases they have, I wouldn't recommended trying to use one in a multinational corporation in internal communications for example, as they'll clobber non euro derived culture) where the user controls both input and output.
Help me summarise my report, draft an abstract for my paper, remove jargon from my email, rewrite my email in the form of a numbered question list, analyse my tone here, write 5 similar versions of this action scene I drafted to help me refine it. All excellent.
Teach me something I don't know (e.g. summarise article, answer question etc?) disaster!
No, they can summarise articles very convincingly! Big difference.
They have no model of what's important, or truth. Most of the time they probably do ok but unless you go read the article you'll never know if they left out something critical, hallucinated details, or inverted the truth or falsity of something.
That's the problem, they're not an intern they don't have a human mind. They recognise patterns in articles and patterns in summaries, they non deterministically adjust the patterns in the article towards the patterns in summaries of articles. Do you see the problem? They produce stuff that looks very much like an article summary but do not summarise, there is no intent, no guarantee of truth, in fact no concern for truth at all except what incidentally falls out of the statistical probability wells.