An LLM is a glorified Guess The Next Word machine, hence what are quaintly called "hallucinations" in that trade. Of that class of GPT, only one (GPT-4o, I *think*) has an explicit attempt at a "show your workings" mechanism; this may not last long, as it's proved even more processor-intensive to run, and those running the bit barns are getting annoyed.

I've vented at length elsewhere about LLMs, so I'll leave it there. Search for the expression "artificial incompetence".