If a certain but fixed amount of "thinking" goes into each letter, then GPT should be able to write the character "..." repeatedly until it solves a problem. The more "..." it writes, the more time it's thought for. Or it could do what mathematicians do, which is write down their working out. This in principle could get around the problems you mention.

I've tried a few times to develop prompts which make ChatGPT interrupt its monologue spontaneously and issue corrections to itself. I haven't got this to work yet.