That's the mistake right there. It didn't think. The text did.

Specifically, humans encoded thought into language, and encoded language into text. GPT modeled the patterns from that text, and used those patterns to restructure it.

Because language is the most dominant pattern, blindly following an inference model is very likely to result in correct language transformations. Because people don't write nonsense, the result of a correct language transformation is very likely to make sense.