> What gives you any confidence that the way GPT4 comes up with answers is qualitatively different from humans?

For a start, GPT-4 doesn't include in its generation the current state of its internal knowledge used so far; any text built can only use at most the few words already generated in the current session as a kind of short-term memory.

Biological brains OTOH have a rhythm with feedback mechanisms which adapt to the situation where they're doing the thinking.