What I'm trying to say is that that's exactly what it's optimized for. They're predicting what sounds plausible based on all the pre-gpt writing about AI.

But GPT was revolutionary! A lot of the pre-gpt blogspam and reddit comments and fiction and so on was wrong about how AI works in exactly the way you've been socialized to find plausible.

In general plausibility is the wrong metric to evaluate GPT on, and it's wronger than it seems like it should be.

Edit: And in contrast a human trying to write good prompts will have data about how GPT works that they've personally observed, and they'll weigh that data much higher than say Star Trek.

gpt can be layered and made into an agent etc. To do the AB testing or to make prompts longer by adding more end cases as time goes by. But the effects of one single word change are far too complex for gpt base output to understand anything about.