Given the way that OpenAI sources their training corpus and the amount of people using GPT-3 I would not be surprised if GPT-4 winds up getting trained on a large amount of GPT-3 output.
Think about it - the best niche GPT-3 has is generating plausible spam. If you just need a lot of text, but you don't care about what it says[0], you're going to write it using the cheapest possible tool. OpenAI's training corpus is sourced through web crawls, so all of that spam is destined to get recycled back into the next generation of GPT.
[0] For example, if you want to be able to post a bunch of political spam that looks like genuine comments on a web forum. See GPT-4chan[1] as a practical example of this.
[1] A tweaked version of GPT-3 using 4chan's politics board as training corpus.
