Let's suppose that the datasets don't increase in size. Look at the difference in speed between the old ChatGPT and turbo ChatGPT. Suppose within 6-8 months they can do that again with GPT4.

I think that would be about five times faster than a human.

But there is more data. Do you really think they have ingested every single YouTube video or movie? Video and video+transcription is the next thing.

Another source of data could be to have the models study and reprocess information into more concise language (possibly new vocabulary) or diagrams with the goal of increasing levels of abstraction or information density.