>There is also the question what happens to the original content these LLMs need to actually make their statistical guess at the next word.
The LLMs get granted the capacity to explore their environment physically and gather data on their own. The recent PaLM-E demo shows a possible direction.