Yes, but is that really why people want the context window to be "infinite"? In my experience, the desire for bigger context is the ability to say, dump an 1000 page book into an LLM and ask questions about any part of it, not just the last chapter.