People will disagree, but my problem with retrieval is that every technique that is popular uses one-hop thinking - you retrieve information that is directly related to the prompt using old-school techniques (even though the embeddings are new, text similarity is old). LLMs are most powerful, IMO, at horizontal thinking. Building a prompt using one-hop narrow AI techniques and then feeding it into a powerful generally capable model is like building a drone but only letting it fly over streets that already exist - not worthless, but only using a fraction of the technology's power.

A concrete example is something like a tool for querying an internal company wiki and the query "tell me about the Backend Team's approach to sprint planning". Normal retrieval approaches will pull information directly related to that query. But what if there is no information about the backend team's practices? As a human, you would do multi-hop/horizontal information extraction - you would retrieve information about who makes up the backend team, you would then retrieve information about them and their backgrounds/practices. You might might have a hypothesis that people carry over their practices from previous experiences, so you look at the previous teams and their practices. Then you would have the context necessary to give a good answer. I don't know of many people implementing RAG like that. And what I described is 100% possible for AI to do today.

Techniques that would get around this like iterative retrieval or retrieval-as-a-tool don't seem popular.