I agree. I also think that deciding that LLMs encode all knowledge perfectly, either now or in an imagined future, is foolish. My experience is that they match the average general state of experts among the field. The sort of thing a junior might read to start to grasp the general ideas and issues in a field. They rarely have opinions, or good intuitions around more specific scenarios. This is why the current equilibrium of a senior piloting one works so well- theyre leaning on it to speed up, but pushing it away from the "average" where circumstances demand.

We can argue about imagined future progress, but I don't see that getting much better, given that the literature doesn't often do that, and how often experts in one scenario end up being poorly suited given another set of facts.