That is far more likely to happen when it is relying on compressed knowledge of documentation and usage for an API it would have seen (comparatively) only a few times in training. That is where the various types of memory, tool calling and supplementary materials being fed in can make them significantly more situationally useful.

The LLMs you mention are first and foremost a “general knowledge” machine rather than a domain expert. In my opinion, Junior developers are the least likely to benefit from their use because they have neither the foundational understanding to know when the approach is wrong, nor the practical experience to correct any mistakes. An LLM can replace a junior dev because we expect the mistakes and potentially poor quality, but you don’t really want a junior developer doing code reviews for another junior developer before pushing code.