But LLMs get their answers from StackOverflow and similar places being used as the source material. As those start getting outdated because of lack of activity, LLMs won't have the source material to answer questions properly.
I regularly use Claude and friends where I ask it to use the web to look at specific GitHub repos or documentation to ask about current versions of things. The “LLMs just get their info from stack overflow” trope from the GPT-3 days is long dead - they’re pretty good at getting info that is very up to date by using tools to access the web. In some cases I just upload bits and pieces from a library along with my question if it’s particularly obscure or something home grown, and they do quite well with that too. Yes, they do get it wrong sometimes - just like stack overflow did too.
The amount of docs that have a “Copy as markdown” or “Copy for AI” button has been noticeably increasing, and really helps the LLM with proper context.
they’re pretty good at getting info that is very up to date by using tools to access the web
Yeah that's a charitable way to phrase "perform distributed denial of service attacks". Browsing github as a human with their draconian rate limits that came about as a result of AI bots is fucking great.
StackOverflow answers are outdated. Every time I end up on that site these days, I find myself reading answers from 12 years ago that are no longer relevant.
There have been many times I have seen someone complain on the meta site about answers being old and outdated, and then they give specific examples, and I go check them out and they're actually still perfectly valid.
