Not to take anything from what you're saying but any person involved in hiring process _for programmers_ will agree that people struggle with trivia far easier than this example.
Yeah but an LLM can't be excused being nervous or not tired or just not operating at its best like a human can. They are always operating at their best, so if they trip up once, that's their limit
It could very well be that they only get smaller, faster, or even worse when they accidentally start training them with hallucinations that spread on the web.
