While I like to call out people for using "hallucinate" for this kind of behavior too (for language models, at least, it might actually be appropriate for visual models ?)-

> One way I've found to get LLMs to frequently hallucinate is to ask the difference between two concepts that are actually the same

-this only confirms my belief that "bulshitting" is an appropriate term to use for this behavior : doesn't exactly the same thing happen with (not savvy enough) human students ?

You call it "anthropomorphizing", and "not having a model of the world", but isn't it more like forcing a model of the world on the student / language model by the way that you frame the question ?

(Interestingly, there might be a parallel here with the article : with the language model not being a real student, but a statistical average over all students, including being "with one breast and one testicle".)