I've experienced LLMs forgetting details (no memory). This is expecially a problem when the information is out of the context window, but I've seen it in other cases as well.

I've experienced LLMs lacking spatial awareness, such as switching locations in a description despite no indication of moving to the new location. The same applies to other concepts that have a visual/spatial component.

I've also experienced LLMs struggling to get subtext, some metaphors, etc., especially when used in casual conversations instead of as a question/answer style prompt.

LLMs are great, but need more work to fix these gaps.