ChatGPT5 can't answer "How many states have R in it's name?" (opens in new tab)

Basically, LLMs are like "Rain Man".

A fundamental lack of logic, reasoning, creativity and understanding of reality still remains --- even after all the data available on the internet has been statistically consumed and cataloged

Conclusion: Statistics can be used to answer a lot of questions but it is a poor substitute for logic regardless of the size of the database being applied.

Try as you might but some problems simply can't be adequately solved with a simple statistical approach.

answering questions by guessing next tokens in a sentence is different from running .contains on a list of strings. if you formulate your question more specifically it will do the duty..

i am not familiar with american states, but it produced me a list of state names with checkmarks nexto each with an r or R in it (one or more) and a total count. both seem correct. I just dont know if the list itself is correct :p

this guy is asking it by having it draw a map thats not a very logical way to answer a question. it skips over logical steps, so it has a bad answer