It sounds like you are [dis]interested in a philosophical discussion about epistemology. So it seems that you've skipped the inquiry yourself and have short-circuited to "don't care". Which is kind of "utilitarian". For other perspectives[0]:
> The school of skepticism questions the human ability to attain knowledge, while fallibilism says that knowledge is never certain. Empiricists hold that all knowledge comes from sense experience, whereas rationalists believe that some knowledge does not depend on it. Coherentists argue that a belief is justified if it coheres with other beliefs. Foundationalists, by contrast, maintain that the justification of basic beliefs does not depend on other beliefs. Internalism and externalism debate whether justification is determined solely by mental states or also by external circumstances.
For my part, I do believe that there is non-propositional knowledge. That a person can look at a set of facts/experiences/inputs and apply their mind towards discerning knowledge (or "truth"), or at least the relative probability of knowledge being true. That while this discernment and knowledge might be explained or justified verbally and logically, the actual discernment is non-verbal. And, for sure, correctness is not even essential--a person may discern that the truth is unknowable from the information at their disposal, and they may even discern incorrectly! But there is some mental process that can actually look behind the words to its "meaning" and then apply its own discernment to that meaning. (Notably this does not merely aggregating everyone else's discernment!) This is "intelligence", and it is something that humans can do, even if many of us often don't even apply this faculty ourselves.
From discussions on HN and otherwise I gather this is what people refer to by "world-modeling". So my discernment is that language manipulation is neither necessary nor sufficient for intelligence--though it may be necessary to communicate more abstract intelligence. What LLM/AGI proponents are arguing is that language manipulation is sufficient for intelligence. This is a profound misunderstanding of intelligence, and one that should not be written off with a blithe and unexamined "but who knows what intelligence is anyway".
[0] https://en.wikipedia.org/wiki/Epistemology