Yes and no. The LLM that sees a JSON structure can decide to use tools to extract and format data as needed, whereas it cannot do the same with natural language.

The Unix philosophy of small, composable tools is still valid in the era of stochastic machines!

I totally agree with what you are saying here and it’s really confusing to me why anyone would think that json is a good format for LLMs. There’s so much redundant text in json. LLMs don’t need that and my experience is that as the document gets bigger it actually hurts the LLM.

I don't disagree, but I'm wondering if there's any evidence of this available.

> After all, LLMs have more natural language training data than JSON training data.

While that is true, data also doesn't usually look like natural language (i.e. a collection of financial records). And when it does (i.e. a collection of chat messages), I wonder if it's more confusing if it's unstructured, even if small.

I expect most frontier models to handle these cases just fine either way, so it may largely depend on context—specifically, how much there is, and where the attention shakes out. Ultimately, a claim one way or the other, for something this context-dependent, would have to be backed up by a lot of testing and would probably conclude that, "in most cases, you should do this"

Agree with JSON. But, surprisingly html and latex perform slightly better than markdown for more complex tables.

Check out this paper - https://arxiv.org/abs/2506.13405