Stop using JSON for LLM structured output (opens in new tab)

(nehmeailabs.com)

2 points44za122mo ago1 comments

1 comments

44za12OP2mo ago

For simple extraction tasks, a delimiter-separated string uses 11 tokens vs 35 for JSON. Output tokens are the latency bottleneck.

j / k navigate · click thread line to collapse

(nehmeailabs.com)

2 points44za122mo ago1 comments

44za12OP2mo ago

For simple extraction tasks, a delimiter-separated string uses 11 tokens vs 35 for JSON. Output tokens are the latency bottleneck.

j / k navigate · click thread line to collapse