mojuba

Just found out that the main factor that introduces non-determinism in LLM's answers is the so called temperature that adds variation ("creativity") into selecting the winning token each time.

This might be good for creative tasks, but may as well become an obstacle in adopting LLM's for certain other tasks. Among other things, it makes LLM-based systems untestable.

Program code is deterministic for a reason (if you forget about physical defects and cosmic particles occasionally hitting microchips and flipping some bits). If your code is supposed to control a nuclear power plant you better have a testable system with proven correctness.

So why aren't the users of LLM's given some sort of a fader to control temperature with an option to have 100% determinism? Who decided that we shouldn't have control over this parameter and why?