undefined | Better HN

0 pointsjayd161y ago0 comments

LLMs are deterministic. We just sample the results, no? Also, no reason AI needs not be deterministic.

0 comments

> LLMs are deterministic.

In theory, yes. In practice, parallelism combined with floating point math make current implementations fundamentally non-deterministic.

Not fundamentally, but extremely costly to keep track of everything. Still deterministic though.

Tycho1y ago

Can you elaborate more on the parallelism aspect?

Temperature cannot ever reach 0 (this causes a division error), so they are not deterministic.

j / k navigate · click thread line to collapse

> LLMs are deterministic.

In theory, yes. In practice, parallelism combined with floating point math make current implementations fundamentally non-deterministic.

Not fundamentally, but extremely costly to keep track of everything. Still deterministic though.

Tycho1y ago

Can you elaborate more on the parallelism aspect?

Temperature cannot ever reach 0 (this causes a division error), so they are not deterministic.

j / k navigate · click thread line to collapse