undefined | Better HN

0 pointsXenophileJKO1mo ago0 comments

I mean you don't have to. With 0 temp, the only source of non-determinism is really the associative math chaning on accumulation of the floating point multiplication. That could be eleminated if we wanted to buy restricting the ordering.

0 comments

bandrami1mo ago

True but at that point you're not really using "an LLM" in the usual sense, you're using a very resource-hungry database.

j / k navigate · click thread line to collapse

0 pointsXenophileJKO1mo ago0 comments

I mean you don't have to. With 0 temp, the only source of non-determinism is really the associative math chaning on accumulation of the floating point multiplication. That could be eleminated if we wanted to buy restricting the ordering.

0 comments

bandrami1mo ago

True but at that point you're not really using "an LLM" in the usual sense, you're using a very resource-hungry database.

j / k navigate · click thread line to collapse