undefined | Better HN

0 pointsstymaar4d ago0 comments

RAM price don't change anything. You can't fit an infinity of tokens in the KV cache, but the ones that are in there when a user request them are still practically free.

0 comments

2 comments · 1 top-level

jgalt2124d ago· 1 in thread

so the utility of the KV cache has nothing to do with available RAM?

stymaarOP3d ago

This proposition has no logical link with any of the above. Read again.

j / k navigate · click thread line to collapse