undefined | Better HN

0 pointsTepix25d ago0 comments

Sounds good. I saw that you use the FP8 version of the model. Do you also quantize the KV cache?

0 comments

no I don't, since there seem to be a silent degradation bug

j / k navigate · click thread line to collapse

no I don't, since there seem to be a silent degradation bug

j / k navigate · click thread line to collapse