undefined | Better HN

0 pointsyencabulator3mo ago0 comments

I think we're using different meaning of "behaves like". I meant "has tokens/sec performance comparable to".

0 comments

1 comments · 1 top-level

I'm using model performance because inference is definitely not comparable to a 17B model when you're streaming model weights on and off disk storage.

j / k navigate · click thread line to collapse