undefined | Better HN

0 pointsyjftsjthsd-h1y ago0 comments

Random sample query ("What shape should a kumquat be?") against a 7B model quantised to 4b running on an i7-9750H (so a good CPU, but also a good laptop CPU from 2019) gives:

  148 tokens predicted, 159 ms per token, 6.27 tokens per second

0 comments

bagels1y ago

Thanks, that helps.

j / k navigate · click thread line to collapse

0 pointsyjftsjthsd-h1y ago0 comments

Random sample query ("What shape should a kumquat be?") against a 7B model quantised to 4b running on an i7-9750H (so a good CPU, but also a good laptop CPU from 2019) gives:

  148 tokens predicted, 159 ms per token, 6.27 tokens per second

0 comments

bagels1y ago

Thanks, that helps.

j / k navigate · click thread line to collapse