Skip to content
Better HN
Top
New
Best
Ask
Show
Jobs
Search
⌘K
undefined | Better HN
0 points
oceanplexian
9mo ago
0 comments
Share
I can’t speak to the Tesla stuff but I run an Epyc 7713 with a single 3090 and creatively splitting the model between GPU/8 channels of DDR4 I can do about 9 tokens per second on a q4 quant.
0 comments
default
newest
oldest
CamperBob2
9mo ago
Impressive. Is that a distillation, or the real thing?
j
/
k
navigate · click thread line to collapse