undefined | Better HN

0 pointsapexalpha1y ago0 comments

I bought two (relatively) old datacenter GPUs with 48gb VRAM total for €200 that gets me 7 token/s for a 70b model.

0 comments

which GPUs?

Not the GP, but I bought a few P40s over the summer for $150 each. Last I checked they're more expensive now, but it's still cheap vram and fast enough at inference for me.

apexalphaOP1y ago

Nvidia M40 and P40.

j / k navigate · click thread line to collapse