undefined | Better HN

0 pointsanigbrowl3d ago0 comments

I literally wrote about running quantized models and how much more affordable it could be in the very next sentence. Please don't reply if you can't be bothered to read the entire comment, it's not that long.

0 comments

2 comments · 1 top-level

sosodev3d ago· 1 in thread

I read the comment, thanks. I just disagree with your cost estimate. Even for a small business that needs high throughput they could probably do it for far less than $300k if they aren’t just blindly buying the first big nvidia setup they can.

anigbrowlOP2d ago

But your estimate was explicitly based on running the 4-bit version. The full uncompressed Deepseek R1 model is 1.4TB. You'd need 16 h100s for this, and considering the new price for an 8 node cluster is about $350k, I think my estimate of $300k was pretty generous, assuming you'd buy used or liquidation or something.

https://www.gmicloud.ai/en/blog/2026-cost-of-renting-or-uyin...

j / k navigate · click thread line to collapse