undefined | Better HN

0 pointsAurornis1d ago0 comments

Running DeepSeek V4 without extreme quantization locally requires a lot of hardware.

The IQ2 quants that fit into 128GB machines are very degraded.

0 comments

That is true, it is a 1.6T parameters model so it requires a great deal of memory. I also heard there's a 2bit quantization that works well on Apple metal.

j / k navigate · click thread line to collapse

0 comments

binyu1d ago

That is true, it is a 1.6T parameters model so it requires a great deal of memory. I also heard there's a 2bit quantization that works well on Apple metal.

j / k navigate · click thread line to collapse