undefined | Better HN

0 pointsundefuser1mo ago0 comments

With that amount of memory can you run 4-bit DeepSeek 4 Flash? It is way more efficient in the KV cache department so may be worth a try

0 comments

1 comments · 1 top-level

I haven't looked into DS4 yet but based on antirez's results on 128 GB Macbooks, it shouldn't be a problem to run it on a pair of RTX6000 Pros.

j / k navigate · click thread line to collapse