undefined | Better HN

0 pointskarel-3d6d ago0 comments

Can I... somehow run this locally? DeepSeek is opensource? Do I even need their API key?

(I have no experience with running anything locally, maybe it's a stupid question)

0 comments

Waiting for official support in llama.cpp. There is a fork that can run a lightly quantized (Q2 expert layers) DeepSeek V4 Flash in 128GB RAM without offloading weight fetches from disk.

karel-3dOP6d ago

Ouch. Can't run that on my M4 mac with 48GB RAM.

j / k navigate · click thread line to collapse

0 comments

zozbot2346d ago

Waiting for official support in llama.cpp. There is a fork that can run a lightly quantized (Q2 expert layers) DeepSeek V4 Flash in 128GB RAM without offloading weight fetches from disk.

karel-3dOP6d ago

Ouch. Can't run that on my M4 mac with 48GB RAM.

j / k navigate · click thread line to collapse