undefined | Better HN

0 pointsPlatoIsADisease2mo ago0 comments

What models are you running locally? Just curious.

I am mostly restricted to 7-9B. I still like ancient early llama because its pretty unrestricted without having to use an abliteration.

0 comments

mark_l_watson2mo ago

I experimented with many models on my 16G and 32G Macs. For less memory, qwen3:4b is good, for the 32B Mac, gpt-oss:20b is good. I like the smaller Mistral models like mistral:v0.3 and rnj-1:latest is a pretty good small reasoning model.

j / k navigate · click thread line to collapse

0 comments

mark_l_watson2mo ago

j / k navigate · click thread line to collapse