undefined | Better HN

0 points2ndorderthought3d ago0 comments

I see your updated post. Switch over to llamacpp and look up recommended quants and settings. A good place for this info is on /r/localllama

0 comments

Yep! I'm currently trying vllm, then I'll give llamacpp a try too

j / k navigate · click thread line to collapse

Yep! I'm currently trying vllm, then I'll give llamacpp a try too

j / k navigate · click thread line to collapse