undefined | Better HN

0 pointsnarrator2y ago0 comments

Isn't it much better to get a Mac Studio with an M2 Max and 192gb of Ram and 31 terraflops for $6599 and run llama.cpp?

0 comments

4 comments · 2 top-level

magic_hamster2y ago· 2 in thread

Macs don't support CUDA which means all that wonderful hardware will be useless when trying to do anything with AI for at least a few years. There's Metal but it has its own set of problems, biggest one being it isn't a drop in CUDA replacement.

adam_arthur2y ago

You can do LLM inference without CUDA just fine. Download Ollama and see for yourself

doublepg232y ago

I'm assuming this won't support CUDA either?

egorfine2y ago

For LLM inference - yes absolutely.

j / k navigate · click thread line to collapse