undefined | Better HN

0 pointsnarrator1y ago0 comments

Isn't it much better to get a Mac Studio with an M2 Max and 192gb of Ram and 31 terraflops for $6599 and run llama.cpp?

0 comments

Macs don't support CUDA which means all that wonderful hardware will be useless when trying to do anything with AI for at least a few years. There's Metal but it has its own set of problems, biggest one being it isn't a drop in CUDA replacement.

adam_arthur1y ago

You can do LLM inference without CUDA just fine. Download Ollama and see for yourself

doublepg231y ago

I'm assuming this won't support CUDA either?

egorfine1y ago

For LLM inference - yes absolutely.

j / k navigate · click thread line to collapse