undefined | Better HN

0 pointsomneity2y ago0 comments

A friend just got an M3 Max with 128GB (V)RAM and he's extremely happy with it (AI workloads wise). That could be an option if you can run your simulations on macOS.

0 comments

AnthonyMouse2y ago

That machine with 128GB is $5000, with 48GB is still well over $3000, and has as much memory bandwidth as a $400 GPU. At the current spot price, 128GB of GDDR6 is <$400 and 48GB is <$150, implying that they could be paired with any existing <$1000 GPU to produce something significantly faster for dramatically less money. If anyone could be bothered to make one.

omneityOP2y ago

I agree with you, that’s how it should be, but that’s not how it currently is.

Looking at what’s available right now. You need 3 A100 40GB to get this amount of VRAM which will cost you way north of 20000$.

Doing it with A6000s is still about 15k$.

There’s not that many high VRAM options out there you know..

AnthonyMouse2y ago

If all you want is VRAM you can get old P40s with 24GB for $175 and it's 144GB for $1050. Then you need a big machine to put six of them in but that doesn't cost $4000.

But all of this is kludges. The Radeon RX 7900 XTX has more than twice the memory bandwidth of the M3 Max with much better performance per watt than an array of P40s. What you want is that with more VRAM, not any of this misery.

1 more reply

cyanydeez2y ago

lastgen A6000 is about 5k

j / k navigate · click thread line to collapse

0 comments

AnthonyMouse2y ago

omneityOP2y ago

I agree with you, that’s how it should be, but that’s not how it currently is.

Looking at what’s available right now. You need 3 A100 40GB to get this amount of VRAM which will cost you way north of 20000$.

Doing it with A6000s is still about 15k$.

There’s not that many high VRAM options out there you know..

AnthonyMouse2y ago

If all you want is VRAM you can get old P40s with 24GB for $175 and it's 144GB for $1050. Then you need a big machine to put six of them in but that doesn't cost $4000.

1 more reply

cyanydeez2y ago

lastgen A6000 is about 5k

j / k navigate · click thread line to collapse