undefined | Better HN

0 pointschpatrick3mo ago0 comments

But they're pretty fast and can have loads of RAM, which would be prohibitively expensive with Nvidia.

0 comments

11 comments · 1 top-level

chocochunks3mo ago· 10 in thread

A 128GB 2TB Dell Pro Max with Nvidia GB10 is about $4200, a Mac Studio with 128GB RAM and 2TB storage is $4100. So pretty comparable. I think Dell's pricing has been rocked more by the RAM shortage too.

adgjlsfhk13mo ago

Unfortunately the GB10 is incredibly bandwith starved. You get 128gb ram, but only 270GB/s bandwidth. The M3 Ultra mac studio gets you 820GB/s. (The M4 max is at 410GB/s. I'm not aware of any workload that gets the GB10 to it's theoretical peakflops.

chocochunks3mo ago

You can't get a 128GB M3 Ultra, it's also more expensive. For some workloads the Studio is better, for others the GB10.

midnight_eclair3mo ago

~not unified memory tho~

mciancia3mo ago

It is unified memory on this one

2 more replies

ctxc3mo ago

I took ~ to be a "singing tone" for some reason till I saw sibling and realized it might be an attempted strikethrough xD

benoau3mo ago

That won't hold much benefit as SOCAMM2 and LPCAMM2 get more popular.

traceroute663mo ago

> So pretty comparable.

The Mac Studio almost certainly uses at least half the power

(educated guess, I'm too lazy to go look at all the spec sheets and run the numbers)

bigyabai3mo ago

It's actually reversed. The GB10 chipset has a TDP of 140w, whereas M2/M3 Ultra pulls over 250w from the wall: https://support.apple.com/en-us/102027

1 more reply

plagiarist3mo ago

Not quite, what is the vRAM bandwidth of each? The bandwidth is a huge contributor to LLM performance.

embedding-shape3mo ago

AFAIK, for the unified bandwidth, it depends mostly on the CPU, for M4 Max (I think it's the default today?) it does ~550 GB/s, while GB10 does ~270 GB/s, so about a 2x difference between the two. For comparison, RTX Pro 6000 does 1.8 TB/s, pretty much the same as what a 5090 does, which is probably the fastest/best GPUs a prosumer reasonable could get.

1 more reply

j / k navigate · click thread line to collapse