undefined | Better HN

0 pointspiperswe3y ago0 comments

You could probably run a QDR+QSFP Infiniband card at around 32Gbps (minus overhead) through an external "GPU" enclosure. I don't see why MPI wouldn't work on Asahi Linux with such a setup once there's Thunderbolt support.

0 comments

4 comments · 2 top-level

jabl3y ago· 2 in thread

QDR Infiniband is, like, 2007 level tech. Today we have NDR Infiniband where a typical 4xHCA gets you 400 GB/s. Seems like a hypothetical Mac cluster would be severely limited by this compared to the typical x86 based server clusters.

bee_rider3y ago

I’m sure such a switch would be serving some pretty beefy nodes, though, right? Maybe the compute:communication can be held constant with less-powerful Mac mini nodes?

pipersweOP3y ago

The problem is that the Thunderbolt ports don't support that bandwidth, and there isn't any other way to connect PCIe peripherals.

bee_rider3y ago

It is apparently possible to do networking over some Thunderbolt interfaces, would it be possible to connect the devices over Thunderbolt to one another directly? Four ports each, so form a mesh! I guess TB4 can go up to 40Gbps, although it sounds like there’s a bit of overhead when using it as a network, and also I have no idea if there’s some hub-like bottleneck inside the chip…

j / k navigate · click thread line to collapse