undefined | Better HN

0 pointsthrowaway4good2y ago0 comments

The M2 is a chip designed to be in a laptop (and it is quite powerful given its low power consumption). Presumedly they have a different chip or at least completely different configuration (RAM, network, etc.) in their data centers.

0 comments

12 comments · 2 top-level

mrweasel2y ago· 8 in thread

The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads. On the Windows (and Linux) side of things, there's no common platform, no assumption that the users will have an NPU or GPU capable of doing what you want. I think that's also why Microsoft was initially going for the ARM laptops, where they'd be sure that the required processing power is available.

qwytw2y ago

> The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads

Also that a significant proportion (majority?) of them will have just 8 GB of memory which is not exactly sufficient to run any complex AI/ML workloads.

talldayo2y ago

Easy solution; just swap multiple gigabytes of your model to SSD-based ZRAM when you run out of memory. What could possibly go wrong?

skohan2y ago

I believe MS is trying to standardize this, in the same way as they do with DirectX support levels, but I agree it's probably going to be inherently a bit less consistent than Apple offerings

pjmlp2y ago

DirectML can use multiple backends.

InsomniacL2y ago

That sounds like a big issue, but surely assuming for either case is bad.

I expect OS's will expose an API which, when queried, will indicate the level of AI inference available.

Similar to video decoding/encoding where clients can check if hardware acceleration is available.

treprinum2y ago

How does it help me (with maxed out M3 Max) that Apple might have some chip in the future right now? I do DL on A6000 and 4090, not waiting until Apple produces a chip someday that is faster than 1650 in ML...

sofixa2y ago

That's probably where Microsoft's "Copilot+ PCs" come in.

pjmlp2y ago

Plus DirectML, wich as the name implies, builds on top of DirectX, allowing multiple backends, CPU, GPU, NPU.

skohan2y ago· 2 in thread

There was a rumor floating around that Apple might try to enter the server chip business with an AI chip, which is an interesting concept. Apple's never really succeeded in the B2B business, but they have proven a lot of competency in the silicon space.

Even their high-end prosumer hardware could be interesting as an AI workstation given the VRAM available if the software support were better.

jitl2y ago

> Apple's never really succeeded in the B2B business

Idk every business I’ve worked and all the places my friends work seem to be 90% Apple hardware, with a few Lenovo issued for special case roles in finance or something.

neodymiumphish2y ago

They mean server infrastructure.

j / k navigate · click thread line to collapse

0 comments

12 comments · 2 top-level

mrweasel2y ago· 8 in thread

qwytw2y ago

> The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads

Also that a significant proportion (majority?) of them will have just 8 GB of memory which is not exactly sufficient to run any complex AI/ML workloads.

talldayo2y ago

Easy solution; just swap multiple gigabytes of your model to SSD-based ZRAM when you run out of memory. What could possibly go wrong?

skohan2y ago

I believe MS is trying to standardize this, in the same way as they do with DirectX support levels, but I agree it's probably going to be inherently a bit less consistent than Apple offerings

pjmlp2y ago

DirectML can use multiple backends.

InsomniacL2y ago

That sounds like a big issue, but surely assuming for either case is bad.

I expect OS's will expose an API which, when queried, will indicate the level of AI inference available.

Similar to video decoding/encoding where clients can check if hardware acceleration is available.

treprinum2y ago

sofixa2y ago

That's probably where Microsoft's "Copilot+ PCs" come in.

pjmlp2y ago

Plus DirectML, wich as the name implies, builds on top of DirectX, allowing multiple backends, CPU, GPU, NPU.

skohan2y ago· 2 in thread

Even their high-end prosumer hardware could be interesting as an AI workstation given the VRAM available if the software support were better.

jitl2y ago

> Apple's never really succeeded in the B2B business

Idk every business I’ve worked and all the places my friends work seem to be 90% Apple hardware, with a few Lenovo issued for special case roles in finance or something.

neodymiumphish2y ago

They mean server infrastructure.

j / k navigate · click thread line to collapse