Will be interesting to see how Apple and Nvidia's approaches to AI compare and contrast over the coming months/years.
But I really did think that about the Mac Mini power button!
...and then NPUs sorta did nothing. They run a few tiny models, maybe, but for any "serious" inference tasks Apple will automatically prioritize your 10x more powerful GPU hardware. Oftentimes the GPU is more efficient too, depending on the task.
So now Apple has a choice to make. They can either attempt to scale-up the NPU hardware and leave it on-device as dark silicon 99% of the time, or they can renovate their GPU hardware to support complex GPGPU operations and axe the NPU altogether. Right now it seems like Nvidia has the right idea, Apple just needs to find out how to scale it down as well as they can.
Again, I'm not saying the are. They're the only ones that could, though.
Networking compute isn't an issue. Compute architecture is.
Where do you get that macOS is running code like this? I've never heard of shared compute on my computer for anything other than software I deliberately installed. I haven't paid that close attention to the past couple of OS propaganda films at the start of WWDC. Did I miss something?
There's absolutely zero evidence of that, and it would be easy to observe it happening. So why are you pushing some kind of totally false narrative?
None? I don't know what you're referring to. Apple's not running any kind of cloud on my private device. Their Private Cloud Compute is about running inference on Apple's cloud but with privacy guarantees -- not on consumer devices.
> It's one thing for them to offer that as a do-gooder service to their customers, which it certainly is.
Again, no idea what you're talking about. Who's doing good to who? Apple's providing a service to get you to buy their hardware. Consumers aren't doing good for other consumers.
> why wouldn't they use spare compute at the edge for inference? It's smart.
You're talking about on people's devices? Because it would use up their battery and heat up their devices and connectivity can be spotty which leads to terrible latency. I mean, that would be as bad as secretly mining Bitcoin on people's devices. How would that ever be smart?
And Apple's in the business of selling hardware. Not helping people save money by sharing hardware. You can't even share an iPad with different logins for family members.
