undefined | Better HN

0 pointsMediaSquirrel6mo ago0 comments

Our app wasn't running on CPU or GPU –– the actual software we built was running entirely on Apple Neural Engine and it was crazy fast because we designed the architecture explicitly to run that specific chip.

We were just calling the iPhone's built-in face tracking system via the Vision Framework to animate the avatars. That's the thing that was running on GPU.

0 comments

llm_nerd6mo ago

Okay, though I'm not sure what that has to do with my comment. I understood that from the post: you were concurrently maxing out multiple parts of the SoC and it was overheating as they all contributed to the thermal load. This isn't new or novel -- benchmarks that saturate both the CPU and GPU are legendary for throttling -- though the claim that somehow normal thermal management didn't protect the hardware is novel, albeit entirely unsubstantiated.

That is neither here nor there on CoreML -- which also uses the CPU, GPU, and ANE, and sometimes a combination of all of them -- or the weird thing about MLX.

MediaSquirrelOP6mo ago

I don't get what's so weird about MLX. Apple's focus is obviously on MLX / Metal going forward.

The only reason to use CoreML these days is to tap into the Neural Engine. When building for CoreML, if one layer of your model isn't compatible with the Neural Engine, it all falls back to the CPU. Ergot, CoreML is the only way to access the ANE, but it's a buggy all-or-nothing gambit.

Have you ever actually shipped a CoreML model or tried to use the ANE?

llm_nerd6mo ago

>Apple's focus is obviously on MLX / Metal going forward.

This is nonsensical.

MLX and CoreML are orthogonal. MLX is about training models. CoreML is about running models, or ML-related jobs. They solve very different problems, and MLX patches a massive hole that existed in the Apple space.

Anyone saying MLX replaces CoreML, as the submission does, betrays that they are simply clueless.

>The only reason to use CoreML these days is to tap into the Neural Engine.

Every major AI framework on Apple hardware uses CoreML. What are you even talking about? CoreML, by the very purpose of its design, uses any of the available computation subsystems, which on the A19 will be the matmul units on the GPU. Anyone who thinks CoreML exists to use the ANE simply doesn't know what they're talking about. Indeed, the ANE is so limited in scope and purpose that it's remarkably hard to actually get it to use the ANE.

>Have you ever actually shipped a CoreML model or tried to use the ANE?

Literally a significant part of my professional life, which is precisely why this submission triggered every "does this guy know what he's talking about" button.

1 more reply

j / k navigate · click thread line to collapse

0 comments

llm_nerd6mo ago

That is neither here nor there on CoreML -- which also uses the CPU, GPU, and ANE, and sometimes a combination of all of them -- or the weird thing about MLX.

MediaSquirrelOP6mo ago

I don't get what's so weird about MLX. Apple's focus is obviously on MLX / Metal going forward.

Have you ever actually shipped a CoreML model or tried to use the ANE?

llm_nerd6mo ago

>Apple's focus is obviously on MLX / Metal going forward.

This is nonsensical.

Anyone saying MLX replaces CoreML, as the submission does, betrays that they are simply clueless.

>The only reason to use CoreML these days is to tap into the Neural Engine.

>Have you ever actually shipped a CoreML model or tried to use the ANE?

Literally a significant part of my professional life, which is precisely why this submission triggered every "does this guy know what he's talking about" button.

1 more reply

j / k navigate · click thread line to collapse