undefined | Better HN

0 pointsm0zg6y ago0 comments

Some kinds of inference are expensive, yes, not going to dispute that. But 99.95% of it is actually surprisingly inexpensive. Hell, a lot of useful workloads can be deployed on a cell phone nowadays, and that fraction will increase over time, further reducing inference costs or eliminating them outright (or rather moving them to the consumer).

For the vast majority of people the main expense is creating the combination of a dataset and model that works for their practical problem, with the dataset being the harder (and sometimes more expensive) problem of the two.

The dataset is also their "moat", even though most of them don't realize it, and don't put enough care into that part of the pipeline.

0 comments

3 comments · 1 top-level

fxtentacle6y ago· 2 in thread

The algorithms that run on cell phones tend to be specially optimized and quality-reduced neural networks. For example, https://arxiv.org/abs/1704.04861

jiofih6y ago

Apple does a pretty good job at that, with no compromise in quality.

fxtentacle6y ago

I believe that just due to memory constraints, running any high-quality neural network on phones is currently impossible.

State of the art optical flow tracking needs about 10 GB of GPU memory to execute on full HD frames. I wouldn't know of any mainstream phone with that much RAM.

That, BTW, is also the reason why autonomous drones usually downsample the images before AI tracking, which has the nasty side effect of making thin branches, fences, telephone wires, etc. invisible.

j / k navigate · click thread line to collapse