Skip to content
Better HN
Top
Best
Ask
Show
New
Jobs
Search
⌘K
0 points
wpm
2y ago
0 comments
Save
Share
AFAIK there is no general purpose, "do this on the ANE" API. You have to be using specific higher level APIs like CoreML or VisionKit in order for it to end up on the ANE.
0 comments
2 comments · 1 top-level
top
newest
oldest
bt1a
2y ago
· 1 in thread
This, plus metal acceleration works quite well. 7~8B parameter models quantized to 3bpw or so run with good tok/s on my iphone 15 pro
sanxiyn
2y ago
It works quite well as long as you don't care about battery.
j
/
k
navigate · click thread line to collapse
