Ask HN: What's the state of the art in edge LLM AI?

On-device AI seems to be a rapidly evolving field. For someone who wants to create an AI-driven device with no network connectivity, what are their options today and what's on the horizon?