mdp2021

Premise: I have been mostly out of access to the news in the past six months.

What is the current state of portable, minimal hardware (not assembled desktop boxes with extra cards) to run LLMs and contemporary NNs (Text-to-Image, Speech-to-Text etc.) in general at decent speed and with an eye to power efficiency?

I have read about about boards with shared memory, SoC with NPUs, and about mini-pc from leading brands (that did not run standard Linux though). I do not know if any good solutions are on the market, or if we are still waiting for a leap from future contenders.

Edit: not just "power efficiency", but the overall cost, the overall "dollar-per-token" value would be interesting.

I am of course more interested in the experience of the community than in rumors.