And when that happens, the pitch to non-techy users is "Free ChatGPT you can use offline with zero privacy risk". Once hardware accessibility and LLM efficiency advance to the point that this becomes feasible, I suspect it'll result in a much bigger hit to the cloud AI market than many expect.
AIaaS might keep an edge with multi-modal agentic workflows, but for 80% of general use cases, no "secret sauce" needed, the open weight models are already there, and tooling is constantly getting better.
The bottleneck is the cost of local hardware right now.
"Free, private, offline ChatGPT so long as your laptop has X GB of RAM"
Beyond that, I wouldn't underestimate the incentive of "because I can". The "secret sauce" you refer to is effectively just a DB & a while loop that feeds text to a bunch of tensors. If an indie dev decides they want to release something that dismantles the OpenAI & Anthropic moats, there really isn't all that big of a technical barrier stopping them.
This basically creates a bottleneck at the oldest/cheapest Apple Silicon machines, which are already crippled for context prefill.
I think the bigger hang up is that they're still slower and less capable than the frontier models, especially at the hardware specs most home users are likely to have.
If the first thing (for example) my mom sees upon installing the app is a dropdown model picker that contains things like "Qwen3.6-35b-a3b-mlx" she will 100% be bouncing off of it.
IMO the best version of this is a custom app/harness with a couple of pre-selected (and ideally fine-tuned) open models that immediately start downloading after checking the system's hardware specs. This would likely be a turn-off to most devs, but is absolutely essential if building an app for general consumers.
the current dielmma for me is how do I install a model on a remote LM Studio device without bypassing Lm Studio to SSH or remote in?
> lms link [servername] get model ?
> lms get [servername] model ?
> lms get model --link [servername] ?
Maybe I need to read the docs again but I swear the only way is remote or go to that device and download via the GUI, ssh in and use the local cli.
Maybe can copy/paste from one device's downloads dir to the server? Maybe I need to try hosting models on my NAS and see if I can download from device 1 then run on device 2 without install/setup?
