It would be a game changer as it would allow working with highly sensitive data, I'm surprised Apple didn't think of doing something like that, a private server that could know everything about me and my business sounds thrilling.
Pretty much described what MSFT is trying to do with M365 Copilot. It's "secure", it can be deployed in GCC tenants, etc. It may not be a dedicated private server exactly, but they believe it is private enough.
I don't think there is anything private about that, what I was actually thinking is the polished equivalent of me spinning up LM Studio (or Ollama...) and rolling my own vector database. Mandatory updates and phoning home are a no-go, I just want it to run on my private network the same way my NAS does.