I don’t think on site is going to be necessary. Even the US intelligence community trusts that Amazon isn’t spying on the spies.
But a model that can run on a private cluster is certainly something that there’s going to be demand for. And once that exists there’s no reason it couldn’t be run on site.
You can see why OpenAI doesn’t want to do it though. SaaS is more lucrative.
No, the grandparent poster was right. That’s other agencies, not the intelligence community. He’s right that the cloud I was thinking of is on prem but with Amazon personal (that are cleared).
So not the greatest analogy. But still I think most doctors, lawyers etc should be okay with their own cluster running in the cloud.
> You can see why OpenAI doesn’t want to do it though.
Except they already do offer private cluster solutions, you just need usage in the hundreds of millions of tokens per day before they want to talk to you (as in they might before that, but that’s the bar they say on the contact us page).
VMware charges people per GB RAM attached to a VM. Selling on-prem software on consumption is very much possible. It's closed source software, so as long as they require 443 outbound to tick consumption that'd work.
