>But I don’t want an AI to „operate a computer“…
You don't and that's fine but certainly many people are interested in such a thing.
>maybe I’m missing the point of this but I just can’t imagine a usecase where this is a good solution.
If it could operate computers robustly and reliabily then why wouldn't you ? Not everything someone does on a computer is a task they wouldn't like to automate away but can't with current technology.
>For everything browser based, the burden of making an API is probably relatively small
It's definitely not less effort than stricking to a GUI
>and if the page is simple enough, you could maybe even get away with training the AI on the page HTML and generating a response to send.
Sure in special circumstances, it may be a good idea to use something else.
>And for everything that’s not browser-based, I would either want the AI embedded in the software (image editors, IDEs…) or not there are all.
AI embedded in software and AI operating the computer itself are entirely different things. The former is not necessarily a substitute for the latter.
Having access to SORA is not at all the same thing as AI that can expertly operate Blender. And right now at least, studios would actually much prefer the latter.
Even if they were equivalent (they're not), then you wouldn't be able to operate most applications without developers explicitly supporting and maintaining it first. That's infeasible.
