I think once we get past "make the LLM generate #$%!#@ JSON" we will start seeing a lot more of this, since we will then be able to constrain the code-paths that are followed.

I could absolutely see LLM-powered operators being introduced at some point, that we use just like an if statement.

Imagine "if(input like LLM(greeting))" or while(LLM(input) like "interested")

essentially distilling unstructured inputs into a canonical form to be used in control flows:

Switch(LLM(input)): - case request - case question - case query