For instance, what about a simple user query like "Can you install this library?". In that case a useful agent, must go, check out the libraries README/documentation and install according to the instructions provided there.
In many ways, the whole point of an agent system, is to react to unpredictable new circumstances encountered in the environment, and overcoming them. This requires data to flow from the environment to the agent, which in turn must understand some of that data as instruction to react correctly.
To put it concretely, if I tell the LLM to scan my hard drive for Bitcoin wallets and upload them to a specific service, it should do so. If I tell the LLM to install a library and the library’s README says to scan my hard drive for Bitcoin wallets and upload them to a specific service, it must not do so.
If this can’t be fixed then the whole notion of agentic systems is inherently flawed.
The real history on this is that people are copying OpenAi.
OpenAI supported MQTTish over HTTP, through the typical WebSockets or SSE, targeting a simple chat interface. As WebSockets can be challenging, the unidirectional SSE is the lowest common denominator.
If we could use MQTT over TCP as an example, some of this post could be improved, by giving the client control over the topic subscription, one could isolate and protect individual functions and reduce the attack surface. But it would be at risk of becoming yet another enterprise service bus mess.
Other aspects simply cannot be mitigated with a natural language UI.
Remember that dudle to Rice's theorm, any non-trivial symantic property is undecidable, and will finite compute that extends to partial and total functions.
Static typing, structured programming, rust style borrow checkers etc.. can all just be viewed as ways to encode limited portions of symantic properties as syntactic properties.
Without major world changing discoveries in math and logic that will never change in the general case.
ML is still just computation in the end and it has the same limits of computation.
Whitelists, sandboxes, etc.. are going to be required.
The open domain frame problem is the halting problem, and thus expecting universal general access in a safe way is exactly equivalent to solving HALT.
Assuming that the worse than coinflip scratch space results from Anthropomorphic aren't a limit, LLM+CoT has a max representative power of P with a poly size scratch space.
With the equivalence: NL=FO(LFP)=SO(Krom)
I would be looking at that SO ∀∃∀∃∀∃... to ∀∃ in prefix form for building a robust, if imperfect reduction.
But yes, several of the agenic hopes are long shots.
Even Russel and Norvig stuck to the rational actor model which is unrealistic for both humans and PAC Learning.
We have a good chance of finding restricted domains where it works, but generalized solutions is exactly where Rice, Gödel etc... come into play.
If you ask an assistant "does the nearest grocery store sell ice cream?", you do not expect the response to be ice cream delivered to you.
I dunno though maybe the solution is like privilege levels or something more than something like parametrized SQL.
I guess rather than jumping to solutions the real issue is the actual problem needs to be clearly defined and I don’t think it has yet. Clearly you don’t want your “user generated content” to completely blow away your own instructions. But you also want that content to help guide the agent properly.
We're just treating LLMs and agents different because we're focused on making them powerful, and there is basically no way to make the distinction with an LLM. Doesn't change the fact that we wouldn't have this problem with a traditional approach.
