Show HN: Deterministic security solution for AI agents – OpenClaw and 2 more

4 pointssteadeepanda4d ago8 comments

I wanted to share a solution that I made initially for myself for OpenClaw, that helps control what your ai agents can reach when you let it do stuff without impacting its power, I hope it's useful to you.

Basically the solution lets you experiment freely with your agent within safe boundaries.

It's deterministic on purpose (doesn't include any Al layer) which means the solution follows clear and already defined rules, to maximize safety/security and predictability.

Rules are heavily tested on detecting prompt injection attempts and other security cases (explained in detail in the docs).

Everything is local and lives on your computer including the docs site.

It gives you a control panel to monitor and control boundaries. When boundaries are about to get crossed you receive an approval request which lets you see what your openclaw was trying to do.

It also (currently) supports Tailscale, so you can connect your Tailscale IP address and receive everything on your phone and you can also chat normally, approve or deny requests. It lets access the control panel via your tailscale IP address (a private one is recommended) from anywhere. Currently only Telegram Channel is supported.

Only supports linux os for now and Opencode Claude Code & OpenClaw runners.

The things you need to get started are explained in the readme, also include quick demo/showcase images so you can see how it looks.

I'll be happy to hear feedback from you guys, especially having it tested against prompt injections to see how it handles it, don't hesitate to open a ticket on the GitHub for any issue that you found, I'll do my best to fix them.

Link here: https://github.com/steadeepanda/agent-ruler/

Thank you for reading. I'll be happy to discuss about it.

8 comments

derrak4d ago

> It's deterministic on purpose (doesn't include any Al layer)

I wouldn’t use the word deterministic here. I would use the word symbolic. Determinism, meaning that you always get the same output on the same input, isn’t what you want here. For instance, you can use an LLM without temperature, etc. and its output will be deterministic. More over, if you had a symbolic, non-deterministic algorithm you would probably also be happy to use that.

steadeepandaOP4d ago

LLMs are probabilistic by nature so even if you're using without temperature it doesn't remove completely this fact, it would just narrow the output. However here we're aiming for an already defined set of rules on purpose, with no LLM including in the decision workflow on purpose. You can't safely rely on LLM for security, it's contradictory because of the current nature of LLMs, which is one of the issues that we have today, and that we're trying to propose a solution for. But yeah it's possible to include an LLM in the decision workflow it's just that in comes with cons that I was trying to mitigate with this solution

derrak4d ago

I think your solution is a good idea. I was just pushing back on why it’s a good idea. Determinism isn’t the crux. The crux is that you’re using a symbolic algorithm with well-defined formal semantics.

I was trying to show that determinism is not the crux by pointing out that there are ways to get a deterministic output from an LLM. And that thought experiment shows that determinism isn’t what’s essential.

And I will disagree about merely narrowing the outputs. If I download a local model and set the temperature to zero and give it the same prompt twice, I will get the same output. Not one of several outputs in a narrow set. LLMs are functions.

1 more reply

gbnwl4d ago

My understanding is that yes they are probabilistic by nature in that they give you a probability distribution over all tokens in the vocabulary for the next token, but if you just take the policy of using max prob for each token their output becomes deterministic right? Same output always results for a given input.

1 more reply

jaylew19974d ago

nice

steadeepandaOP4d ago

Thanks

j / k navigate · click thread line to collapse

Show HN: Deterministic security solution for AI agents – OpenClaw and 2 more

4 pointssteadeepanda4d ago8 comments

Basically the solution lets you experiment freely with your agent within safe boundaries.

It's deterministic on purpose (doesn't include any Al layer) which means the solution follows clear and already defined rules, to maximize safety/security and predictability.

Rules are heavily tested on detecting prompt injection attempts and other security cases (explained in detail in the docs).

Everything is local and lives on your computer including the docs site.

It gives you a control panel to monitor and control boundaries. When boundaries are about to get crossed you receive an approval request which lets you see what your openclaw was trying to do.

Only supports linux os for now and Opencode Claude Code & OpenClaw runners.

The things you need to get started are explained in the readme, also include quick demo/showcase images so you can see how it looks.

Link here: https://github.com/steadeepanda/agent-ruler/

Thank you for reading. I'll be happy to discuss about it.

8 comments

derrak4d ago

> It's deterministic on purpose (doesn't include any Al layer)

steadeepandaOP4d ago

derrak4d ago

1 more reply

gbnwl4d ago

1 more reply

jaylew19974d ago

nice

steadeepandaOP4d ago

Thanks

j / k navigate · click thread line to collapse