What this is: CuaBot is a TUI to launch any CLI agent (Claude Code, OpenClaw, Codex, etc.) or GUI app inside a sandbox with computer-use. Agent windows appear natively on your desktop with a colored border. Your agent interacts with them through its own cursor, without any mouse/focus hijacking or invasive full-desktop screenshots.
This enables co-op mode: you and your agent work in the same windows with separate cursors.
What you can do:
$ npx cuabot claude > "Write a 2-player tic-tac-toe game, then let's play. I'll go first"
Claude Code will open the game in a sandboxed window on your desktop. When ready, you click your move through the native window while the agent watches and waits to click its move. The agent can see your cursor and its windows while keeping your full desktop isolated.
# Run agents in parallel:
$ npx cuabot -n research openclaw
$ npx cuabot -n coding codex
# Or script the CLI:
$ npx cuabot libreoffice --writer &
$ npx cuabot --click 150 48
$ npx cuabot --type “I Cua!”
Right now my cuabot agent is exploring mobile/desktop apps to turn into cuabench RL environments. I watch the windows appear, intervene when it gets stuck, and let it continue until it opens the completed GUI gym for me to interact with.
Why we built this:
We built Cua (https://github.com/trycua/cua), our open-source SDK for building and benchmarking computer-use systems with GUI sandboxes. We kept seeing two common patterns:
1. Agent screenshots your desktop and controls your mouse – Works with your data, but unsafe and locks you out
2. Agent runs in a sandbox with an external VNC desktop – Safer (our default), but clunky to monitor, hard to interact with, and tedious for data transfer
General computer-use should be frictionless. Asking your agent to debug a GUI app shouldn't require opening an entire desktop stream. The GUI app should just appear alongside your windows, sandboxed and ready.
How it works:
cuabot [command] launches cuabotd, which manages a Ubuntu + Xpra Docker container, a multi-cursor overlay, an Xpra computer-use MCP server, and an Xpra seamless client. It auto-configures your agent (Claude, Aider, etc.) to connect to the computer-use MCP, then pipes terminal I/O through WebSocket. The Xpra client automatically detects and streams windows launched in the container, with H.264 encoding, audio, and customizable clipboard sharing.
Since the computer-use MCP interacts through an Xpra client, the agent only sees the windows it needs, sparing it from your desktop clutter!
GitHub: https://github.com/trycua/cua (monorepo; libs/cuabot directory)
Docs: https://cua.ai/docs
npm: https://www.npmjs.com/package/cuabot
We'll be here to answer questions!
No comments yet.
