Trope CUA

Background computer use for desktop agents on Windows and macOS.

Trope CUA lets an AI agent inspect and operate real desktop apps while avoiding accidental movement of your real mouse cursor or changes to your active app when the OS and target app support safe routes. It exposes screenshots, accessibility trees, action receipts, and trajectory recording over MCP, a daemon, or direct CLI calls.

The Windows implementation lives at the repository root. The macOS Swift implementation lives under native/macos/trope-cua.

Demo

Click the demo to watch the full video (redirects to X).

Install From Source

Trope CUA is distributed as source. Clone the repository, or download a GitHub-generated source archive from the releases page, then run the installer script locally.

Windows requirements:

• Windows 10 1903+ or Windows 11.
• PowerShell.
• .NET SDK matching global.json.

Install a self-contained Windows build:

.\scripts\install-windows.ps1 -SelfContained

macOS requirements:

• macOS 14 or newer.
• Xcode Command Line Tools.
• Accessibility and Screen Recording permissions for TropeCUA.app.

Install a local macOS build:

./scripts/install-macos.sh

Verify the installed command on either platform:

trope-cua --help
trope-cua check_permissions
trope-cua list_windows

Register with an MCP client:

trope-cua mcp-config
trope-cua mcp-config --client cursor

Or point any MCP client at:

trope-cua mcp

Safety Contract

Mutating actions return receipts. Treat an action as background-safe only when all three fields say:

{
  "background_safe": true,
  "cursor_moved": false,
  "foreground_changed": false
}

ok=true means the requested route completed. It does not by itself mean the route preserved the user's foreground work. Unsafe routes and fallbacks are reported in the receipt instead of being hidden.

Capabilities

• List desktop windows and apps.
• Snapshot a specific target window as pixels, accessibility tree, or both.
• Click, type, press keys, scroll, and set values through background-safe routes when the target exposes them.
• Run as an MCP stdio server, a long-running daemon, or direct CLI tool calls.
• Show a visual agent cursor overlay without moving the user's hardware cursor.
• Record and replay trajectories for debugging and regression work.
• Refuse known unsafe parent-session routes instead of pretending delivery succeeded.

Known Limits

• Trope CUA is not a VM or sandbox. Same-session tools operate on the real user desktop.
• Not every app exposes a safe UIA/MSAA route.
• Chromium and Electron parallel work need separate profiles and CDP ports for true isolation.
• Games, DirectX, Unity, Unreal, raw-input surfaces, and many canvas-heavy apps need child-session or AppBroadcast-style isolation.
• Trope CUA is distributed as source; install from the repository or GitHub source archive.

Read the full known limits before giving an agent authority over authenticated apps.

First Loop

Use MCP or the daemon for workflows that act on element_index values from a snapshot. With the daemon, keep serve running in one terminal and call tools from another:

trope-cua serve --instance demo

trope-cua call --instance demo list_windows '{}'
trope-cua call --instance demo get_window_state '{"pid":1234,"window_id":456789}'
trope-cua call --instance demo click '{"pid":1234,"window_id":456789,"element_index":14}'

For the full workflow, see the quickstart.

Documentation

• Installation
• Quickstart
• Workflows
• Tools and command modes
• Safety
• Privacy
• Threat model
• Known limits

Development

Windows:

dotnet build trope-cua.sln
.\scripts\build.ps1
dotnet format --verify-no-changes --no-restore --verbosity minimal
.\scripts\run-tests.ps1

macOS:

cd native/macos/trope-cua
swift build
./scripts/test.sh
./scripts/install-local.sh

macOS development and permission checks must run on macOS. Windows integration tests must run on Windows.

License

Trope CUA is released under the MIT License. See LICENSE and NOTICE.

Trademarks

Apple, macOS, Microsoft, Windows, OpenAI, Anthropic, Claude, Cua, and other referenced names are trademarks or registered trademarks of their respective owners. This project is not affiliated with or endorsed by those companies or projects.