OpenComputerUse

Computer use for agents that runs in the background, as an MCP server. Start a session with an app and get back a session id. Drive the app with that id, then end the session. The app works behind your other windows, and your pointer, keyboard focus and frontmost app stay where they are. Sessions end when the MCP server exits, including when it is killed.

Install

On macOS, install with Homebrew:

brew tap IAmJSD/OpenComputerUse https://github.com/IAmJSD/OpenComputerUse
brew trust iamjsd/opencomputeruse
brew install --cask opencomputeruse

Homebrew won't load casks from a third-party tap until you trust it, which is what brew trust is for. The cask puts OpenComputerUse.app in /Applications and links the opencomputeruse command into Homebrew's bin. The app updates itself, so brew upgrade leaves it alone. Or download OpenComputerUse.dmg from the latest release. Linux and Windows releases are a single binary. Run it with no arguments for the settings window (the same one as the Mac app's); MCP clients run it as opencomputeruse mcp.

Then add it to your client (see Install into a client).

Tools

Tool	What it does
start_session	Start an app (a .app path, bundle id or name on macOS; an executable elsewhere) and return a session id and its windows; foreground: true (macOS, Windows) brings it to the front for every action instead of keeping it in the background, and active_window: true (macOS, Windows) attaches to the window in front instead of starting an app; display: "wayland" (Linux) runs it on a headless Wayland compositor instead of Xvfb
end_session, list_sessions, list_windows	Manage sessions
screenshot	Capture a session window, even a covered one; ui_tree: true adds the accessibility tree
get_ui_tree	One line per element: [e12] Button "Save" @(x,y wxh) actions=press
click, move_mouse, drag, scroll	Pointer actions at window coordinates, or click with element: "e12"; count: 2 or 3 double- or triple-clicks, and click and drag take held modifiers
type_text, press_key	Text, and chords such as cmd+s or ctrl+shift+tab enter
set_value, element_action	Set an element's value (including a dropdown's option), or run press, focus, showmenu, increment and similar actions
choose_file	Answer the file picker the app is showing (macOS): the paths to pick or save to, or none to cancel. Actions say when one is waiting
wait	Let the app catch up
run_recipe	Run a fixed list of steps with a decision model (see below)
extract	Read typed values, and the elements they belong to, from a window's tree and screenshot with Cloudflare's Clairic (see below)
permissions	What the OS needs granted, and whether it is
phone_list, phone_apps, phone_start_session	Phones and tablets connected to this computer (Android over adb; iPhones and iPads on macOS), when turned on
ios_simulator_list, ios_simulator_apps, ios_simulator_start_session	iOS simulators (macOS only), when turned on
android_emulator_list, android_emulator_apps, android_emulator_start_session	Android emulators (when the emulator is installed), when turned on

Every action returns a fresh screenshot unless you pass screenshot: false. Pass ui_tree: true to also get the tree. Coordinates are points from the window's top-left, the same grid as its screenshot.

The *_start_session tools for devices return an ordinary session id, and every other tool works on it (see Phones, tablets, simulators and emulators).

Recipes

run_recipe runs straight-line chores without the calling model in the loop:

{ "session_id": "…", "steps": [
  "click the address bar",
  "type \"example.com\" into the address bar",
  "press enter",
  { "click": "the More information link" }
] }

For each step, the runner sends the window's interactive elements to a decision model as the options of one choice question. It acts on the chosen element when the model is confident enough. There is no branching, and the model generates no text. A step it cannot place stops the recipe and returns a screenshot and the tree so the caller can take over. Two models are supported, chosen in the app's settings:

• TypeSafe Jev (api.typesafe.ai, model jev-latest)
• Cloudflare Clef / Clef-flash / Clef-omni on Workers AI (@cf/cloudflare/clef-flash; set the model to @cf/cloudflare/clef or @cf/cloudflare/clef-omni in the settings). Clef-omni takes audio and video too, which recipes don't use; on screenshots it is weaker than the other two, and can't read the numbered grid used for windows without a tree.

A step whose request is too long for the model's context window is split: its questions go in separate requests, and a list of elements too long for one question is asked in smaller groups. A single question too long for Jev's 32k tokens is asked again with TypeSafe's long context (TypeSafe-Danger-Zone-Long-Context: true, 262k tokens). That mode is slower and less accurate, so it is only used when needed.

Both models are sent the window's screenshot with each step unless that is turned off in the settings. When a window has no accessibility tree, the step's target is found in the screenshot alone. Jev puts a box question around it (Jev 1.14; while jev-latest is still 1.13 those questions go to jev-preview), then checks a dot at the click point before acting. Clef narrows a numbered grid instead. Jev's images and box questions are in TypeSafe's "danger zone" and are sent with the TypeSafe-Danger-Zone-Images: true header.

Extraction

extract asks Cloudflare's Clairic extraction model (@cf/cloudflare/clairic on Workers AI) for typed values from a session's window. The window's accessibility tree is the text Clairic reads, and its screenshot goes with it as an image (turn that off in the settings, or per call with screenshot: false). A window with no tree is read from the screenshot alone.

{ "session_id": "…", "fields": {
  "save": { "type": "text", "instructions": "The Save button" },
  "unread": { "type": "number", "integer": true, "instructions": "How many unread messages" },
  "subjects": { "type": "list", "items": "text", "instructions": "Each message's subject" }
} }

Each field comes back as a value of its type (text, number, date, time, datetime, duration, list, estimate or forecast) with a confidence and p_null, or null when the window doesn't show it. Clairic generates no tokens, so this is one fast pass. A text answer is a span of the tree, so it comes back with element, the id of the tree line it was read from (elements for lists). Pass that id straight to click, set_value or element_action, which makes extract a way to find a control by description without reading a large tree yourself.

It uses the Cloudflare account ID and API token from the recipe settings (or CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN), and is offered once they are set unless it is turned off in the app ("extract": {"enabled": false}, or OCU_EXTRACT=0).

From the command line:

# Start an app, read its window, end the session.
opencomputeruse extract --app TextEdit --field "words=number:How many words the document has" \
    --field "save=text:The Save button"
# Or read text and images you give it.
opencomputeruse extract --state-file invoice.txt --fields @fields.json --image scan.png

Platforms

Each platform implements the Platform and Session traits in crates/ocu-core. Everything above them is shared: session ids, ownership, cleanup, the MCP tools and recipes.

macOS (crates/ocu-macos)

• Launch: apps open without activating. Your app's windows are put back on top, so the session window sits behind them.
• Screenshots: ScreenCaptureKit captures the window itself, so covered windows capture as they look. The live view streams it with SCStream.
• Tree and element actions: the Accessibility API, which works on background windows.
• Pointer and keys: posted to the app's process through SkyLight. The app is first told its window is active, without being raised; this "focus without raise" approach comes from yabai and trycua/cua.
• File pickers: choose_file answers them, most smoothly first:
  • Chromium browsers started with their own --user-data-dir get a DevTools pipe (--remote-debugging-pipe, so no port is opened). A page's <input type=file> then never shows a panel; its files are set over the pipe. showOpenFilePicker and kin still open the panel.
  • Firefox started with its own -profile opens WebDriver BiDi on a local port, and OCU takes its only session at once. File pickers are held back and answered with input.setFiles.
  • With the optional App Management permission turned on (the app never asks for it; its Turn On button opens System Settings), apps whose signature lets DYLD_INSERT_LIBRARIES through (no hardened runtime, or both allow-dyld-environment-variables and disable-library-validation; never sandboxed ones) load OcuPanelHook.dylib (packaging/macos/panelhook), which hands their open and save panels to OCU instead of showing them. Others never get the variable: where it is honoured, a library that fails to load stops the app.
  • Everything else shows AppKit's panel, which is drawn by a separate process (openAndSavePanelService). Input goes to that process, the path goes in through the panel's Go to sheet, and screenshots compose the panel as the screen draws it.
• Dropdowns: set_value picks a web <select>'s option without opening it, and an app's pop-up button's through its menu, which shows for a moment.

The MCP server is a thin client. The OpenComputerUse app owns the sessions, holds the Accessibility and Screen Recording permissions, and draws a halo and a gliding cursor over the window being driven. It is built with GPUI (the IAmJSD/gpui fork). The MCP server starts the app through LaunchServices when needed, so the app keeps its own permissions whichever client started the server. Opening the app shows its window: permissions, one-click install into Claude Code, Claude Desktop, Codex or OpenCode, live sessions, and recipe settings.

CODESIGN_IDENTITY="Developer ID Application: …" scripts/bundle-macos.sh   # universal (arm64 + x86_64)
cp -R dist/OpenComputerUse.app /Applications/ && open /Applications/OpenComputerUse.app

The icon is assets/icon.svg; packaging/macos/icon.sh rebuilds the .icns from it.

Sign the bundle with a real identity. An ad hoc signature changes on every build, and macOS then asks for the permissions again.

Linux (crates/ocu-linux)

Each session gets a display server of its own, so nothing the app does touches your desktop. start_session takes display: "x11" (the default) or "wayland". Change the default with OCU_LINUX_DISPLAY=wayland or "linux_display": "wayland" in the settings file.

• X11: a private Xvfb with its own cookie. The app starts on it via DISPLAY and XAUTHORITY, with Wayland disabled. Input goes through XTEST; screenshots read the framebuffer, so menus and popups are included. Xvfb is found at $OCU_XVFB, next to the binary, or on PATH.
• Wayland: a headless compositor (Smithay, drawing in software with pixman) that runs inside the server, so there is nothing to install and no GPU is needed. The app gets WAYLAND_DISPLAY (an absolute path in a private directory), no DISPLAY, and software rendering (GSK_RENDERER=cairo, LIBGL_ALWAYS_SOFTWARE=1 and the like). Windows fill the virtual output; dialogs sit over their parent; popups and menus are drawn into their window. Input goes through the compositor's seat. Characters a US keyboard lacks are typed by briefly remapping some of its keys to them, as a person on another layout would, so type_text handles any text in any app (characters with no keysym, such as control characters, are dropped). Needs libpixman-1 and xkb-data, which Xvfb and every desktop already have.
• Chromium on Wayland: Chromium, Chrome, Edge, Brave and Vivaldi are started with --ozone-platform=wayland --disable-gpu --disable-dev-shm-usage, plus --no-sandbox when running as root (as in most containers), unless the session's own arguments already set them.
• Cleanup: displays and apps get PR_SET_PDEATHSIG, so they die with the server.
• File dialogs: off by default, apps draw their own. With Answer open and save dialogs through a portal on in the window (linux_file_portal, or OCU_LINUX_FILE_PORTAL=1), each app gets a private session bus whose file chooser portal is answered by choose_file; everything else is forwarded to your own bus. Needs dbus-daemon.
• Tree and element actions: AT-SPI, the same on both displays since it runs over D-Bus rather than the display. Each session gets a private bus (dbus-daemon) on which the server is the accessibility registry: the app registers its tree there and the server reads it with method calls, a level at a time, each call with a timeout and the whole read within 10 seconds, so a busy app slows a read but never the display or input. Nothing else is on the bus, so sessions never see each other's apps, and no desktop session or at-spi2-registryd is needed. Frames are in window coordinates, matching screenshots on Wayland too. set_value uses EditableText or Value, else focuses the field and types; element actions use the Action interface (a press without one is a click at the element's centre) and focus uses Component.GrabFocus. Recipes work with either decision model.
• What apps get for it: AT_SPI_BUS_ADDRESS pointing at the session's bus, GNOME_ACCESSIBILITY=1, QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1, QT_ACCESSIBILITY=1 and ACCESSIBILITY_ENABLED=1, with NO_AT_BRIDGE and GTK_A11Y removed. Chromium-based browsers also get --force-renderer-accessibility, so web pages are in the tree and not just the browser's own controls. Electron apps need that switch in the session's args.
• In a container: install dbus (for dbus-daemon); without it sessions run without a tree. GTK 3 and Chromium bring their AT-SPI bridge (libatk-bridge2.0-0) with them, and GTK 4 and Qt have theirs built in; at-spi2-core is not needed. Run the server under an init (docker run --init, or tini), or apps' helper processes are never reaped and some apps never open a window.
• Not yet: apps whose toolkit has no AT-SPI (Xt and Motif apps such as xcalc, SDL, most games) have no tree, nor do Java apps without the ATK wrapper; use screenshots and coordinates for those. Wayland stays opt-in until it has run widely; apps that only speak X11 need the X11 display.
• Watching a session: see Live view, which works on both. An X11 session's display can also take a VNC server, for example x11vnc -display :N -auth <xauthority>; both are in the session details.

docker build -f scripts/linux/Dockerfile -t ocu-linux . && docker run --rm --init ocu-linux

Windows (crates/ocu-windows)

• Launch: apps start suspended inside a kill-on-close Job Object, so their whole process tree dies with the server. They are shown without activating and sent to the back.
• Screenshots: PrintWindow with full-content rendering. The live view streams windows through Windows.Graphics.Capture instead.
• Input: window messages posted to the control under the point, or to the focused control.
• Tree and element actions: UI Automation.
• File dialogs: choose_file fills in the dialog's file name and presses its button over UI Automation. With Hand open and save dialogs to agents on in the window (windows_panel_hook, or OCU_WINDOWS_PANEL_HOOK=1), a hook loaded into each app hands its dialogs over without showing them; see packaging/windows/panelhook.

Phones, tablets, simulators and emulators (crates/ocu-mobile)

Off by default: none of these tools are offered until Phones, simulators and emulators is turned on in the app (elsewhere, "mobile": true in the settings file or OCU_MOBILE=1). Clients are told when the tool list changes, and sessions already running carry on when it is turned off.

Sessions on devices take the same tools as desktop ones. Coordinates are points (iOS) or density-independent pixels (Android), and screenshots are scaled to match. A click is a tap, a right click a long press, drag a swipe, and scroll swipes the content by dx/dy. press_key also takes the device's buttons: home, back and recents (Android), power, volume_up, volume_down. There is no hovering, so move_mouse fails.

On macOS the app owns device sessions, as it does desktop ones. Elsewhere the MCP server does. Downloads and emulator logs go in the settings directory's mobile folder.

Android (any desktop) is driven over adb. It needs Android's platform-tools, found through $ANDROID_HOME, $ANDROID_SDK_ROOT, Android Studio's and Homebrew's SDK locations, then PATH (or $OCU_ADB).

• Phones: turn on USB debugging (or wireless debugging) and accept the prompt. phone_list says what a device still needs.
• Emulators: the android_emulator_* tools are listed when the emulator is installed. A session on an emulator that isn't running boots its AVD headless and shuts it down when the last session on it ends.
• The helper: sessions push OpenComputerUse's APK (OpenComputerUse.apk from the matching release, fetched once) to /data/local/tmp and run its shell helper with app_process, the way scrcpy runs its server. Nothing is installed. The helper gives the app a private virtual display, so the phone's own screen, pointer and keyboard are left alone. It also reads the accessibility tree through UiAutomation, injects input into that display, and types any Unicode text into the focused field. This needs Android 11 (API 30) or later. main_display: true runs the app on the device's own screen instead, for apps that refuse a secondary display. home and recents aren't available on a private display, since it has no launcher.
• Without the helper (an older Android, or no APK), sessions use the device's own screen with adb's input, screencap and uiautomator dump. That only types ASCII.
• Launching: app is a package, which starts its launcher activity, or an activity such as com.android.settings/.Settings. args are passed to am start, for example ["-d", "https://example.com"].

iOS (macOS, with Xcode) is driven through Appium's WebDriverAgent, the XCUITest server; OCU_WDA_VERSION picks another release.

• Simulators: the ios_simulator_* tools use WebDriverAgent's prebuilt simulator runner. The app bundles it, made universal, signed, notarized on its own (the notary service doesn't look inside the tarball) and packed with xz by scripts/wda-sim-runner.sh (3.8 MB, with WebDriverAgent's BSD licence beside it), and unpacks it once, so nothing is downloaded. A build outside the app downloads it from WebDriverAgent's release once. A session boots the simulator headless if it isn't running, so nothing appears on screen. show_window: true opens Simulator.app. The simulator shuts down when the last session that booted it ends.
• iPhones and iPads: the device has to be paired and in Developer Mode. The first session builds WebDriverAgent from source with xcodebuild and signs it with your Apple development team, taken from your "Apple Development" certificate or OCU_IOS_TEAM; Xcode must be signed in to that team. That takes a few minutes once. It's built again by itself when its provisioning profile expires (after a week on a free Apple account) or doesn't yet include the device, or when the device refuses its signature. It's installed and launched with devicectl and reached over CoreDevice's tunnel, so USB or the local network both work. The app runs on the device's screen, so keep the device unlocked.
• Launching: app is a bundle id, and args and env are the app's launch arguments and environment. The *_apps tools list bundle ids.

Developer builds use dist/OpenComputerUse.apk from this checkout (android/build.sh), or OCU_ANDROID_APK.

An Android phone as a host (android/)

The same APK is also an app that makes a phone a host by itself, with no computer: other devices drive the phone's apps over HTTP, the way they drive a computer's Other devices server. It uses the same API, keys, skill and hosts file. An accessibility service reads the screen and taps, swipes and types, and a foreground service serves on port 8642.

Install OpenComputerUse.apk from the release, open it, turn it on in Accessibility settings, turn on Serve over HTTP, and tap Generate key… for each device that will drive the phone. A phone offers the session tools only, for apps on itself. The app's own screen, where keys are made, can't be reached over HTTP. See android/README.md, including setup over adb.

The skill is the same from every host, phones included. opencomputeruse skill prints it, and the app ships that text as android/res/raw/skill.md. The tests fail when the two differ.

Live view

A person can watch a session's window and drive it alongside the agent, from a browser. On Linux the view is the session's own display. On Windows and macOS it's the window's own capture stream (Windows.Graphics.Capture, ScreenCaptureKit's SCStream), sending a picture whenever the window changes, with the watcher's clicks and keys sent as the session's actions; where a window can't be streamed, its screenshots are polled instead. On macOS, run it with serve, since the app's MCP server can't share its sessions. Start the server with a view address and a token (at least 16 letters, digits or -._~):

opencomputeruse mcp --view-listen 0.0.0.0:7900 --view-token "$TOKEN"
# or OCU_VIEW_LISTEN=0.0.0.0:7900 OCU_VIEW_TOKEN=$TOKEN opencomputeruse mcp

With the view on, sessions are shared: every client can use every session, and a session lasts until it is ended (or the server exits), not until the client that started it goes. start_session answers with the session's view_path.

One server in a container

For one long-lived server that several harnesses and a web UI share, run serve with a token. The token is accepted beside the keys of other devices, and with it sessions are shared too:

OCU_LINUX_DISPLAY=wayland opencomputeruse serve --port 7800 --token "$TOKEN" \
  --view-listen 0.0.0.0:7900

Harnesses connect to http://<host>:7800/mcp with Authorization: Bearer $TOKEN (for example claude mcp add --transport http ocu http://<host>:7800/mcp --header "Authorization: Bearer $TOKEN"). The view uses the same token unless --view-token sets another. --token can also come from OCU_SERVE_TOKEN.

HTTP API

Every request but GET /health needs the token, as Authorization: Bearer <token> or ?token=<token> (browsers can't set headers on a WebSocket). Answers are JSON; errors are {"error": "..."}. CORS is open, since the token is the only credential.

Request	Answer
GET /sessions	{"sessions": [Session]}
POST /sessions with {"app", "args"?, "env"?, "cwd"?, "display"?, "display_width"?, "display_height"?}	201 and the new Session
GET /sessions/{id}	Session
DELETE /sessions/{id}	{"ok": true}, ending it
GET /sessions/{id}/screenshot[?window=ID]	a PNG
GET /sessions/{id}/view	the WebSocket below

A Session is {"session_id", "session": {"id", "app", "pid", "backend", "details"}, "windows": [Window], "view_path"}, and a Window is {"id", "title", "frame": {"x", "y", "width", "height"}, "on_screen"}.

The WebSocket

GET /sessions/{id}/view, upgraded to a WebSocket. Optional query parameters: window (a window id; the session's first window otherwise), format (jpeg, the default, or png) and quality (JPEG, 1 to 100, default 75). The server accepts the ocu-view subprotocol when offered, and needs none.

The server sends JSON text messages:

• {"type": "hello", "session": {...}, "windows": [Window], "window": ID, "format": "jpeg"} first.
• {"type": "windows", "windows": [Window], "window": ID} when windows open, close, move or are retitled. When the watched window goes, the view moves to the first one (window is then null if there is none).
• {"type": "error", "message": "..."} for a message it could not act on. The connection stays open.
• {"type": "ended"} when the session ends, before closing.

and binary messages, each a 20-byte header (little-endian) then an image:

Offset	Type	Field
0	u8	kind: 1 (an image)
1	u8	format: 1 JPEG, 2 PNG
2	u16	flags: bit 0 set for a keyframe (the image is the whole window)
4	u32	window id
8	u16	x of the image in the window
10	u16	y
12	u16	the image's width
14	u16	the image's height
16	u16	the window's width
18	u16	the window's height

The first picture, and the first after the window's size changes, is a keyframe; after that only the box that changed is sent, to draw at (x, y) over what the client holds. Pictures come when something changed, at most about 30 a second on Wayland and 10 on X11.

The client sends JSON text messages. Coordinates are the window's pixels, the same grid as its pictures and the agent's tools:

• {"type": "pointer", "action": "move" | "down" | "up", "x", "y", "button": 0 | 1 | 2}: button is MouseEvent.button (left, middle, right), default 0.
• {"type": "wheel", "x", "y", "dx", "dy"}: WheelEvent.deltaX and deltaY in pixels (multiply by about 40 for deltaMode 1); positive dy scrolls down. 40 pixels make one wheel notch.
• {"type": "key", "action": "down" | "up", "code", "key"}: KeyboardEvent.code and .key. A key that types one printable character with no Ctrl, Alt or Meta held (AltGr is fine) is typed as that character, so any layout types what its keys say; its release is ignored. Other keys (Enter, arrows, Backspace, shortcuts, modifiers) are pressed by code, as the key in that place on a US keyboard. Send each browser auto-repeat keydown as it comes: apps do not repeat keys themselves. Keys still held when the socket closes are released.
• {"type": "text", "text"}: text to type, such as an input method's or a dead key's composition (compositionend, beforeinput) or a paste.
• {"type": "resize", "width", "height"}: resizes the display and the windows filling it (Wayland; 64 to 8192 pixels). X11 answers with an error.
• {"type": "window", "id"}: watches another window.
• {"type": "refresh"}: the next picture is a keyframe.

The agent keeps working while a person is connected, and both see each other's changes; neither locks the other out.

Install into a client

From the app's window, or:

opencomputeruse install claude          # claude mcp add --scope user opencomputeruse -- <path> mcp
opencomputeruse install claude-desktop  # an "mcpServers" entry in claude_desktop_config.json
opencomputeruse install codex           # codex mcp add opencomputeruse -- <path> mcp
opencomputeruse install opencode        # an "mcp" entry in ~/.config/opencode/opencode.json(c)
opencomputeruse install kimi            # an "mcpServers" entry in ~/.kimi-code/mcp.json
opencomputeruse clients                 # which clients run this copy

Claude Desktop reads its config when it starts, so quit and reopen it after installing. Kimi means Kimi Code (kimi); it keeps its config in $KIMI_CODE_HOME, ~/.kimi-code by default. The legacy kimi-cli is not supported. A client whose entry runs another copy (an old build, or the app before it moved) shows as "points elsewhere"; installing again points it here.

For other clients, use { "command": "<path to opencomputeruse>", "args": ["mcp"] }.

Other devices (HTTP)

Off by default. Turn on Serve over HTTP in the app (or the local MCP server's http_server tool) and other devices can drive this computer through an HTTP API on port 8642, usually over Tailscale. While it is on, it starts again at login, so it survives reboots.

Each device needs a key. Generate Skill asks for the device's name and the URL it reaches this computer at (this computer's Tailscale name by default), and shows the key once, in a dialog with two tabs:

• Prompt for an agent: one prompt to paste into an agent on the device (Claude Code, Codex, OpenCode, …). It holds the skill and this computer's hosts-file entry, and asks the agent to install the one and add the other.
• Skill file: the hosts-file entry and the skill, to put in place by hand. The skill is the same for every computer, so a device needs it only once; each computer it drives adds an entry. Save to hosts file merges the entry into this computer's ~/.config/opencomputeruse/hosts.yaml (replacing an entry of the same name, mode 0600), and Download Entry saves it to ~/Downloads.

Under Generate Skill, the Claude Code, Codex and OpenCode buttons install the key-free skill into that agent's skills folder on this computer.

Where there is no app to make keys with, such as in a container, serve --token TOKEN (or OCU_SERVE_TOKEN) accepts one more key, given on the command line; see One server in a container.

Keys are stored only as hashes, so that is the one place a key appears. Devices are listed in the app with Regenerate Key and Remove, and in the local MCP server as list_devices, generate_skill, regenerate_key and remove_device. Regenerating or removing a key ends that device's sessions. None of this management is reachable over HTTP.

The skill and the hosts file

The skill (opencomputeruse-remote) holds no key. It teaches the agent to read host names, URLs and keys from ~/.config/opencomputeruse/hosts.yaml on the device that drives the others, so one file serves several computers:

hosts:
  work-mac:
    url: "http://work-mac.example.ts.net:8642"
    key: "ocu_..."
  studio:
    url: "http://studio.example.ts.net:8642"
    key: "ocu_..."

Only this small shape is read: a hosts: map, each host with url: and key: values (bare, or in single or double quotes), # comments, and the last entry of a name winning. Create the file with umask 077 and chmod 600. Keys may hold only letters, digits and ._~+/=-. The skill has the agent save its shell helper to ~/.config/opencomputeruse/ocu.sh; ocu HOST TOOL [JSON] reads the key itself and gives it to curl on stdin, so it never shows up in a command line or the agent's transcript. It needs a POSIX shell, awk and curl, so not Windows.

The API:

• POST /v1/tools/<tool> with JSON arguments returns {content, isError}, the same as an MCP tool call. GET /v1/tools lists the tools.
• POST /mcp is MCP over HTTP: claude mcp add --transport http <name> <url>/mcp --header "Authorization: Bearer <key>".
• Every request needs Authorization: Bearer <key>, except GET /health.

There is no TLS, so use it over Tailscale or another trusted network. On Linux and Windows the window serves while it is open, opencomputeruse serve runs the server without it, and serve --install starts that at login.

It listens on every network adapter unless you limit it, for example so it is reachable over Tailscale but not on a coffee shop's Wi-Fi. Untick Every network adapter to every host in the app, pass serve --listen-on, or set http.listen_on in the settings file to a list of:

• an adapter, such as en0, or tailscale for whichever adapter has this computer's Tailscale address,
• an address, such as 192.168.1.5,
• a range, such as 192.168.1.0/24.

The server follows these as addresses come and go, and waits while none of them has one: opencomputeruse serve --listen-on tailscale listens once Tailscale connects.

opencomputeruse ssh-setup HOST --name NAME, with an SSH alias or user@host as HOST, reaches a Linux host without opening its port: it installs this version there, serves on loopback only, creates a device key and keeps an SSH tunnel up from this computer, writing the tunnel's http://127.0.0.1:PORT URL to the hosts file. With --use-installed [--remote-port P] it connects to an opencomputeruse already running on the host (Linux or macOS, HTTP server on) instead, changing nothing there but the new device. A name the host already has a device for is refused; --replace gives that device a new key. The app does the same from Other devices → Set Up Over SSH, once the host's key is known (connect with ssh once) and a key logs in without a prompt.

Settings

The settings file is at opencomputeruse config-path. On macOS it is edited from the app. Elsewhere, edit it by hand or set TYPESAFE_API_KEY, CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_API_TOKEN, OCU_RECIPE_PROVIDER, OCU_MOBILE (1 turns the mobile tools on, 0 off) and OCU_LINUX_DISPLAY (x11 or wayland). Set OCU_LOG=debug for logs on stderr. The macOS agent logs to ~/Library/Application Support/OpenComputerUse/agent.log.

Releasing

Bump version in Cargo.toml, commit, and push a matching tag (v0.2.0). .github/workflows/release.yml builds the signed universal app (OpenComputerUse.zip for the updater, OpenComputerUse.dmg for first installs), plain Linux and Windows binaries of the MCP server, and OpenComputerUse.apk for Android, then publishes them as a GitHub release. The app checks for releases daily and from its menu, and installs them in place when they are signed by the same team.

After the release is published, bump version and sha256 in Casks/opencomputeruse.rb to the new OpenComputerUse.zip (shasum -a 256 OpenComputerUse.zip) so new Homebrew installs get it.

The macOS job signs and notarizes with these repository secrets, and builds unsigned without them: MACOS_CERT_P12_BASE64, MACOS_CERT_P12_PASSWORD (a Developer ID Application certificate), APPLE_ID, APPLE_APP_SPECIFIC_PASSWORD and APPLE_TEAM_ID.

The Android job builds OpenComputerUse.apk with android/build.sh and signs it with ANDROID_KEYSTORE_BASE64, ANDROID_KEYSTORE_PASSWORD and ANDROID_KEY_ALIAS. Without them it signs with a throwaway key, which is fine for the helper but stops the phone app updating in place.

Development

cargo test                                     # core, recipes and the updater
cargo run -p ocu-macos --example smoke         # drive TextEdit directly
python3 scripts/mcp_smoke.py                   # drive it through the MCP server

The widget kit in src/agent/ui comes from Schist (MIT; see LICENSE-SCHIST).

License

MIT, © 2026 Astrid Gealer. See LICENSE.