Chrome Agent Platform

Chrome as the agent platform: a new-tab agent hub that orchestrates the web with persistent named agents, per-site sub-agents (WebMCP tool discovery), and the generative-UI surface. Built on agent-do patterns + the Vercel AI SDK (swappable models).

The hub is the command center: a task composer, a conversation surface with live progress, a task thread list, and the agents you've created — each with its own isolated OPFS memory, run history, skills, and avatar.

What it does

• The agent model — persistent, named agents (avatar + name + role) with their own OPFS sandbox (memory + run history + installable skills + a memory_grep tool). Named, site, and background agents are all isolated. The master hub agent creates + manages them (create/update/delete/delegate).
• Unified agent access — one shared <agent-picker> everywhere: the side panel's Agents view (browse/search/select → the agent's own conversation → direct a task, live updates, per-session restore), every composer's + menu "Choose agent" (a removable agent chip routes the run by canonical ID), and the /agent slash command (grouped, keyboard-complete, stale selections rejected).
• Local folder context — grant one or more directories in Settings → Local folders, then type /files or /files:name in any composer to search by file name and attach a result. Text reads are capped at 1 MiB; binary and larger files stay metadata-only references, and lapsed access always shows the re-grant path.
• Sites as sub-agents — lightweight detection-only scripts notice WebMCP on every http(s) page. When an open page offers tools, the hub shows a chip above the composer — "shop.example offers 5 tools — use them?" — and that one click is the grant (it names the exact origin) and makes the site a Site Agent; Find site tools is the explicit picker for the same path and lists only open tabs known to expose tools; plain pages never become setup candidates. Choosing one then installs the authenticated invocation bridge (first-run approval per tool). Discovery is observable: enable Settings → Site agents → Diagnostics for gated [WebMCP] logs (page DevTools console) + a status readout (last discovery, origin, script state, tool count). Failed calls keep their full page-thrown error and argument shape in that page-local console; only the redacted error name crosses into the extension and model path.
• Tasks as threads — a task is a distinct thread (auto-named, full-screen, with live progress + per-task error detail + a nudge/continue composer).
• Skills — reusable capabilities (the recipes reworked into skills): include a skill in a task (/skill:<id>), attach one to an agent, schedule one, or import an external skill (the skill-loader pattern: a GitHub repo/URL → SKILL.md).
• Generative UI + artifacts — the agent generates HTML UI rendered live in a sandboxed double-iframe, saved as a reusable artifact.
• Agent-generated scripts — the agent writes JavaScript that runs repeatedly (on a schedule/hook) in a sandboxed worker, without re-invoking the model.
• System hooks — the full chrome.* on* event surface as candidate hooks, with an owner-only authoritative deny-list.
• Agent workers — each agent runs in its own shared worker hosted by the offscreen document and bootstrapped through the service worker. One crashed or leaky agent no longer takes the router and every other agent down with it. The UI holds a live MessagePort with redacted progress; background agents run with zero visible pages. Destructive browser commands are authorised by the owner's browser-control grant (checked atomically in the service worker) and fenced to their run.
• 139 Chrome tools, every chrome.* call audited against the Chromium IDL/JSON schemas (191 capability rows total, tests/chrome-tool-capabilities.test.ts:68) — tabs and tab groups, windows, downloads, history, cookies (names and metadata only — values are never returned to the model), bookmarks, reading list, content settings, MHTML capture, network rules, extension management, privacy/proxy/font/power settings, TTS, and user scripts. All grant-gated; the extension can never act on itself.
• Live bounded lazy tool provider — every run receives exactly four fixed protocol tools: search_tools, list_tools, execute_tool, and run_pipeline (extension/lib/lazy-tool-wire.js:21), regardless of how large the catalog is, so provider context stays constant. Search derives bounded metadata from the live built-in, browser, management and WebMCP sources and returns expiring single-use references bound to run/task/agent/origin/document/generation; it never grants, approves, installs or executes. Execute accepts only a returned reference and revalidates catalog, source, capability, permission, grant, enrollment, document, run, expiry and replay fences before validation, before dispatch, and after dispatch.
• 60 bundled Wasm tools — 60 single-tool packages (including sed, jq, oxipng, jxl, avif, zxing, compressops, imageops, hashwasm-blake3, sqlite3, gzip, csvtool, diff, patch, awk, date, and core Unix utilities). Each ships with an exact manifest, CAS digest, SBOM and licence record, verified at build time by a bounded raw import/memory scan (build.mjs:108, extension/wasm/manifests/).
• Your WebAssembly files — Settings → WebAssembly files stores owner-selected files locally with a name and description. Upload size and file count have no product-imposed caps. Files stream into digest-keyed OPFS storage; identical bytes update the existing labels, while different files may share a name. Storage accepts files without validating that they are executable Wasm. Once a module's digest is allowed for an agent, user-wasm tools are adapted into the callable catalog and execute in a fresh wasm-execution-worker.js through the offscreen host, with a 15-second deadline. See user-uploaded WebAssembly storage and execution.
• Usage + cost accounting — per-call token/cost records against the bundled llm-prices table, aggregated per run and per agent. Providers that don't report usage are recorded as unknown, never faked.
• Observability — npm run build produces a debug build with source maps and a namespaced, levelled, timed logger plus performance marks across grants, every tool call, model round-trips and task loading. Redacted: no prompt or page content ever appears. Security assertions are identical in both builds.
• Transparency — an error console (full error detail + copy) and a security shield (CSP/permission state) in the hub.

Security model

• Four permissions are required at install (alarms, offscreen, sidePanel, storage); every other capability permission (tabs, scripting, bookmarks, … — 31 in all) is optional and requested on a user gesture at the moment of need — never silently.
• Host access is install-granted: the manifest declares host_permissions: ["<all_urls>"] and two content scripts on every http(s) page, so the extension can read every page in order to notice when a site offers WebMCP tools; it acts on a site only after you allow it. This is a deliberate, settled posture for a private tool (owner decision Q18, 2026-08-31, option (a) in docs/OPEN-QUESTIONS.md): the broad read is what lets the hub notice tools passively, and every mutation still waits on your grant. Screenshots go via captureVisibleTab/activeTab.
• One privacy page — Settings → About → "What this extension sends and stores" (extension/privacy/privacy.html) lists every host a request can go to (derived from the provider presets), every storage class the factory reset wipes (with what each holds, including that the provider key is stored unencrypted in the profile), the host-access posture above, and how to remove it all. Its lists are built from the same constants the code runs on, and tests/privacy-statement.test.ts fails when they drift.
• No debugger. It was re-declared as an optional permission at 0.2.286 for the CDP power tools and removed again on 2026-08-27 (owner decision): it carries Chrome's all-sites permission warning and a persistent "started debugging this browser" bar. The four CDP tools went with it; the browser-tool count is 139 (191 capability rows total). tests/chrome-tools-t12.test.ts guards the removal, so bringing it back has to be a deliberate act rather than a side effect of the next tool tranche.
• Origin-keyed OPFS — one agent/origin can never read another's memory.
• The standing security suite (npm run test:security) acquires the canonical Chrome lock, supervises one fresh exact profile under a hard timeout, and proves network exfiltration, sandbox escapes, and prompt-injection → destructive-tool are all blocked. Do not execute scripts/security-suite.ts directly.
• Every change is reviewed, and the review is labelled for what it was (Paul, 2026-08-27): independent review when a fresh session read only the diff, author review otherwise — and an author review must clear the falsification gates (a changed test is shown RED then GREEN, a fix is shown to fix in a real loaded extension, deleted coverage leaves a guard, the full suite is green at the tip). See AGENTS.md, "Review without a second model".

See docs/CONSTITUTION.md for the full security/accessibility/design/memory constraints.

Load + run

npm install
npm run build          # bundles the AI SDK + zod into the service worker
npm test               # unit tests
npm run test:chrome    # the CDP journeys (drives the real extension)
npm run test:components  # the component gallery smoke
npm run test:security  # the sandbox-boundary security suite
npm run package        # fresh exact-inventory Store production ZIP
npm run package:store  # same exact Store boundary (explicit alias)

Dependency order (deno install runs LAST). build.mjs resolves the AI SDK and @modelcontextprotocol/sdk from the Deno peer-context store under node_modules/.deno/, so deno install must run AFTER npm install/npm ci:

npm ci && deno install && npm run build

Running npm ci/npm install AFTER deno install flattens node_modules/zod into a real directory while a stale node_modules/.deno store survives; deno install alone then cannot re-link the flattened zod, and the build fails with cap-ai-sdk-dedup: no @modelcontextprotocol/sdk instance is bound to the extension's zod. If you hit that, recover with rm -rf node_modules && npm ci && deno install (deno install LAST).

Chrome journey CDP timeouts retain their 15-second deadline and method prefix. The error also identifies the request number and available target/session IDs; only Chrome's 32-hex-character ID format is printed, otherwise unavailable. Target IDs are reported when supplied on the request, not inferred from a session. URLs, expressions and request payloads are not included. This aids diagnosis; it does not change the gate's verdict classification.

Then in Chrome: chrome://extensions → enable Developer mode → Load unpacked → select the extension/ directory. The new-tab page becomes the hub.

Demo: a site as a sub-agent, in under a minute

deno run -A fixtures/webmcp-server.ts     # serves the Showcase Shop at http://127.0.0.1:8934/shop

1. Open http://127.0.0.1:8934/shop in a tab — a small shop that declares five WebMCP tools (list_products, search_products, add_to_cart, remove_from_cart, cart_total) and shows its cart.
2. Open a new tab (the hub). Above the composer: "127.0.0.1:8934 offers 5 tools — use them?" Nothing has been granted yet.
3. Click it. That one click grants scripting plus that exact origin and adds the site as a Site Agent; the status line names the origin and the chip reads "Using 127.0.0.1:8934 · 5 tools".
4. Type @, choose the site, then ask: add the cheapest widget to my cart and tell me the total (with a real model). The task is routed to that site's own sub-agent, whose tools are the page's tools. With Settings → Developer features on (the deterministic demo model), append @demo-site-tool add_to_cart {"sku":"widget-basic"} to make it call that tool with no API key. The shop's cart changes and the transcript's tool card shows add_to_cart.

npm run test:webmcp drives exactly this on a fresh profile (chip within 3 s, one-click grant naming the origin, the page changing, under 60 s end to end, and the same path after a service-worker restart) and keeps the screenshots (showcase-chip.png, showcase-grant.png, showcase-cart-changed.png).

Production packaging never copies the local extension/ tree wholesale. It combines Git-tracked regular files with the current generated dist and the byte-identical generated changelog, rejects symlinks/special files, verifies a fresh ZIP's exact names and hashes, then atomically replaces the final archive. dist.complete v2 is canonical JSON bound to the Git commit, current indexed source bytes, the exact hash + size of EVERY generated bundle (service worker, options, NTP, side panel, diff-core, agent worker), and the declared store target; lock owners, PIDs, stage paths and wall-clock timestamps remain build custody and never enter package bytes. The target is an intent/mismatch gate, not proof of content. The package step revalidates the marker around inventory hashing, then independently scans the actual tracked and generated package bytes. Identical source builds therefore produce byte-identical markers and ZIPs, while ignored local bundles, stale/legacy/cross-target markers, and files removed since an older ZIP cannot survive.

Store packaging uses the same inventory, content hashes, ZIP construction, and atomic replacement without transforming bytes. The explicit --target=store boundary additionally requires the exact MV3 extension CSP, the bundled-reviewed- only lane, zero unmanifested Wasm, no alternate Worker literal, and no statically visible remote script/code-loading URL. Computed and simple aliased Worker, SharedWorker, importScripts, and remote JavaScript fetch sinks are rejected. The manifest sandbox evaluator alone has an exact-path exemption; generated service- worker/options bundles do not. These scans are bounded heuristic defense in depth; exact CSP, package hashes, marker bindings, and archive verification remain mandatory. Static PASS is not Chrome Web Store policy approval.

Architecture

extension/
  manifest.json                 MV3 manifest (install-granted permissions; <all_urls>)
  background/service-worker.js  the router + agent core (bundled → dist/)
  ntp/                          the agent hub (new-tab page — the command center)
  sidepanel/                    the driven-page surface (chrome.sidePanel)
  options/                      the settings (providers, agents, permissions, hooks,
                                advanced system prompts, usage, data & memory)
  artifacts/                    the artifact gallery
  content/                      passive WebMCP detector + enrolled invocation bridge (MAIN + isolated)
  shared/                       the single-source Web Components (the design system)
  lib/                          provider, memory, named-agents, skills, hooks, artifacts,
                                tool-catalog/search/selection shadow authorities,
                                script-host, management-tools, scheduler, system-prompts, ...

System prompts (Settings → Advanced)

Every system prompt the platform sends is composed by ONE authority (extension/lib/system-prompts.js — see docs/SYSTEM-PROMPTS.md): the versioned built-in base + the owner's per-scope customization (append/prepend/replace)

• the agent role + the per-run skills + the immutable protected runtime policy (lib/runtime-policy.js, always the FINAL layer). Hub, named-agent, background, scheduled, hook, and site-worker runs all resolve through it, so the Settings → Advanced preview is the exact platform composition a run is built with — and every run records a run-bound attestation (the SHA-256 digest + UTF-8 bytes of the exact provider-bound system message, keyed receipts, never content) proving the wire message embeds that composition. A product update to a built-in prompt never silently overwrites a customization — the UI flags it with an old-vs-new diff and explicit keep/edit/reset choices.

The design system

20+ reusable Web Components (the single source extension/shared/components.js), testable in isolation in the component gallery — https://paulkinlan.github.io/chrome-agent-platform/components.html — which imports the same file (a drift guard fails the build on divergence). See docs/DESIGN.md + PRODUCT.md for the design tokens (the paper/teal palette).

Models

lib/provider.js is the pluggable model layer — OpenAI-compatible endpoints (OpenAI, Anthropic, Gemini, DeepSeek, Ollama), each with a per-provider model dropdown + a Test connection button. Per-agent provider override is supported. The model list + the usage pricing come from the bundled llm-prices.com table (cost tracking + spending limits work out of the box).

The plan, the history, and the remaining work

This README is the overview. The document map, in precedence order:

Current gate status: build clean (store bundle ceilings on every surface — SW ≤ 3.0 MB, agent worker ≤ 2.0 MB, options ≤ 880 kB, NTP ≤ 870 kB, side panel ≤ 660 kB, diff-core ≤ 17 kB) · full test suite green (npm test per-file runner over 445+ test files, 4100+ tests) · Chrome journeys 127/127 · security suite PASS. The journey suite had been red at 26/127 from 0.2.313 until 0.2.320; the causes are at the top of PLAN.md.