fiji-mcp

https://github.com/user-attachments/assets/2933af17-6cb6-4a4c-a7e8-6c1391c49db7

Prior art & related projects — several other projects bridge bioimage tools with LLMs via MCP. This project was developed independently but shares the same problem space:

• fiji/fiji-llm project - Chat with AI assistants directly in Fiji to get help with image analysis, scripting, macros, and general questions.
• fiji_mcp (NicoKiaru) — Fiji MCP via PyImageJ (in-process, Groovy)
• fiji-mcp-bridge (kusumotok) — Fiji MCP via TCP sockets, curated tool wrappers
• napari-mcp (royerlab) — napari MCP server, PyPI published
• bioimage-mcp (cqian89) — artifact-based I/O, isolated conda envs
• bioimage-mcp-server (Ichoran) — Java MCP for Bio-Formats
• BioImage-Agent (LLNL) — napari plugin over MCP via sockets
• cellpose_mcp — Cellpose segmentation MCP

An MCP server that gives LLM agents (Claude Code, Claude Desktop, Cursor, etc.) a live scripting interface into a running Fiji instance via WebSocket.

LLM Agent  --(MCP/stdio)-->  Python MCP Server  --(WebSocket)-->  Fiji + Java Plugin

What it does

• Run macros and scripts in Fiji from your LLM conversation (run_ij_macro, run_script, run_command)
• Inspect images — list open images, get metadata, save to disk
• Visual feedback via thumbnails — get_thumbnail returns a display-ready PNG with the current LUT and overlays baked in, so the LLM can see what it's working with
• Observe user actions — the plugin captures what the biologist does in Fiji's GUI (commands, image opens/closes, ROI edits) and streams it to the LLM via an event log
• Export workflows — convert a recorded action sequence to a runnable ImageJ macro
• Search commands — find Fiji menu commands by name

Thumbnails and visual models

This project is designed for use with vision-capable LLMs (Claude, GPT-4o, Gemini, etc.). The get_thumbnail tool is the primary way the agent sees image data — it returns a scaled PNG with the current LUT, brightness/contrast, and overlays already applied, exactly as the image appears in Fiji.

Agents should call get_thumbnail liberally: after opening an image, after each processing step, and before reporting results. This gives the model visual ground truth to catch mistakes (wrong channel, bad threshold, off-target ROI) that text-only metadata would miss. The thumbnail is file-path-based — no base64 in the protocol — so it works efficiently even for large images.

Available tools

Requirements

• Fiji (any version with Java 8+)
• Python 3.11+
• uv (Python package manager)

Installation

1. Install fiji-mcp

git clone https://github.com/SteffenPL/fiji-mcp.git
cd fiji-mcp
uv sync

2. Install the bridge plugin into Fiji

# Auto-discovers Fiji in standard locations:
uv run fiji-mcp install

# Or specify your Fiji path explicitly:
uv run fiji-mcp install --fiji-home /path/to/Fiji.app

This copies the bridge plugin JAR into your Fiji's plugins/ directory, checks the bundled Java version, and saves the Fiji path so the MCP server can find it at runtime.

3. Configure your MCP client

Add fiji-mcp to your MCP client.

Claude Code

Project scope (run from your project root):

claude mcp add --scope project fiji-mcp -- \
  uv run --directory /absolute/path/to/fiji-mcp fiji-mcp

Global scope:

claude mcp add --scope user fiji-mcp -- \
  uv run --directory /absolute/path/to/fiji-mcp fiji-mcp

Codex CLI / Codex app

In the Codex app, use the Show Terminal button and run the same commands below.

Global scope:

codex mcp add fiji-mcp -- \
  uv run --directory /absolute/path/to/fiji-mcp fiji-mcp

For project scope, use a local .codex/config.toml file as shown below.

Optional: config files

Claude config

{
  "mcpServers": {
    "fiji-mcp": {
      "command": "uv",
      "args": ["run", "--directory", "/absolute/path/to/fiji-mcp", "fiji-mcp"]
    }
  }
}

Use this JSON either in a project .mcp.json for Claude Code, or in Claude Desktop's config file at ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows).

Codex project or global config

[mcp_servers.fiji-mcp]
command = "uv"
args = ["run", "--directory", "/absolute/path/to/fiji-mcp", "fiji-mcp"]

Use this block in .codex/config.toml for a project-only install, or in ~/.codex/config.toml for a global install.

To override the saved Fiji path at runtime, pass --fiji-home:

uv run --directory /absolute/path/to/fiji-mcp fiji-mcp --fiji-home /other/Fiji.app

Usage

The first tool call auto-launches Fiji if it's not already running. You can also start Fiji manually and go to Plugins > fiji-mcp > Start Bridge.

Example conversation:

You: Open the sample image "blobs.gif" and apply a Gaussian blur with sigma 2

Claude: (uses run_ij_macro with code open("http://imagej.net/images/blobs.gif"); run("Gaussian Blur...", "sigma=2");)

Configuration

Architecture

Two processes connected by WebSocket:

• fiji-mcp (Python) — MCP server using fastmcp 3.x with stdio transport. Connects to Fiji as a WebSocket client. Buffers events in a 10,000-entry ring buffer.
• fiji-mcp-bridge (Java) — Fiji plugin. Runs a WebSocket server inside Fiji. Executes macros/scripts, captures user actions via ImageJ listeners, and pushes events to the MCP server.

Image data is file-based only (Fiji saves to temp directory, MCP server returns file paths). No base64 encoding in the protocol.

Event capture

The plugin captures user and MCP-triggered actions across 7 categories:

Every event is tagged with "source": "user" or "source": "mcp" so the LLM can distinguish what the biologist did from what it triggered itself.

Development

# Install dev dependencies
uv sync --dev

# Run Python tests (no Fiji required)
uv run pytest -v

# Run smoke test (requires Fiji with bridge started)
uv run python tests/smoke_test.py

# Build the Java plugin from source (requires Maven + JDK 8+)
cd fiji-plugin && mvn package -q
# Copy to package data:
cp target/fiji-mcp-bridge-0.1.0.jar ../src/fiji_mcp/data/

License

MIT