Fetch URL MCP Server

An MCP server that fetches web pages and converts them to clean, readable Markdown.

Overview

This server takes a URL, fetches the page, and strips away everything you don't need — navigation, sidebars, banners, scripts — leaving just the main content as Markdown. It's perfect for feeding into LLMs, giving them the distilled essence of a page without the noise. It also recognizes GitHub, GitLab, Bitbucket, and Gist URLs and rewrites them to fetch the raw content directly.

By default it runs over stdio. Pass --http if you need a proper HTTP endpoint with auth, rate limiting, TLS, and session support.

Key Features

• HTML to Markdown — Turns any public web page into clean, readable Markdown with metadata like title, url, contentSize, and truncated.
• Smart URL handling — Recognizes GitHub, GitLab, Bitbucket, and Gist page URLs and rewrites them to raw-content endpoints before fetching.
• Task mode — Big or slow pages can run as async MCP tasks with progress updates, instead of blocking. In HTTP mode, tasks are bound to the authenticated caller rather than a single MCP session, so they can be resumed after reconnecting as the same authenticated subject. Polling task state exposes task summaries; numeric progress remains out-of-band via notifications/progress.
• Self-documenting — Includes an internal://instructions resource and a get-help prompt so clients know how to use it.
• HTTP mode — Optionally serves over Streamable HTTP with host/origin validation, bearer or OAuth auth, rate limiting, health checks, and TLS.

Web Client

A browser-based client is available if you want to use the server without any MCP setup.

Live app · Source code

Requirements

• Node.js >= 24
• Docker (optional) — only needed if you want to run the container image

Quick Start

Add this to your MCP client config:

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

Client Configuration

{
  "servers": {
    "fetch-url-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

code --add-mcp '{"name":"fetch-url-mcp","command":"npx","args":["-y","@j0hanz/fetch-url-mcp@latest"]}'

{
  "servers": {
    "fetch-url-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

code-insiders --add-mcp '{"name":"fetch-url-mcp","command":"npx","args":["-y","@j0hanz/fetch-url-mcp@latest"]}'

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "servers": {
    "fetch-url-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

extensions:
  fetch-url-mcp:
    name: fetch-url-mcp
    cmd: npx
    args: ['-y', '@j0hanz/fetch-url-mcp@latest']
    enabled: true
    type: stdio
    timeout: 300

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

claude mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"],
      "env": {}
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "amp.mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

amp mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

codex mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest

[mcp_servers.fetch-url-mcp]
command = "npx"
args = ["-y", "@j0hanz/fetch-url-mcp@latest"]

{
  "servers": {
    "fetch-url-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

{
  "context_servers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"],
      "env": {}
    }
  }
}

{
  "mcpServers": {
    "fetch-url-mcp": {
      "command": "npx",
      "args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
    }
  }
}

Use Cases

• Documentation for LLMs — Grab a docs page, blog post, or reference article as Markdown and pass it straight into a context window.
• Repository content — Hand it a GitHub, GitLab, or Bitbucket URL and it resolves the raw content endpoint. Works with Gists too.
• Slow or large pages — Task mode lets big fetches run in the background while sending monotonic progress updates back to the client, while tasks/get exposes the latest task summary fields such as statusMessage, createdAt, lastUpdatedAt, ttl, and pollInterval.

Architecture

[MCP Client]
  ├─ stdio -> `src/index.ts` -> `startStdioServer()` -> `createMcpServer()`
  └─ HTTP (`--http`) -> `src/index.ts` -> `startHttpServer()` -> HTTP dispatcher
       ├─ `GET /health`
       ├─ `GET /.well-known/oauth-protected-resource`
       ├─ `GET /.well-known/oauth-protected-resource/mcp`
       └─ `POST|GET|DELETE /mcp`

`createMcpServer()`
  ├─ registers tool: `fetch-url`
  ├─ registers prompt: `get-help`
  ├─ registers resources:
  │    - `internal://instructions`
  ├─ enables capabilities: logging, resources, prompts, tasks
  └─ installs task handlers, log-level handling, and shutdown cleanup

`fetch-url` execution
  ├─ validate input with `fetchUrlInputSchema`
  ├─ normalize URL and block local/private targets unless allowed
  ├─ rewrite supported code-host URLs to raw endpoints when possible
  ├─ fetch content via the shared pipeline
  ├─ transform HTML into Markdown in the transform worker path
  └─ validate `structuredContent` with `fetchUrlOutputSchema`

Request Lifecycle

[Client] -- initialize {protocolVersion, capabilities} --> [Server]
[Server] -- {protocolVersion, capabilities, serverInfo} --> [Client]
[Client] -- notifications/initialized --> [Server]
[Client] -- tools/call {name, arguments} --> [Server]
[Server] -- {content: [{type, text}], structuredContent?, isError?} --> [Client]

MCP Surface

Tools

fetch-url

Takes a URL and returns Markdown. Read-only, with no JavaScript execution. Supports running as a background MCP task for large or slow pages. In task mode, tasks/get and tasks/list expose task summaries including status, statusMessage, createdAt, lastUpdatedAt, ttl, and pollInterval; numeric progress remains out-of-band via notifications/progress.

You get text content back by default. If output validation passes, the response also includes structuredContent with typed fields: url, resolvedUrl, finalUrl, title, metadata, markdown, fetchedAt, contentSize, and truncated. A true value for truncated means the content hit a server-side size limit.

To opt into progress updates, include _meta.progressToken in the tool call. The token may be a string or number, and the server may emit monotonic notifications/progress updates while the fetch runs.

To run the tool in task mode, include params.task = { ttl?: <ms> }. tasks/result waits until the task reaches a terminal status and then returns the stored output or a terminal error payload for cancelled or failed tasks. Task summaries and final results include _meta["io.modelcontextprotocol/related-task"] = { "taskId": "<server-task-id>" }.

{
  "method": "tools/call",
  "params": {
    "name": "fetch-url",
    "arguments": {
      "url": "https://example.com/docs"
    },
    "task": {
      "ttl": 300000,
      "pollInterval": 1000
    },
    "_meta": {
      "progressToken": 7
    }
  }
}

1. [Client] -- tools/call {name: "fetch-url", arguments} --> [Server]
2. [Server] -- dispatch("fetch-url") --> [src/tools/fetch-url.ts]
3. [Handler] -- validate(fetchUrlInputSchema) --> normalize / fetch / transform
4. [Handler] -- validate(fetchUrlOutputSchema) --> assemble content + structuredContent
5. [Server] -- result or tool error --> [Client]

Resources

Prompts

MCP Capabilities

Tool Annotations

Structured Output

The tool declares an outputSchema and includes structuredContent in the response when validation passes. Clients that support structured output get typed data directly; the rest use the text fallback.

Configuration

All configuration is through environment variables. For basic stdio usage, nothing needs to be set.

HTTP Server

Authentication & OAuth

TLS

Fetching

Tool Output

Tasks

Transform Workers

Content Cleanup

Logging

HTTP Endpoints

Security

Development

Essential Commands

Build and Release

• npm run prepublishOnly runs lint, type-check, and build as a single release gate.
• CI workflows are under .github/workflows/.
• Dockerfile and docker-compose.yml are included for containerized runs.
• Published on npm as @j0hanz/fetch-url-mcp.

Troubleshooting

Credits

Contributing and License

Pull requests welcome. Please make sure these pass before submitting:

1. npm run lint and npm run type-check
2. npm test
3. npm run format

License

MIT License. See LICENSE for details.