web-reader-mcp

MCP server that fetches URLs and extracts clean, readable content as Markdown or plain text. Built on the Model Context Protocol so AI assistants (Claude, Cline, OpenCode, etc.) can read web pages.

Features

• Extracts main article content from any URL using Readability
• Converts HTML to clean Markdown via Turndown
• Extracts metadata (title, author, publish date, Open Graph tags)
• Optionally extracts all links from the page
• Optional JavaScript rendering via Playwright for SPAs and JS-heavy sites
• Exposes a single webReader tool over MCP Streamable HTTP transport

Quick Start

bun install
bun run dev

The server starts on http://localhost:3001 (or PORT env var).

Tool: webReader

Example Response

{
  "title": "Article Title",
  "url": "https://example.com/article",
  "content": "# Article Title\n\nArticle content in markdown...",
  "excerpt": "A brief summary of the article.",
  "byline": "Author Name",
  "siteName": "Example",
  "publishedTime": "2025-01-15T10:00:00Z",
  "metadata": { "og:description": "..." }
}

MCP Client Configuration

Claude Desktop

{
  "mcpServers": {
    "web-reader": {
      "url": "http://localhost:3001/mcp"
    }
  }
}

Cline (VS Code)

{
  "mcpServers": {
    "web-reader": {
      "url": "http://localhost:3001/mcp",
      "transportType": "streamable-http"
    }
  }
}

JavaScript Rendering (Optional)

By default, pages are fetched with a simple HTTP client (got). For JavaScript-heavy sites (SPAs, React apps), enable Playwright:

bun add playwright
npx playwright install chromium

Then set render_js: true when calling the tool. Note that Playwright requires ~300–500 MB RAM per browser tab.

Scripts

Docker

docker compose up --build

Runs on port 3000 with health checks enabled.

Architecture

src/
├── index.ts              # Express server + MCP session management
├── types.ts              # Shared TypeScript interfaces
├── tools/
│   └── web-reader.ts     # Tool registration (Zod schema + handler)
└── lib/
    ├── fetcher.ts        # HTTP fetching (got)
    ├── fetcher-playwright.ts  # JS rendering (Playwright, optional)
    ├── parser.ts         # HTML → structured content (Readability)
    └── converter.ts      # HTML → Markdown (Turndown)

Tech Stack

• Runtime: Node.js (ES2022)
• Package Manager: Bun
• HTTP Server: Express
• MCP SDK: @modelcontextprotocol/sdk v1
• HTML Parsing: jsdom + Readability
• Markdown: Turndown
• HTTP Client: got
• JS Rendering: Playwright (optional)

License

MIT