English | 中文

This project is heavily vibe-coded and not ready for production use. Breaking changes may happen. Use at your own risk.

docs-mcp-rs

A documentation indexing and search MCP (Model Context Protocol) server implemented in Rust. It can scrape, index documentation websites, and provide intelligent search capabilities to AI assistants via the MCP protocol.

Features

• Hybrid Search - Combines vector search and full-text search (FTS5) for more accurate results
• Intelligent Scraping - Supports headless browser rendering to scrape JavaScript-rendered documentation sites
• Multi-Version Management - Supports independent indexing and management of multiple versions of the same library
• MCP Protocol - Supports both HTTP and stdio transport modes, can be directly invoked by AI tools like Claude and Cursor
• Web Interface - Built-in Web UI with Chinese and English language support for easy management and search
• High Performance - Implemented in Rust with low memory footprint and fast speed
• Multiple Embedding Models - Supports OpenAI and Google embedding models with configurable custom API endpoints

Installation

Build from Source

git clone https://github.com/Eslzzyl/docs-mcp-rs.git
cd docs-mcp-rs
cargo build --release

The compiled binary is located at target/release/docs-mcp-rs.

Quick Start

1. Configure Environment Variables

Copy the example configuration file:

cp .env.example .env

Edit the .env file to configure your API keys:

# OpenAI API configuration
OPENAI_API_KEY=sk-xxxxxxx
OPENAI_API_BASE=https://api.openai.com/v1

# Embedding model (format: provider:model)
DOCS_MCP_EMBEDDING_MODEL=openai:text-embedding-3-small

2. Start the Server

HTTP mode (default):

docs-mcp-rs serve

The server will start at http://localhost:26301. Visit http://localhost:26301 to open the Web UI.

stdio mode (for MCP client integration):

docs-mcp-rs serve --stdio

3. Index Documentation

Use CLI to scrape documentation:

# Index React documentation
docs-mcp-rs scrape react https://react.dev --version 18

# Index Vue documentation with max pages limit
docs-mcp-rs scrape vue https://vuejs.org/guide --version 3 --max-pages 500

4. Search Documentation

# Search indexed documentation
docs-mcp-rs search react "how to use hooks"

# Search in a specific version
docs-mcp-rs search vue "computed properties" --version 3

CLI Commands

serve - Start MCP Server

docs-mcp-rs serve [OPTIONS]

Options:
  -p, --port <PORT>  HTTP server port [default: 26301]
      --stdio        Run in stdio mode

scrape - Scrape and Index Documentation

docs-mcp-rs scrape <LIBRARY> <URL> [OPTIONS]

Arguments:
  <LIBRARY>  Library name
  <URL>      Starting URL

Options:
  -v, --version <VERSION>      Version number
  -p, --max-pages <MAX_PAGES>  Maximum pages to scrape [default: 1000]
  -d, --max-depth <MAX_DEPTH>  Maximum crawl depth [default: 3]
  -c, --concurrency <NUM>      Concurrency number [default: 5]

search - Search Documentation

docs-mcp-rs search <LIBRARY> <QUERY> [OPTIONS]

Arguments:
  <LIBRARY>  Library name
  <QUERY>    Search query

Options:
  -v, --version <VERSION>  Version number
  -l, --limit <LIMIT>      Result limit [default: 5]

list - List Indexed Libraries

docs-mcp-rs list

remove - Remove Index

docs-mcp-rs remove <LIBRARY> [OPTIONS]

Arguments:
  <LIBRARY>  Library name

Options:
  -v, --version <VERSION>  Version number (removes entire library if not specified)

MCP Tools

This server provides the following MCP tools for AI assistants:

Environment Variables

Technical Architecture

• Web Framework: Axum
• Database: SQLite + sqlite-vec (vector extension)
• MCP Protocol: rmcp
• Embedding APIs: async-openai, reqwest
• HTML Parsing: scraper, fast_html2md
• Browser Automation: headless_chrome

Project Structure

docs-mcp-rs/
├── src/
│   ├── cli/          # Command-line argument parsing
│   ├── core/         # Core configuration and types
│   ├── embed/        # Embedding model integration
│   ├── events/       # Event bus
│   ├── mcp/          # MCP server implementation
│   ├── pipeline/     # Scraping pipeline management
│   ├── scraper/      # Web scraping
│   ├── splitter/     # Document chunking
│   ├── store/        # Data storage
│   └── web/          # Web interface
├── migrations/       # Database migrations
├── public/           # Static assets
└── data/             # Data directory

Related Projects

• docs-mcp-server - Original TypeScript/Node.js implementation

License

MIT License