- Conduid
- Marketplace
- #crawler
MCP servers tagged crawler
75 live MCP servers tagged "crawler", ranked by trust score. Tags come from package metadata and repository topics, so this list covers servers across every category that work with crawler.
Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here f…
MCP Router
MCP Router — development, support and security updates ended 2026-09-18. Historical source and final release for existing users.
Web Scout MCP
An MCP server providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling A…
just-every/mcp-read-website-fast
Quickly reads webpages and converts to markdown for fast, token efficient web scraping
Afdiantomarkdown
爱发电爬虫(afdian.com)
AI Gov Content Curator
💡An end-to-end solution for aggregating, summarizing, and displaying news articles using an AI-powered backend, an automated CRON crawler & newsletter emailer, and a responsive Ne…
Webclaw
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
Douyin TikTok Download API
🚀 Self-hosted TikTok & Douyin scraper and no-watermark video downloader — async REST API, MCP server, CLI and web console for posts, profiles, comments and playlists. Self-healing…
Crw
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search…
Drissionpage MCP Server
DrissionPage MCP Server · Browser automation for Claude Code, Codex, and MCP clients
Supadata
Official Supadata MCP Server - Adds powerful video & web scraping to Cursor, Claude and any other LLM clients.
Tools
First CH Tools の計算ロジック（WCAGコントラスト比・文字数/Xウェイト・WebP変換・JSON-LD生成・llms.txt生成・文字コード/改行コード変換・Marp Markdown→スライド レンダリング・テストデータ生成・テキスト/コード差分・Cron式の解説・Base64/Data URI変換・URLパラメータの分解/再構築・HTM…
Maxun
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
Fess
Open-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud sources, 20+ languages, REST API, and AI/RAG & semantic search. Apache…
Mcpdocsearch
This project provides a toolset to crawl websites wikis, tool/library documentions and generate Markdown documentation, and make that documentation searchable via a Model Context…
Parse Video
Golang短视频去水印：抖音,皮皮虾,火山,微视,最右,快手,全民小视频,皮皮搞笑,西瓜视频,虎牙,梨视频,acfun,好看视频...
TikHub API Python SDK
High-performance asynchronous Douyin(抖音) TikTok Xiaohongshu(小红书) Kuaishou(快手) Weibo(微博) Instagram YouTube(油管) Twitter(X) Captcha Solver(验证码解决器) Temp Mail(临时邮箱) API(接口).
serkan-ozal/driflyte-mcp-server
The Driflyte MCP Server exposes tools that allow AI assistants to query and retrieve topic-specific knowledge from recursively crawled and indexed web pages.
0xMassi/webclaw
configures Claude, Cursor, Windsurf, Codex, OpenCode.
WebReaper
AI-native web scraper. Single binary with a bundled Claude Code skill. MIT-licensed alternative to Firecrawl.
Requests
Web Content Retrieval (full webpage, filtered content, or Markdown-converted), Custom User-Agent, Multi-HTTP Method Support (GET/POST/PUT/DELETE/PATCH), LLM-Controlled Request Hea…
Anansi
Scrape data from unstable websites using automated self-healing scrapers, TLS fingerprinting, and Pydantic validation to ensure reliable extraction.
Crawlie
Scan websites for technical SEO and GEO issues using this local crawler, CLI tool, and MCP server.
Browserpaw
BrowserClaw - Control your everyday Chrome browser from AI agents, without losing logins or focus
BrowserClaw
BrowserClaw - High-efficiency Chrome browser automation MCP server
ByteCrawl
The focused-crawling MCP server for AI agents. Rank a whole site by your topic, read any page as clean Markdown, extract anything
SouWen
Unified search, crawling, and archiving toolbox for AI agents and automation scripts.
CNKICrawlerMCP
基于GO爬虫的知网MCP工具-知网MCP-CNKI-MCP
Agent Lighthouse
Lighthouse-style audits for AI-agent readiness, LLM crawlers, MCP clients, and the agentic web
Maw
Crawl any website into a single searchable file. Query it forever, offline.
Cog
An MCP integerated intelligent RAG that gives relevent context to LLM's through crawled Docs
Crawl
AI-powered web crawler — self-hosted Firecrawl alternative. Crawl, extract, render JS, search. CLI + MCP + REST API + Dashboard.
Clado AI
Clado Model Context Protocol Server
Wigolo
Local-first web intelligence MCP server for AI coding agents
Turbo Surf
Native-speed, browserless web crawler + MCP server for AI agents. A standalone Rust engine (turbo-dom + a V8 JS-render tier) — fetch + parse + extract + run page JS with no headle…
Web Docs
MCP server for crawling and indexing web documentation - works with any website
Crawlforge
Open source alternative to Screaming Frog SEO Spider — LLM-native, MCP server included, 269 rules, DuckDB+Parquet storage, MIT licensed.
Scrapling Rs
Adaptive web scraping, built in Rust. A high-performance port of Python Scrapling.
Writ Agent
The Rust fleet agent for Writ — connects to your self-hosted coordinator and does the browsing.
AI Crawler
AI-driven network-first crawler compiler for authorized workflows
Better Crawler4agent
When I was using agents, I found that most agents rely on curl commands to scrape web pages, which has a very low success rate and is slow, so I made this.
Anti Bot
Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scale up to concurrent, multi-session crawls. 🕷️ An adaptive Web…
Spartan Scraper
Local-first scraping workbench for scrape, crawl, and research workflows
Seoforge
Screaming Frog-grade SEO/AEO crawler, site audit engine & Model Context Protocol (MCP) server
Synoppy MCP
Synoppy MCP server — give your AI agent the whole web. Read, crawl, map, extract, classify & enrich live pages from Claude, Cursor, or any MCP client.
Olostep CLI
CLI for the Olostep API — scrape, map, crawl, answer, batch the web from your terminal. Pure JS rewrite of olostep-cli.
Deepspider
智能爬虫工程平台 - 基于 DeepAgents + Patchright 的 AI 爬虫 Agent | Intelligent Web Scraping Platform - AI-powered Crawler Agent built on DeepAgents + Patchright
Cockroach Crawler
Give AI agents bounded eyes on the public web: crawl, search, and normalize evidence without an unrestricted browser.
Pbx Estimation
PBX Market Estimation & VoIP Trend Forecasting
Scrapo
AI-native, agent-first web scraping for Python. Cost-aware tiered fetching (HTTP, browser, stealth, agent), model-agnostic LLM extraction with self-healing selector caches, API-fi…