1. Conduid
  2. Scraping
  3. webcrawl-mcp
MCP server · Scraping

webcrawl-mcp

· github.com/andyliszewski/webcrawl-mcp

Lightweight MCP server for web scraping, search, and crawling. Uses local trafilatura/DuckDuckGo by default with optional Firecrawl fallback for transport-blocked pages.

Unclaimed scraping
InstallView source
34Low

Scored 5 months ago · breakdown

About webcrawl-mcp

webcrawl-mcp is an MCP server in the Scraping category: lightweight MCP server for web scraping, search, and crawling. Uses local trafilatura/DuckDuckGo by default with optional Firecrawl fallback for transport-blocked pages.

Install

From source
git clone https://github.com/andyliszewski/webcrawl-mcp

This server has no ConduID identity, so agent calls to it are not receipted. Pin the version you install and review the source before granting it credentials.

Tools (4)

webcrawl_crawlBFS crawl multiple pages (each result includes provenance)

Parameters not published.

webcrawl_mapDiscover same-domain URLs from a starting page

Parameters not published.

webcrawl_scrapeFetch a single URL → `{content, source}` (markdown + provenance)

Parameters not published.

webcrawl_searchDuckDuckGo search (optionally scrape results, each with provenance)

Parameters not published.

Ask AI

✦

Ask AI about webcrawl-mcp

Powered by Claude · Grounded in docs

I know everything about webcrawl-mcp. Ask me about installation, configuration, usage, or troubleshooting.

Security checks

6 checks have not been run yet.

Questions

About webcrawl-mcp

How do I install webcrawl-mcp?

No package has been published for this server, so install it from source: clone the repository and follow its README.

Is webcrawl-mcp safe to use with an AI agent?

Its trust score is 34 out of 100 (low). Conduid hasn't run static security checks on this repository yet, so review the source yourself before granting it credentials. It has no ConduID identity yet, so agent calls to it are not receipted.

Is webcrawl-mcp still maintained?

Conduid hasn't recorded a commit date for this repository yet. Check the repository directly for recent activity.