About Crawl Census
Crawl Census is an MCP server in the Scraping category: ask before you fetch: will this domain serve your crawler, refuse it, or charge it?.
Install
git clone https://github.com/taylorsmithgg/crawl-census-clientThis server has no ConduID identity, so agent calls to it are not receipted. Pin the version you install and review the source before granting it credentials.
Tools (6)
agent_profilewhat does this census publish about my crawler, and how do I correct it?
Parameters not published.
census_factsthe headline findings as dated records with denominators and citation lines
Parameters not published.
census_statscorpus-level totals
Parameters not published.
crawl_preflightwill these domains serve my agent, refuse it, or charge it?
Parameters not published.
scan_sitemeasure a domain now
Parameters not published.
site_reportthe stored audit for one domain
Parameters not published.
Ask AI
Ask AI about Crawl Census
Powered by Claude · Grounded in docs
Security checks
6 checks have not been run yet.