LLMTEXT - Tools for the adoption of the llms.txt standard

Read the announcement post

Get started

1. Visit llmtext.com to install MCP servers from popular documentation sites, or paste any llms.txt URL to create your own.

2. Create your own llms.txt from your entire website (not just the docs) by using our extract-from-sitemap tool. See this template to easily create your own.

Libraries

• getassetmanifest: deploy static assets already present in your fs as a llms.txt.
• extract-from-sitemap: Generate llms.txt from a sitemap
• llms-txt-fetch: fetch all contents of a llms.txt
• llms-txt-generate: Use this in combination with getassetmanifest if you want to deploy static assets already present in your fs as a llms.txt.
• llms-txt-parse: parse llms.txt according to the standard
• llms-txt-validate: Check llms.txt validity

Services

Repo	Website	Description
explore-sitemaps	https://map.llmtext.com	crawls a website for rss, llms.txt, and sitemaps to find pages.
llms-txt-mcp	https://mcp.llmtext.com/{domain}/mcp	converts a llms.txt into a dedicated MCP server.
shared-crawler	https://crawl.llmtext.com	Crawls a website and stores results in DOs so anyone can access them cheaply.
llmtext		the website. Also exposes https://llmtext.com/%s where %s can be any url to get markdown for it. Depends on shared-crawler for that.
llmtext.login		X OAuth flow for the MCP
llmtext.reader		Converts HTML into simplified "browser-reader HTML", drastically improving token density

Other interesting repos

• markdown-renderer: Chrome/Safari Extension that renders markdown responses
• markdownbrowser
• openwebgate - shadow site principle

Sponsors