RustXNG

日本語 · 한국어

RustXNG is a self-hosted metasearch engine written in Rust, based on SearXNG. It combines results from external search providers and can fetch page text for AI agents through a JSON API or MCP.

RustXNG does not maintain a web-scale search index. You can add a local index of selected sites with rustxng-index and import Wikipedia dumps separately.

Quick start

Install from the releases page or with Cargo. Release binaries are available for Linux, macOS and Windows; check the matching asset and its SHA-256 sidecar before running it.

cargo install --git https://github.com/sahenjp/RustXNG.git --tag v0.31.2 --locked

Rust 1.89 or newer and a C build toolchain are required for source installation. The agent setup guide covers release binaries, verification, platform requirements and service reuse.

Start a local server:

rustxng --bind 127.0.0.1 --port 29483

Search from another terminal:

curl --fail --get 'http://127.0.0.1:29483/search' \
  --data-urlencode 'q=tokio runtime documentation' \
  --data 'format=json' --data 'max_results=5'

Results contain a title, URL, snippet and source metadata. For automated clients that need errors for unknown engine names, add strict_engines=1 (the default preserves SearXNG-compatible fallback behavior). Useful optional parameters include language=ja, engines=wikipedia, include_domains=docs.rs, exclude_domains=example.com, time_range=day, and include_raw_content=1. Time filtering depends on provider support.

The web interface is at http://127.0.0.1:29483/. The search endpoint also supports CSV, RSS and HTML output.

Use from an AI agent

If the agent is expected to install RustXNG itself, the agent setup guide is written for exactly that: it covers release-binary selection and checksum verification, starting a loopback service, verifying a small search, and the record to report back.

Start the built-in MCP server:

rustxng mcp

MCP exposes two read-only tools:

• search returns up to 10 ranked results. The default is 5, with snippets limited to 500 characters.
• read_page retrieves text from a URL returned by search in the same session. It can return up to three passages relevant to an optional query, selected from the extracted page.

read_page returns 4,000 characters by default and allows max_chars up to 20,000. Use offset and next_offset to continue. truncated and source_truncated indicate limits. MCP enforces serialized character and byte budgets. Supply both tokenizer (cl100k_base or o200k_base) and max_payload_tokens for an exact budget on serialized MCP structuredContent; this does not include the JSON-RPC envelope or client/model framing. Output contract

Page fetching checks destinations and redirects for non-public addresses and follows robots.txt (RFC 9309). It does not accept arbitrary URLs through MCP. Treat all returned web text as untrusted content.

Use accept_language, for example "ko-KR,ko;q=0.9,en;q=0.8", to request a page-language preference in MCP read_page or HTTP include_raw_content. Omission sends no language header; each preference has its own cached page. See validation, providers, and pagination.

For page text, include_raw_content=1 uses the HTTP API. PDF text extraction requires Poppler (pdftotext); scanned PDFs require a separate OCR tool. JavaScript-rendered pages need the optional Firefox renderer. Neither is enabled automatically by a standalone release binary.

With include_raw_content=1, every result reports raw_content_status (ok, not_requested, too_large, fetch_failed, robots_blocked, extract_failed, render_failed, or unknown) and, where available, a short raw_content_error message, so a client can decide whether to retry or give up from the response alone. The page body limit is configurable through raw_content_max_bytes (default 4 MiB, bounded at 32 MiB), and a failed page renderer keeps the static text by default (page_renderer_fallback = "static").

Search profiles

Choose a provider configuration with --profile. The default is standard.

Standard's shipped entries are documented APIs, RSS or JSON endpoints — no human-facing search-result HTML pages are fetched by default — and each entry's reviewed conditions are recorded in docs/PROVIDERS.md. Full adds HTML providers, including sources whose terms forbid automated access; starting it now prints a terms reminder at startup.

Profile	Command	Sources
Standard	rustxng	34 keyless entries with recorded endpoint/use conditions, plus local indexes
Clean	rustxng --profile clean	Search only prebuilt local crawl and Wikipedia-dump indexes; no external search-provider requests
Full	rustxng --profile full	Additional HTML providers, including sources with restrictive or unverified automation terms, plus billed Exa Search (skipped at startup when EXA_API_KEY is absent)

Full is not a permission bypass. Do not enable a provider without the permissions required for your use. A private instance does not override provider terms. Clean is a configuration choice, not a certification of legal compliance.

The selected profile also works with MCP, for example rustxng mcp --profile clean. Standard and Full send queries to their enabled external providers even when RustXNG runs locally. Clean searches local indexes only, but explicitly requested raw page fetching or other network-backed features can still make outbound requests. Full's Exa integration is opt-in and billed; an API key alone does not enable Exa in Standard.

Standard excludes unreviewed providers from its defaults. Reviews apply to the documented retrieval scope, not every commercial use, content licence, storage policy or redistribution scenario. See provider conditions and dated measurements.

Installation and configuration

A Docker image is available:

docker run --rm -p 127.0.0.1:29483:29483 \
  -e SEARXNG_PORT=29483 ghcr.io/sahenjp/rustxng:latest \
  --bind 0.0.0.0 --port 29483

By default, the service binds to loopback. Exposing it to another network requires appropriate access controls.

Settings are embedded. To override a value, create a small my-settings.toml, for example:

[server]
idle_timeout = 600

rustxng --config my-settings.toml

A custom settings file is layered over the selected profile; avoid copying all shipped settings or adding an unintended engine override. Use rustxng --check-update to check for a release or rustxng --update for a supported binary installation. For managed packages and containers, use the original installation method.

The /healthz endpoint checks the process, not the availability of upstream providers. Use a real JSON search to verify that results are returned. Runtime metrics are exposed at /stats and /metrics.

Search quality and limitations

The November 4 comparison covers older search profiles and is not a benchmark of v0.30.0. After provider permission changes, the November 5 Standard runs recorded Hit@5 of 67–69/100, down from the earlier configuration's 84/100 on the same documentation question set. The newer offline-only Clean profile reached 85/254 Hit@10 on SealQA with a 42.7 ms median in its recorded run. These are different datasets and cannot be ranked against each other.

All figures measure whether a reference page was retrieved, not the accuracy of generated AI answers. The question sets informed development, some indexes cover relevant documentation sites, and comparisons to stored commercial API results were not simultaneous or held out. These dated experiments are not guaranteed v0.30.0 performance. Do not infer general superiority over Exa or Tavily.

Every number is self-reported: one maintainer, one machine, one network. Reproduce them yourself with the commands in docs/BENCHMARK.md before relying on any figure.

Also note the structural difference behind any comparison to Exa or Tavily: RustXNG merges many independent sources into one ranked list, while those APIs return results from a single index. Merging itself raises hit rates, so the measured gap is not a like-for-like measure of per-source quality. Read the figures as an engineering record, not a head-to-head win.

Documentation

• Agent installation and verification
• Provider terms and supported configurations
• Local crawler and search index · Wikipedia index
• SearXNG compatibility · Historical benchmarks
• Security policy · Contributing · Release procedure

Contributing and license

Bug reports and contributions are welcome through Issues, Discussions and pull requests. See CONTRIBUTING.md, GOVERNANCE.md and MAINTAINERS.md.

RustXNG is licensed under AGPL-3.0-or-later. It builds on SearXNG's design and engine behavior; upstream and third-party attribution is preserved in NOTICE. A code license does not grant permission to collect or redistribute data from external search providers.