- Conduid
- Marketplace
- #benchmark
MCP servers tagged benchmark
169 live MCP servers tagged "benchmark", ranked by trust score. Tags come from package metadata and repository topics, so this list covers servers across every category that work with benchmark.
AgentInjectionBench
A benchmark for evaluating prompt injection attacks in agentic tool-use pipelines
Ironharness
Safe I/O harness for LLM agents: serial, Modbus TCP, MQTT, file sandbox and ESP32 flashing behind an MCP server, with a JSONL audit journal - plus ironbench, a pass@k benchmark fo…
Al Buddy Memory
Portable, governed, model-agnostic memory for AI agents: facts invalidated never deleted, raw text as truth, one documented export format.
Mechapad CLI
Open Multi-Model AI Control Surface & Native MCP Server — Run Claude Fable 5.1, GPT-6 Astra, DeepSeek V4.1 Flash, and Gemini 3.8 in parallel
Sverklo Bench
bench:primitives — the public, reproducible benchmark for code-intelligence MCP servers. 90 tasks across 3 datasets, 5 baselines. Companion to github.com/sverklo/sverklo.
Copeca
Cost per correct answer — a neutral, reproducible, verifiable benchmark for CLI-based coding agents
1c AI Codegen Core
Вычислительное ядро платформы для оценки LLM генерации кода 1С:Предприятие (BSL). Артефакт научно-исследовательской работы для НДР-2026. Развитие проекта: github.com/genlab-1c/pri…
Companmem
the quest for ultimate ai memory
AnalogGym Opt
GRPO-based multi-objective analog circuit sizing on AnalogGym benchmarks (ngspice + SkyWater sky130), with LLM-agent integration via Claude Code skills and an MCP server. Companio…
Agent Capability Benchmark
Outcome-verified, vendor-neutral benchmark for agent capability infrastructure
Benchmarks
Evaluación comparativa de la precisión de Normativa frente a preguntas reales de exámenes y pruebas oficiales. Contiene los tests, las plantillas de referencia y las respuestas ge…
Jev Tool Search
Tool search for LLM agents: BM25 vs embeddings vs rerankers vs Jev on 525 real MCP tools, plus an experimental Jev search engine
Bounty Operator
Find the hole in your bug bounty report before the triager does. Pre-submission review for bounty reports and smart-contract findings, on your own model key.
Stoneburner
Local-first LLM eval: cost, quality, and security suites. Install stoneburner-atomics; CLI is atomics.
Grandpa
Grandpa makes AI coding agents work smarter, not harder. Zero-bloat architecture, 8 skills, CLI scanner, and benchmark harness for 25+ AI agents.
Decisionkit
Fast, cheap, probability-scored decisions for coding agents, scripts and CI. CLI + MCP server + Claude Code plugin for the OpenAI Decisions API and Jev (TypeSafe AI).
Eval
🧐 Privacy-preserving MCP server evaluation: deterministic CI probe battery (18 probe kinds, readiness score, SARIF) plus production friction capture with findings that verify and…
Nethack MCP
An MCP server that lets AI agents play NetHack. It spawns NetHack under a pseudo-terminal, parses the rendered screen into an LLM-friendly structured state, and exposes a small ac…
Nethack MCP
An MCP server that lets AI agents play NetHack. It spawns NetHack under a pseudo-terminal, parses the rendered screen into an LLM-friendly structured state, and exposes a small ac…
Adversarial Suite
🔒 Simulate adversarial behaviors to test and strengthen MCP defenses without real exploitation or risk, ensuring robust security evaluations.
Thetacog MCP
npx thetacog-mcp — a decidable, on-chip, LLM-free placement receipt a stranger recomputes byte-for-byte. Free for builders; patent-licensed for production & financial use (US Pate…
Rewind Bench
A benchmark separating repairs that passed from repairs that held up. Bugs are injected by AST mutation; every accepted repair is re-run against a hidden second dataset
Ground Truth Evals
An LLM eval harness graded by computed ground truth, not an LLM judge. Worked example: poker, served to models as MCP tools — gpt-4o-mini goes from 26/75 to 75/75 with them.
Boundary Bench
Deterministic crash tests for agent control planes: paired worlds, hard stops, budgets, scopes, approvals, and hash-linked receipts.
Sonata Labs
Clone a company into a fake Gmail, Slack and Calendar, run your agent through a simulated workday inside it, and score how much of the job it finished without a human.
Residoo
Find secrets your AI coding agent leaked to disk, watch for new ones, and optionally block a sensitive read before it happens. Free, MIT, zero deps, no network calls by default. O…
Conceptual Layer
A database's schema grounded in its data and described in sentences, ready for a text to SQL agent to read
Spinal
Give your AI agent a spinal cord: real-time game agents (a model sets goals, reflexes act every frame) and the Spinal Score, a Doom benchmark where 100% is elite human play. Try i…
Web Agent Bench
A fair benchmark of the ways to give an LLM a web browser (Playwright MCP, Vercel agent-browser, browser-use, Stagehand) on token usage, speed, and reliability.
Cadclamp
Benchmark for AI-generated parametric CAD, scored on whether the part can actually be printed
Awesome Workplace Agent Environments
A curated list of environments, benchmarks, datasets, and RL training resources for workplace AI agents: email, chat, calendar, docs, and enterprise apps.
Goku
Goku is an HTTP load testing application written in Rust
Adversarial Suite
🔒 Simulate adversarial behaviors to test and strengthen MCP defenses without real exploitation or risk, ensuring robust security evaluations.
Alethia Anvil
Alethia's reference app + Playwright/PW-MCP benchmark. Fork to evaluate Alethia against your own agent or reproduce the numbers.
Local Code Navigator
Symbol-aware and semantic code navigation for Claude Code and Codex, plus the ablation harness that measures whether it actually helps.
Wozniak Coffee Test
The Wozniak coffee test as an embodied MCP game for AI agents
Jev Filter
Typed semantic filtering for AI agents: turn retrieved records, code and logs into compact evidence with source IDs. CLI · Python · TypeScript · MCP.
Coder Bench
Benchmark tool for measuring MCP server effectiveness in LLM-assisted development
Meatloaf
🥩 AI Agent Sandbox Runtime — Build. Run. Test. Ship. Give AI agents a body to interact with the digital world.
ContextTax
Measure the context-window tax an MCP server charges your AI agent — the real token cost of its tool schemas + responses. Ground truth (Anthropic count_tokens) or keyless estimate…
Honeybadger
formal VM benchmark and inspectable reasoning runtime for testing whether language models can follow machine-like execution semantics on synthetic tasks.
2origin
本象协议 Benxiang · 一源万影：保存本象，按需投影。面向 AI 的持久对象表示层——AI 只提交语义事务，确定性编译器校验后才落地，每个字段都能回答「凭什么」。A2 消融实测（2026-08-13，README §5）：唯一幸存的分类性主张是结构性的——提示词结构上产不出证据链；A3 全套机制给 15.0%/27.5% 受测字段可溯源证据，所有纯…
Web Agent Comparison
Reproducible 8-dimension benchmark of 7 browser-automation MCP servers (Playwright, Lightpanda, browser-use, Chrome DevTools, Firecrawl, Cloakbrowser, Obscura) against frozen loop…
Inferbench
Benchmark local LLM inference speed (tokens/sec) on your own hardware — llama.cpp native + cloud APIs, 124-model catalog, optimal-quant picker, and an MCP serve mode.
Groundtruth Geo
Deterministic, government-cited, reproducible benchmark of parcel-precise US property facts — on the exact questions frontier models fail. Benchmark + grader + MCP server.
Fastevals
Give your AI agents an evaluation tool - provider-agnostic LLM eval runner with an MCP server for Claude, structured output validation, datasets, evaluators and honest cost metrics
Agent Bench
Measure whether an MCP server or skill actually improves your coding agent. A/B arms, deterministic gates, your own models. Negative results published.
Hpml Assetopsbench Smart Grid MCP
SmartGridBench: a smart-grid transformer-maintenance benchmark for tool-using LLM agents, built on IBM AssetOpsBench with MCP/direct-call tool surfaces.
N8n Validation Benchmark
Reproducible validation-honesty benchmark for n8n workflow validators exposed over MCP — probe fixtures, result schemas, methodology. No results published yet.
Neoxider MCP Hub
One MCP tool instead of every schema you own — a lazy capability broker that cuts resident tool context by a measured 94.4%. Search, inspect, enable and call MCP servers and skill…