- Conduid
- Marketplace
- #evaluation
MCP servers tagged evaluation
171 live MCP servers tagged "evaluation", ranked by trust score. Tags come from package metadata and repository topics, so this list covers servers across every category that work with evaluation.
AI Eng Prep
Hands-on AI-engineering repo: a Python RAG pipeline that grows into a model-decided agent — with retrieval & trajectory evals, an MCP server, and per-run cost/token/latency observ…
Claude Code Operating Framework Example
Sanitized Claude Code operating framework teaser showing sub-agents, skills, hooks, MCP, guardrails, and evaluation.
AI Financial Scenario Analyst
Decision-support demo that separates language interpretation from deterministic financial calculations, with typed tools, audit traces, and regression tests.
Self Improving Agent Harness
An AI coding agent that fixes bugs in a sandboxed repo -- and a harness that learns from its own failures, writes guardrails against them, and proves with regression evidence they…
Chunktuner
Benchmark chunking strategies for your RAG corpus. Get a recommended config. CLI, Python library, and MCP server.
Memstrata
Semantic memory MCP server (FastMCP + Postgres/pgvector) with tiered recall depth for token economy. Ships a retrieval-quality eval suite — precision@5 0.69, recall@10 0.82.
Northstar Workload Optimizer
Governed expense-operations platform with deterministic policy execution, durable workflows, provenance, evaluations, observability, and MCP interfaces.
Betterthantoon
GCF vs TOON benchmark data
Evalrank
Open-source EvalRank core: evidence-ranked evaluation primitives, schemas, and SDK boundary.
VouqisVerify
Vouqis Verify (VouqisVerify) generates AI Change Evidence Packs from merged GitHub pull requests, with AI change classifications, CI checks, and reviewer evidence exported to Exce…
Special War
A jeopardy-style CTF where the competitors are AIs. Play via MCP, no account needed.
Awesome MCP Testing
A curated list of tools for testing, evaluating, validating and debugging MCP (Model Context Protocol) servers: inspectors, conformance suites, eval harnesses, test libraries, sca…
AI Workflow Orchestrator
Classifies and prioritises incoming tickets, emails and logs, applies an auditable rules engine, triggers actions safely, and exposes the pipeline to agents over MCP.
Engramia
Reusable execution memory and evaluation infrastructure for AI agent frameworks — BUSL 1.1
Rag MCP Agent
RAG agent over the Russian Civil Code: 1736 articles chunked into 2543 passages in ChromaDB, exposed to a LangGraph ReAct agent through an MCP tool server. Ships a 20-query evalua…
Gradia Guard
Open-source, proof-bound evidence recorder and verifier for explicit AI-system execution boundaries.
Memory Rag
Personal-memory RAG for an AI coding agent: BM25 + bge vectors + file-level RRF, CLI/MCP dual outlets, 18-query golden eval (18/18, MRR .880)
Investigation Agent Client
AI investigation agent for risk analysis with LLM planning, tool calling, MCP capability reuse,policy grounding, structured artifacts, and evaluation.
Multi Agent Research Assistant
An autonomous multi-agent research assistant built with LangGraph, Gemini Flash, MCP search tools, self-reflection quality loops, and SQLite observability dashboard.
LLM Eval
MCP server for LLM evaluation gates.
Coding Agent Guardrail
Configurable pre-execution safety layer for coding agents, with a falsifiable anti-cheat evaluation. Blocks test-suppression cheats (.skip, @ts-ignore, || true) at the write point.
LLM Framework Benchmark
Can an AI coding agent build the same framework-neutral spec on different stacks? External-oracle-graded benchmark. First: Spring Boot vs Tiko DI.
Novaops Company Brain
Permission-aware AI workflows for onboarding, IT access, vendor intake, and renewals. Python, MCP, Bedrock, durable approvals, and evaluation.
LLM Engineering Portfolio
Thirteen measured experiments on one RAG pipeline: retrieval evaluation, hallucination defense, prompt injection, MCP, and stateful agents. Every number produced by code in this r…
Review Rubric
What code review enforces, mined from 2,899 comments across 9 repos, and how to feed it back to an LLM reviewer
AI Ticket Triage
Production-minded LangGraph support triage agent with MCP tools, bounded failure, and a 20-case evaluation suite.
Agentops Research Intelligence
Production-oriented multi-agent research intelligence platform combining LangGraph orchestration, Model Context Protocol (MCP), hybrid vector retrieval, claim verification, and ev…
Oculus Scoped Observation
User-controlled scoped browser observation and evaluation harness for AI agents
Toolassay
Evaluation harness for MCP tool contracts: does a model pick the right tool, finish the task, and what does it cost.
Cherry Evals
Search, cherry-pick, and export examples from public AI evaluation datasets. Built for AI engineers and AI agents.
Mg Ax
Evaluation framework for the agentic experience (AX) of MCP tools: static lint rules over tools/list manifests plus a cassette-backed trace pipeline that replays recorded agent se…
Productagent Evalbench
A local and optional-real-provider Agent evaluation bench for RAG, Memory, Skills, MCP-style tools, tracing, model benchmarking, and cost-controlled evaluation.
Toolsweep
Which tool-schema decision is costing you accuracy? Controlled factorial ablation for LLM tool APIs.
Macs Xval Go
MACS §5: dual-model cross-validation for subjective agent outputs — consensus/partial/disagree adjudication
Researchops
Autonomous research & experiment engineering agent: stateful graph orchestration, hand-written MCP servers, sandboxed code execution with failure recovery, human-in-the-loop appro…
Rag MCP Agent Demo
Reference RAG + MCP agent with a Text-to-SQL safety gate and an offline eval harness. Dependency-free, tested.
Fleet MCP Platform
Claude agent that detects and fixes faults in a simulated 100-vehicle fleet through a Go MCP server. Seeded fault simulator, ground-truth eval harness, two-layer write validation.
Artwork Preflight Agent
Automated print-readiness checking for custom print orders: measures artwork against the ordered product on CPU in ~1s, clears clean files, explains the fix, escalates the rest. 7…
Forsee MCP
MCP server for the Forsee AI Forum: browse and answer posts, and take a free six-tier capability ladder. Dependency-free stdio server.
Seamlens
Visual end-to-end map of API, gRPC and database contracts with system-design checks, breaking-change diffs and AI-coding-agent impact analytics. CLI, web UI and MCP server.
Analytics Evals
Agent skills for testing analytics and text-to-SQL agents: mine golden test cases from your query history, score with pass^k and confidence intervals, and find which context files…
AgentGym
A verifiable Kubernetes incident gym for post-training and serving tool-using agents. Real control plane, deterministic workload simulator, executable rewards.
Ungrounded
Measure how often your LLM agent reaches for the wrong tool when it can't ground an entity in a request.
DevAgent
AI coding agent that reproduces a GitHub issue with a failing test, fixes it in a no-network Docker sandbox, and opens a draft PR only after human approval. FastAPI + Next.js, MCP…
E2H
Evidence-to-Harness toolkit for reproducible AI-agent evaluations, observable evidence, deterministic replay, and verifiable releases.
Claim Studio
营销文案事实审校与视频合成 Agent：Qwen 经 MCP 检索资料逐句核查，缺证据即停止；5 种检索策略 Hit@5 10%→43.3%，4B/14B 对照留出集 75%→100%；148 测试、OTEL/Prometheus 可观测、人工确认绑定快照后渲染 FFmpeg 视频。 | Marketing fact-checking agent: Qwe…
Advance Fraud Analyst
Multi-model fraud detector using Hugging Face + LangChain + MCP
Eval
CI-embeddable eval harness for MCP agents — scores the sequence of tool calls your agent makes, not just text output
Audit
An evaluation harness for MCP servers: tool-selection accuracy, schema-drift (breaking-change) detection, description linting, and a production-readiness scorecard. Runs as an MCP…
Aegis
Open-source agent simulation and runtime control platform for Claude Code