- Conduid
- Marketplace
- #evaluation
MCP servers tagged evaluation
171 live MCP servers tagged "evaluation", ranked by trust score. Tags come from package metadata and repository topics, so this list covers servers across every category that work with evaluation.
Hegelion
Dialectical reasoning architecture for LLMs (Thesis → Antithesis → Synthesis)
Interviewer
Catch MCP server issues before your agents do.
MCP
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
Server
Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.
Kiln
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
Kiln
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
Ragbits
Building blocks for rapid development of GenAI applications
Opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Tracely AI
Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
Deepfabric
Generate High-Quality Synthetics, Train, Measure, and Evaluate in a Single Pipeline
Tracely
Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
As A Judge
MCP as a Judge is a behavioral MCP that strengthens AI coding assistants by requiring explicit LLM evaluations
Primitive Bench
The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivere…
Arksim
Find your agents errors be fore your real users do
Agentevals
agentevals is a framework-agnostic evaluations solution based on OpenTelemetry traces
DishwasherBench
A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.
Skill Optimizer
Benchmark, evaluate, and optimize skills to ensure reliable performance across all LLMs
Awesome Agent Observability
Curated list of tools, standards, and platforms for LLM and AI-agent observability: OpenTelemetry GenAI conventions, tracing, evals, guardrails, gateways, and MCP tooling.
Assay
Framework-agnostic evaluation harness for Go — test your MCP servers and AI agents with scored, CI-ready checks.
Kb Arena
Benchmark 19 retrieval architectures (vector, contextual, QnA, knowledge graph, LightRAG, hybrid, RAPTOR, PageIndex, BM25, metadata-filtered, temporal, rerank, HyDE, multi-query,…
Assay
An open, reproducible trust layer for AI artifacts — skills, MCP servers, agents, and plugins. Assay inspects code, runs sandboxed behavioral audits, and generates verifiable repo…
Code Analyzer
A Model Context Protocol server for Python code analysis with Claude. Again, works with warning now. I'm missing something here.
Dojo.md
University for AI agents. 92 courses, 4400+ scenarios, any model via OpenRouter. Auto-training loops generate per-model SKILL.md documents. Works with Claude Code, OpenClaw, Curso…
Harness Eval
Did this change to the coding-agent harness make the code better or worse? Paired, repeated, isolated evaluation with a verdict you can trust.
Scorecard
Scorecard — independent third-party profile of a public API surface, by API Evangelist. Scorecard is a simulation and evaluation platform for building, testing, and deploying fron…
LLM Wiki Agent
Convert raw content into an interlinked Obsidian wiki using this Kotlin MCP server and Claude Code. Build a knowledge base without RAG indexing.
Metacoder
Wrapper for multiple AI coders (goose, claude code, codex)
Tuneforge
An MCP server for dataset generation, fine-tuning, RL, and evaluation of LLMs — directly from your coding agent.
Decibench
The open testing standard for voice AI agents. Deterministic + semantic + RAG augmented evaluation. Local first. Zero telemetry.
Trustmodel
Score any AI for trust — Eval, Monitor, Govern. 10 trust dimensions, one free API key (5 credits / $500).
Host
MCP client manager: stdio + SSE transports, exp-backoff reconnect, server registry, built on @modelcontextprotocol/sdk
Mcplab MCP Server
MCP server that exposes MCPLab evaluation tools — query runs, results, and traces via the Model Context Protocol
Mcpbr CLI
Model Context Protocol Benchmark Runner - CLI tool for evaluating MCP servers
Dyno
Put your MCP server on the dyno — holistic, LLM-driven analysis of efficiency, cost, context-bloat, correctness, and reliability, with rigorous before/after error bars.
Agent Interview
AI Agent / LLM application engineer Chinese interview question bank: 2,038 questions with reference answers
Agent Registry
A vendor-neutral, agent-framework-agnostic registry for AI agents, skills, and MCP servers. Metadata store with evaluation signals, promotion lifecycle, and supply chain visibilit…
Memorystress
The first longitudinal benchmark for AI memory systems. Test LLM agents, RAG pipelines, MCP servers, and memory backends across 1,000 sessions.
Multi Agent Orchestrator
Multi-agent orchestration built as production infrastructure: typed planning, a sandboxed tool registry, race-safe human-in-the-loop, deterministic replay, and a provably-terminat…
MissionControl
Control plane for human-directed, agent-executed software delivery—with governed execution, verification, evidence, and human-approved pull requests.
Copeca
Cost per correct answer — a neutral, reproducible, verifiable benchmark for CLI-based coding agents
Companmem
the quest for ultimate ai memory
91ai
中文 AI 资料库：AI Coding、Agent、MCP、RAG、评测与学习路线图 | Chinese AI guide and curated resources.
Rag Best Practices
Educational RAG playbook + stdlib Python toolkit: chunking, hybrid search, eval, claim-check anti-hallucination, canaries, citations
Stoneburner
Local-first LLM eval: cost, quality, and security suites. Install stoneburner-atomics; CLI is atomics.
Caliber Suite
Open-source MLflow plugin for AI agents and agentic workflows: prompts, tools, skills, MCP servers, RAG knowledge bases, evaluation, deployment, observability, and Aria copilot.
RAG Codebase
Code-aware RAG over a repository: AST-boundary chunking (tree-sitter), hybrid dense+BM25 retrieval with RRF & reranking, a code graph (imports/calls + PageRank), grounded citation…
Awesome AI Application Engineer
Practical AI application engineering roadmap: LLM basics, Prompt Engineering, RAG, Agents, MCP, evaluation, production, tutorials, checklists, and templates.
Finance Agent
LangGraph and MCP multi-agent prototype for structured A-share research with bounded evaluation and reflection.
PromptRack
Version prompts. Run them across models. Find what works.
LLM Migrate
Plan, research, and evaluate LLM application migrations between models, providers, and platforms (Claude, GPT, Bedrock) before changing code