- Conduid
- Marketplace
- #evals
MCP servers tagged evals
128 live MCP servers tagged "evals", ranked by trust score. Tags come from package metadata and repository topics, so this list covers servers across every category that work with evals.
Gutfeel
Usability testing for MCP servers. A decision model that doesn't deliberate gets a bare user mission and walks your tools for real: every path, wrong turn and dead end, mapped and…
Clew
Lineage-based impact analysis and remediation for workflow runs. What a bad input or an incident report reached, what to re-run, delete or disclose, sealed as evidence an auditor…
Eval
🧐 Privacy-preserving MCP server evaluation: deterministic CI probe battery (18 probe kinds, readiness score, SARIF) plus production friction capture with findings that verify and…
Checkpoint
The release gate for AI agents — run your real agent N times against stateful twins and ship only when the confidence interval says so.
Triage
Reads our meetings, Slack, GitHub, email and docs. Finds work that needs doing, and updates the task board. Pauses for a human when needed, and checks the live board first. Offlin…
Geocheck
AI-search brand-visibility (GEO) checker - CLI + MCP server with a reproducible eval harness
Mini Antemortem CLI
Deterministic, offline linter that catches silent traps in your prompt-eval setup — train/test leakage, judge bias, homogeneous variants — before they fake a passing score. 11 tra…
Agent Librarian
A framework-neutral CLI that catalogs, de-duplicates, and reviews local collections of agent skills, prompts, tool specs, and agentic AI artifacts.
Agentic Architecture Audit
Evidence-first architecture audit for AI-assisted codebases. A discovery pass records what a repo claims to be. An eleven-phase audit tests those claims, with citations.
AI Skill
Agent skill for AI Systems — build model-powered behavior: agents, retrieval, evals, and guardrails
PromptRack
Version prompts. Run them across models. Find what works.
Awesome Verified Agents
Tools that produce evidence about what an AI agent actually did — gates, attestation, output verification, benchmarks. Inclusion bar: name the artifact a human can inspect later.
Gauntlet
A CI linter for what your MCP server publishes: tool poisoning and prompt injection across every surface an agent sees, definition drift between runs, and schema rot.
AI LLM Automation Systems
AI/LLM automation reference: RAG, tool calling, human review, evals, observability, safety gates, and local deterministic samples.
Mini Omega Lock
Measure your LLM judge's noise floor — an A/B improvement smaller than the judge's own noise isn't real. Pre-flight probes for prompt-eval setups; works standalone and with omegap…
Building Agent Study
一个系统学习 Agent 工程的中文项目：从理论到实践，由浅入深。
TracePad
LLM observability and evals in a single binary: traces, prompts, datasets, scores and review queues for LLM and agent apps. Self-hosted, SQLite inside, OpenTelemetry-native, MCP a…
Agent Chaos Monkey
Chaos engineering for AI agents. Injects connector failures, expired auth, prompt injection, tool schema drift and truncated payloads, then judges from tool-boundary evidence whet…
Tracepilot Gemini CLI
TracePilot: Gemini CLI fork with Phoenix/OpenInference tracing, MCP self-introspection, safety gates, redaction, evals, and a verified broken-repo repair loop
Memoturn
Open-source AI engineering platform — LLM observability, evals, metrics, prompt management, playground, and datasets. OpenTelemetry-native, self-hostable. Because "it worked when…
Openai Playground Evals Prompt Design
Hands-on experiments with OpenAI Playground covering prompt design, model comparison, tool/function calling, structured outputs, and eval-driven prompt improvement.
Desktop Assistant Agent
A modular Python AI agent runtime with routing, planning, skill selection, tool execution, approval-aware safety controls, recovery, replanning, MCP integration, deterministic eva…
Ground Truth Evals
An LLM eval harness graded by computed ground truth, not an LLM judge. Worked example: poker, served to models as MCP tools — gpt-4o-mini goes from 26/75 to 75/75 with them.
Runcoach AI
Local-first AI running coach for Garmin data, powered by an existing Claude subscription instead of an API key. Deterministic readiness engine, allowlisted agent, prompts tested l…
Patchwork Harness
The verifying harness for AI coding agents: test gates, independent cross-vendor review and grounded answers, with every action on the Patchwork audit trail.
Joyczl Agent
Joy — a local-first personal assistant. Rust core (loop / memory / tools / graph / evals), TypeScript gateways + dashboard, Python thin SDK.
Edgar MCP
MCP server for SEC EDGAR: read one section of a 10-K instead of all of it. Section extraction, XBRL tag merging, and an eval suite.
Vera
Local-first agent harness for sensitive professional document work.
Triage
Reads our meetings, Slack, GitHub, email and docs. Finds work that needs doing, and updates the task board. Pauses for a human when needed, and checks the live board first. Offlin…
Gauntlet
A CI linter for what your MCP server publishes: tool poisoning and prompt injection across every surface an agent sees, definition drift between runs, and schema rot.
AgentOps
Self-hosted LLM operations gateway in one Go binary: OpenAI-compatible router, durable agent tracker, faithfulness evals + drift, traces, MCP server, console
Trip Loom Monorepo
AI-powered travel assistant application + MCP Server for planning trips, booking flights & hotels, and building itineraries.
Omegaprompt
The overfit gate for your prompts: re-test the winning prompt on held-out examples it never tuned on, and block the ship if it doesn't generalize. Sits on top of promptfoo/DSPy. C…
Chainquery
Text-to-SQL agent over public Hedera mainnet data. Ask in English, get the SQL back as the citation — with an eval harness whose cases are proven to reject careless queries. Also…
Groundtruth Geo
Deterministic, government-cited, reproducible benchmark of parcel-precise US property facts — on the exact questions frontier models fail. Benchmark + grader + MCP server.
Harneloop
Build self-evolving AI agent harnesses with portable harness units, artifact-aware testing, trace-backed diagnosis, and evidence-gated promotion.
Briefing Agent
TypeScript briefing agent that uses MCP, model-driven tool calls, and lightweight evals to generate structured briefings from a controlled knowledge source.
AI Portfolio Manager
An autonomous LLM fund that scores its own stock predictions — Brier score, calibration curves, eval + ablation harness — and serves its full decision history over MCP. LangGraph…
Agent Skill Forge
Spec-driven AI agent skill generator and verification pipeline for building, testing, repairing, and packaging production-ready agent skills.
Multivon MCP
MCP server exposing multivon-eval + pdfhell as agent-callable tools. Drop into Claude Desktop, Cursor, Cline.
Coach Kernel
Deterministic safety kernel for AI-agent training plans, with MCP data boundaries, Claude Agent SDK integration, replay evals, and fuzzing.
Pymcpevals
Server-focused evaluation framework for MCP (Model Context Protocol) servers.
MultiChat
Broadcast one prompt to 2-6 AI models side-by-side - or convene them as a deliberative panel: an AI-only Habermas Machine with blind drafts, anonymous peer review, explicit conver…
Deep Dive Capstone
Codebase Q&A with (path:line) citations — the capstone of a from-scratch AI-engineering series. Built in 8 eval-gated stages (chat → RAG → agents → hardening → production), provid…
Urusilla
Open AI-agent interoperability research. Bring your own agent, complete verified public work, and earn contribution credit; if URSL launches, 1 credit = 1 URSL.
Leaseops
AI agent for a property manager's maintenance inbox.
Copilot Studio Simulator
Unofficial local Copilot Studio simulator: declarative agents (topics, tools, MCP, code execution, knowledge, child agents) + evals + auto-tuning, built on the pi harness SDK
Coding Agent Crew
Turns one AI session into hands-on engineers who create or improve Codex-shaped coding agents.
Pipsync MCP Evals
No-execution MCP mock and safety evaluations for trading-oriented tools.
Actuals
Honest metrics for AI tools and features. Agent skills for Claude Code, Codex, Cursor, Copilot, Gemini CLI and 30+ others: designs outcome metrics from your business context, audi…