- Conduid
- Marketplace
- #llm-as-a-judge
MCP servers tagged llm-as-a-judge
16 live MCP servers tagged "llm-as-a-judge", ranked by trust score. Tags come from package metadata and repository topics, so this list covers servers across every category that work with llm-as-a-judge.
DataEval/dingo
based evaluation capabilities and rules&prompts listing.
As A Judge
MCP as a Judge is a behavioral MCP that strengthens AI coding assistants by requiring explicit LLM evaluations
Root Signals
MCP for Scorable Evaluation Platform
Awesome Jev Use Cases
Awesome list of TypeSafe AI Jev use cases: 74 demos ranked by likes, 150+ GitHub repos, limits, cost and API examples. CC0
LLM Council
The LLM Council works together to answer your hardest questions
Jev Judge MCP
Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy de…
Root Signals
improvement capabilities with [Root Signals](https://www.rootsignals.ai/)
Llmswap
Ask, compare, and synthesize across 10 LLM providers. Python SDK, CLI, local Arena, MCP, and privacy-aware Best Answer.
Nasde Toolkit
CLI for benchmarks & evals of AI coding agents — on tasks you already understand, using your Claude / Codex / Gemini individual subscriptions or API keys.
Titration
Let your coding agent fix a prompt until it actually works, and prove it did. A self-hosted MCP server: cross-vendor judges, frozen baselines and memory.
Tribunal
Governed verification and synthesis for Massive Intelligence (IM) outputs: configurable Thinker, Worker, and Verifier panels, model-as-judge, and Ed25519-signed, SHA-256-chained t…
Mdb API Layer
A pytest-based API testing framework for the TMDB REST API, showcasing modern SDET: end-to-end integration tests, Pact contract validation, Dockerized parallel runs, Kubernetes-ba…
AI Agent Testing
A research project to measure AI agent robustness. Contains automated testing pipelines and a benchmarking methodology developed to audit Agentic AI architectures for complex reas…
Tournament
Benchmark LLMs on your own scenarios with a multi-judge panel. Full MCP server (tools, resources, prompts, progress) plus local GUI, CLI, and desktop app. One OpenRouter key.
Lens Kit
Trainable 10-lens validation gate for AI-generated text. Outside-in: a separate model checks the output, fixed checks, inspectable findings, fail-closed. Apache-2.0 — own it and t…
Groundlens Dev
Guardrails for LLM answers. GroundLens checks an LLM answer against its sources without a model, decides PASS, REVIEW or FAIL under your policy, and keeps a signed record.