- Conduid
- Marketplace
- #llm-observability
MCP servers tagged llm-observability
39 live MCP servers tagged "llm-observability", ranked by trust score. Tags come from package metadata and repository topics, so this list covers servers across every category that work with llm-observability.
Opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Tracely AI
Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
Tracely
Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
1flowbase
1flowbase: self-hosted AI gateway with protocol translation, dispatch, chat logs, built-in backend & React blocks to combine AI with business data. All managed by your Agent via M…
OrcaReplay
OrcaReplay — Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.
Tma1
Local-first observability your agent reads back. TMA1 records every LLM call, then routes what it sees into the agent's next turn via hooks and MCP.
Dunetrace
The runtime reliability layer for AI agents.
server
LangWatch — independent third-party profile of a public API surface, by API Evangelist. LangWatch is an open-source LLM observability, evaluation, and AI agent testing platform. B…
Bar Observatory
Deterministic, local-only audit reports for Claude Code AI agent sessions. Rust + SQLite. Zero network calls. MCP server for agents.
Securevector AI Threat Monitor
Security & Observability for AI Agents — audit every tool call, enforce allow/block policies, catch prompt injection and data leaks. Local-first; works with Claude Code, Codex, Co…
Isnad
Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines
AgentProvenance
Security-oriented three-axis observability for sandboxed AI agents: model intent, app context, and runtime telemetry into verifiable evidence graphs for risk, forensics, and audit.
LLM Workbench
Model Context Protocol (MCP) server factory and HTTP handler that exposes an LLM Workbench runtime over MCP.
Agentglass
AgentGlass — a glass box for your AI agents. Capture, inspect, replay, and analyze every agent run (LLM observability & tracing). No SDK, no code changes.
Otelcol Genai Sketches
Bounded GenAI metrics and keyed summaries from OpenTelemetry traces
Rlcd Gateway
Self-hosted gateway for LLMs and decision models. Claude Code, Codex, OpenCode and any OpenAI/Anthropic SDK app reach any provider (OpenRouter, Groq, Ollama…) with context pruning…
Fleetdiff
See what changed in your agent app's token usage, from summary files instead of raw traces.
Fluiq SDK
Instrument any LLM app in 2 lines. Tracing, evaluation, security, and caching for OpenAI, Anthropic, Gemini, LangChain, MCP, and more.
Glasshouse
Observability and a transparent, auditable agent graph for Massive Intelligence (IM) swarms: OpenTelemetry-compatible span tracing, incremental drift detection, failure attributio…
TracePad
LLM observability and evals in a single binary: traces, prompts, datasets, scores and review queues for LLM and agent apps. Self-hosted, SQLite inside, OpenTelemetry-native, MCP a…
Memoturn
Open-source AI engineering platform — LLM observability, evals, metrics, prompt management, playground, and datasets. OpenTelemetry-native, self-hostable. Because "it worked when…
LLM Hops
See every hop of a request through your local LLM system: a tracing server, a live flow map, a waterfall per request, and an MCP server for Claude Code and Cursor. One static Go b…
hugoles/langfuse-mcp
signed releases.
Apitoolkit
APIToolkit (Monoscope) — independent third-party profile of a public API surface, by API Evangelist. APIToolkit (now Monoscope) is an open-source-friendly API observability and mo…
ClaudeCode Token Universe
Local, single-file HTML dashboard for Claude Code usage and estimated USD cost — token totals across all classes (input/output/cache), per-model spend at list API prices, live cur…
FireTrace
Self-hosted LLM & agent tracing with infinite retention: a Next.js dashboard on Vercel backed by your own Firebase project, with scoped API keys, a REST API, a TypeScript SDK, and…
AI Insights
Understand and improve how you use AI coding tools. A local web app that turns raw usage data into a live dashboard — token counts, costs, session history, and configuration healt…
LangfuseMCP
Langfuse MCP server with built-in analytics. 34 tools — traces, observations, sessions, scores, prompts, datasets + accuracy metrics, failure detection, token percentiles, cost br…
CNCF LLM Observability Multi Agent Demo
Meridian Claims: a real multi-agent health-insurance claims-adjudication system (CrewAI + MCP + Langfuse) demonstrating production-grade LLM observability
AIObservability3.0
End-to-end Dynatrace observability for agentic AI workloads - services, agents, tools, prompts, models, topology, FinOps. Detects orchestration overhead vs substantive agents, sur…
Auditlens
EU AI Act compliance agent — audits production LLM traces with Gemini, scores them against the EU AI Act, and writes verdicts back to Arize Phoenix with human approval. Google Clo…
Multi Agent Credit Desk
Architecture-first Python scaffold for an auditable multi-agent corporate credit desk using A2A, MCP, deterministic credit policies, model routing, and OpenTelemetry.
Kubit AI Inc
Kubit AI, Inc. — independent third-party profile of a public API surface, by API Evangelist. Kubit is a warehouse-native digital analytics platform that runs product analytics dir…
Obs Unified
Self-hosted observability for agentic debugging: traces, logs, replay, AI cost, Agent Action Graphs, MCP, profiles, and usage analytics in one evidence graph.
Flightrules
Turns SigNoz traces into deterministic release contracts that catch skipped checks, duplicate side effects, unknown tool paths, and behavioural drift before an AI-agent canary rea…
ChronoLens
Closed-loop predictive SRE control plane built on SigNoz. Forecasts SLO breaches from real p99 traces, takes a reversible action, then proves from measured telemetry that the outa…
Claude Contextlens
See where your coding agent's tokens actually go. Local usage analytics and setup audit for Claude Code: cost, prompt-cache economics, context growth, 5-hour billing windows, and…
Agentops Watchtower
Local-first AgentOps flight recorder and capability firewall for MCP-based coding agents with OpenTelemetry traces.
Cfp Opensearch
Debug AI agents like distributed systems: OpenTelemetry spans through Data Prepper into OpenSearch, ten PPL investigations, and the cross-index join that proves whether an agent's…