1. Conduid
  2. Science
  3. conKurrence
MCP server · Science

conKurrence

· github.com/AlligatorC0der/conkurrence

AI evaluation toolkit that measures inter-rater agreement (Fleiss' κ, Kendall's W) across multiple LLM providers. Evaluate prompt reliability, detect contested outputs, and track consensus trends over time.

Unclaimed science
InstallView source
34Low

Scored 5 months ago · breakdown

About conKurrence

conKurrence is an MCP server in the Science category: aI evaluation toolkit that measures inter-rater agreement (Fleiss' κ, Kendall's W) across multiple LLM providers. Evaluate prompt reliability, detect contested outputs, and track consensus trends over time. It has been installed 0 times through Conduid.

Install

Clone
git clone https://github.com/AlligatorC0der/conkurrence

This server has no ConduID identity, so agent calls to it are not receipted. Pin the version you install and review the source before granting it credentials.

Ask AI

✦

Ask AI about conKurrence

Powered by Claude · Grounded in docs

I know everything about conKurrence. Ask me about installation, configuration, usage, or troubleshooting.

Security checks

  • ·README presentNot checked yet.
  • ·License declaredNot checked yet.
  • ·Tests presentNot checked yet.
  • ·Dependencies pinnedNot checked yet.
  • ·No dynamic code executionNot checked yet.
  • ·Scoped permissionsNot checked yet.

README

ConKurrence

One command. Find out if your AI agrees with itself.

ConKurrence is a statistically validated consensus measurement toolkit for AI evaluation pipelines. It uses multiple AI models as independent raters, measures inter-rater reliability with Fleiss' kappa and bootstrap confidence intervals, and routes contested items to human experts.

Install

npm install -g conkurrence

MCP Server

Use ConKurrence as an MCP server in Claude Desktop or any MCP-compatible client:

npx conkurrence mcp

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "conkurrence": {
      "command": "npx",
      "args": ["-y", "conkurrence", "mcp"]
    }
  }
}

Claude Code Plugin

/plugin marketplace add AlligatorC0der/conkurrence

Features

  • Multi-model evaluation — Run your schema against Bedrock, OpenAI, and Gemini models simultaneously
  • Statistical rigor — Fleiss' kappa with bootstrap confidence intervals, Kendall's W for validity
  • Self-consistency mode — No API keys needed; uses the host model via MCP Sampling
  • Schema suggestion — AI-powered schema design from your data
  • Trend tracking — Compare runs over time, detect agreement degradation
  • Cost estimation — Know the cost before running

MCP Tools

Tool Description
conkurrence_run Execute an evaluation across multiple AI raters
conkurrence_report Generate a detailed markdown report
conkurrence_compare Side-by-side comparison of two runs
conkurrence_trend Track agreement over multiple runs
conkurrence_suggest AI-powered schema suggestion from your data
conkurrence_validate_schema Validate a schema before running
conkurrence_estimate Estimate cost and token usage

Links

License

BUSL-1.1 — Business Source License 1.1

README mirrored from the source repository 5 months ago. The original is authoritative.

Questions

About conKurrence

How do I install conKurrence?

Run git clone https://github.com/AlligatorC0der/conkurrence, then add the server to your MCP client's configuration. Conduid has recorded 0 installs, so the command is known to work with current clients.

Is conKurrence safe to use with an AI agent?

Its trust score is 34 out of 100 (low). Conduid hasn't run static security checks on this repository yet, so review the source yourself before granting it credentials. It has no ConduID identity yet, so agent calls to it are not receipted.

Is conKurrence still maintained?

Conduid hasn't recorded a commit date for this repository yet. Check the repository directly for recent activity.