1. Conduid
  2. Science
  3. conKurrence
MCP server · Science

conKurrence

· github.com/AlligatorC0der/conkurrence

AI evaluation toolkit that measures inter-rater agreement (Fleiss' κ, Kendall's W) across multiple LLM providers. Evaluate prompt reliability, detect contested outputs, and track consensus trends over time.

Unclaimed science
InstallView source
34Low

Scored 6 months ago · breakdown

About conKurrence

conKurrence is an MCP server in the Science category: aI evaluation toolkit that measures inter-rater agreement (Fleiss' κ, Kendall's W) across multiple LLM providers. Evaluate prompt reliability, detect contested outputs, and track consensus trends over time.

Install

From source
git clone https://github.com/AlligatorC0der/conkurrence

This server has no ConduID identity, so agent calls to it are not receipted. Pin the version you install and review the source before granting it credentials.

Tools (7)

conkurrence_compareSide-by-side comparison of two runs

Parameters not published.

conkurrence_estimateEstimate cost and token usage

Parameters not published.

conkurrence_reportGenerate a detailed markdown report

Parameters not published.

conkurrence_runExecute an evaluation across multiple AI raters

Parameters not published.

conkurrence_suggestAI-powered schema suggestion from your data

Parameters not published.

conkurrence_trendTrack agreement over multiple runs

Parameters not published.

conkurrence_validate_schemaValidate a schema before running

Parameters not published.

Ask AI

✦

Ask AI about conKurrence

Powered by Claude · Grounded in docs

I know everything about conKurrence. Ask me about installation, configuration, usage, or troubleshooting.

Security checks

  • ✓README presentThe repository ships documentation.

5 checks have not been run yet.

README

ConKurrence

One command. Find out if your AI agrees with itself.

ConKurrence is a statistically validated consensus measurement toolkit for AI evaluation pipelines. It uses multiple AI models as independent raters, measures inter-rater reliability with Fleiss' kappa and bootstrap confidence intervals, and routes contested items to human experts.

Install

npm install -g conkurrence

MCP Server

Use ConKurrence as an MCP server in Claude Desktop or any MCP-compatible client:

npx conkurrence mcp

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "conkurrence": {
      "command": "npx",
      "args": ["-y", "conkurrence", "mcp"]
    }
  }
}

Claude Code Plugin

/plugin marketplace add AlligatorC0der/conkurrence

Features

  • Multi-model evaluation — Run your schema against Bedrock, OpenAI, and Gemini models simultaneously
  • Statistical rigor — Fleiss' kappa with bootstrap confidence intervals, Kendall's W for validity
  • Self-consistency mode — No API keys needed; uses the host model via MCP Sampling
  • Schema suggestion — AI-powered schema design from your data
  • Trend tracking — Compare runs over time, detect agreement degradation
  • Cost estimation — Know the cost before running

MCP Tools

Tool Description
conkurrence_run Execute an evaluation across multiple AI raters
conkurrence_report Generate a detailed markdown report
conkurrence_compare Side-by-side comparison of two runs
conkurrence_trend Track agreement over multiple runs
conkurrence_suggest AI-powered schema suggestion from your data
conkurrence_validate_schema Validate a schema before running
conkurrence_estimate Estimate cost and token usage

Links

License

BUSL-1.1 — Business Source License 1.1

README copy captured when this server was indexed; it may lag the repository. The original is authoritative.

Questions

About conKurrence

How do I install conKurrence?

No package has been published for this server, so install it from source: clone the repository and follow its README.

Is conKurrence safe to use with an AI agent?

Its trust score is 34 out of 100 (low). It passes 1 of the 1 static security checks run so far. It has no ConduID identity yet, so agent calls to it are not receipted.

Is conKurrence still maintained?

Conduid hasn't recorded a commit date for this repository yet. Check the repository directly for recent activity.