mcp-devils-advocate

MCP server that stress-tests reasoning: devil's advocate, premortem analysis, assumption audits, steelmanning and an all-in-one gauntlet, run as enforced, structured protocols.

Why

LLMs agree too easily. Ask one whether your plan is good and you get a polite yes with three bullet points. This server makes it harder to shortcut adversarial thinking. It never writes content itself. It is a state machine that makes the client LLM finish each phase properly: minimum counts, categories, severity ratings and length floors. It checks every submission and returns errors the model can act on. It will not move to the next phase until the current one is complete. At the end it writes a report with a deterministic assessment.

Length floors alone are easy to game, so every submission also goes through a quality gate (see below). It rejects padding ("x" * 30), near-duplicate items, "counterarguments" that only restate the claim, and rebuttals, mitigations, tests or responses copied from the item they answer. Nothing is saved until the whole batch passes. The gate uses deterministic text heuristics, not a semantic judge. It stops lazy and degenerate submissions. It cannot prove that an argument is good.

Five modes:

Tools

Every validation error reaches the model word for word under both SDK generations, for example item 1: 'text' is a near-duplicate of item 0 (similarity 1.00) — each counterargument must make a distinct point. Under mcp 2.x a plain exception in a tool shows up as just Error executing tool submit, so the server converts every validation error into the SDK's ToolError. The end-to-end tests check this.

Resources & prompts

Quality gate

Text is compared as sets of content words. The text is lowercased, accents are stripped and punctuation is removed. A small English + Spanish stopword and filler list is dropped (negations included, so "we should not do X" adds nothing to "we should do X"). A tiny stemmer then makes rewrite/rewriting and service/services count as the same word. The thresholds are constants in quality.py, and every phase's instructions list them under quality_checks.

All four shortcuts found in the 0.1.0 audit now fail with a message that names the item, and nothing is saved: 30 × x, one counterargument pasted under three categories, counter responses that repeat the claim, and justifications copied from the counterargument. Each of them used to end in "claim survives scrutiny". tests/test_quality.py keeps them as regression tests, next to the README example below, which still passes unchanged.

Assessment rules (deterministic)

In a gauntlet, each lens is assessed with its own rule above, applied to the same review.

How it works

flowchart TD
    S[start_review claim + mode] --> M{mode}
    M -->|devils_advocate| C["counterarguments<br/>≥3, ≥2 categories, severity 1–5"]
    C --> D{any severity ≥ 3?}
    D -->|yes| R["rebuttals<br/>one per severe counterargument"]
    D -->|no| V
    R --> V[all phases complete]
    M -->|premortem| P1[setup: horizon] --> P2["failure_causes<br/>≥4, likelihood × impact"]
    P2 --> P3{any score ≥ 9?}
    P3 -->|yes| P4["mitigations<br/>action + residual risk"] --> V
    P3 -->|no| V
    M -->|assumptions| A1["assumptions<br/>≥4, load_bearing + evidence"]
    A1 --> A2{unverified load-bearing?}
    A2 -->|yes| A3["tests<br/>cheapest verification"] --> V
    A2 -->|no| V
    M -->|steelman| T1["strongest_case<br/>≥3 points for the opposing side"]
    T1 --> T2["responses<br/>concede or counter each"] --> V
    M -->|gauntlet| GA["devil's advocate phases"] --> GB["assumptions phases"] --> GC["premortem phases"] --> GD["steelman phases"] --> V
    Q[[quality gate<br/>junk · duplicates · restating · parroting]] -. checks every submit .-> C
    V --> G["get_verdict<br/>report + risk score + assessment"]
    G --> E["export_report / review://id<br/>Markdown decision memo"]

Every submit is checked in two layers, and the whole batch passes or fails together. First the fields are validated (types, enums, ranges, length floors, target indices). Then the quality gate runs. The server only moves on when the phase's requirements are met. It skips dependent phases that have no targets, for example when no counterargument reached severity 3. State is saved as one JSON file per review in ~/.mcp-devils-advocate/ (override it with the DEVILS_ADVOCATE_DIR environment variable). Reviews written by 0.1.0 still load.

Quickstart

Not on PyPI yet. The publish.yml workflow uploads to PyPI (and the MCP Registry entry in server.json points there) when a v* tag is pushed. No release has been tagged yet, so pip install mcp-devils-advocate / uvx mcp-devils-advocate do not work. Until then, install from GitHub:

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "devils-advocate": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/AleBrito124356/mcp-devils-advocate", "mcp-devils-advocate"]
    }
  }
}

Claude Code:

claude mcp add devils-advocate -- uvx --from git+https://github.com/AleBrito124356/mcp-devils-advocate mcp-devils-advocate

pip (permanent install, then use mcp-devils-advocate as the command):

pip install "git+https://github.com/AleBrito124356/mcp-devils-advocate"

With no arguments, mcp-devils-advocate runs the stdio MCP server, the same as in 0.1.0, so existing client configs keep working.

Example session

User: We're considering rewriting our backend in Rust. Play devil's advocate before we commit.

The assistant calls start_review(claim="We should rewrite our backend in Rust", mode="devils_advocate") and receives (abridged):

{
  "review_id": "rev-k4d7",
  "status": "active",
  "phases": ["counterarguments", "rebuttals"],
  "instructions": {
    "phase": "counterarguments",
    "step": "1/2",
    "claim": "We should rewrite our backend in Rust",
    "goal": "Attack the claim as a devil's advocate. Generate the strongest, most specific counterarguments you can — no strawmen.",
    "item_format": {
      "text": "str, >= 30 characters — the counterargument, concrete and falsifiable",
      "category": "one of: evidence, incentives, base_rates, alternatives, second_order, scope",
      "severity": "int 1-5 — how damaging to the claim if true (5 = fatal)"
    },
    "rules": [
      "Submit at least 3 counterarguments (send them all in one call).",
      "Use at least 2 distinct categories.",
      "Every counterargument with severity >= 3 must be rebutted in the next phase — assign severity honestly."
    ],
    "quality_checks": [
      "'text' needs at least 4 distinct content words (...) — junk padding is rejected.",
      "Each counterargument must be distinct: a Jaccard word-overlap of >= 0.8 with another counterargument in this phase is rejected as a near-duplicate.",
      "A counterargument that only restates the claim (>= 80% of its content words from the claim, or fewer than 2 new ones) is rejected."
    ]
  }
}

The assistant does the actual thinking and calls submit:

submit("rev-k4d7", [
  {"text": "Full rewrites of working systems fail or massively overrun far more often than they succeed; incremental strangler migrations have much better base rates.", "category": "base_rates", "severity": 4},
  {"text": "The team has zero production Rust experience; hiring and ramp-up costs will land exactly while feature delivery is frozen.", "category": "evidence", "severity": 4},
  {"text": "Profiling shows the bottleneck is the database layer, not CPU — a rewrite optimizes the part that isn't slow.", "category": "alternatives", "severity": 5},
  {"text": "Engineers pushing the rewrite are the ones who want Rust on their CV — incentives are not aligned with the business case.", "category": "incentives", "severity": 2}
])

The server validates the batch, marks the phase complete, and returns the rebuttals phase with three targets (the severity ≥3 items). The assistant rebuts each one honestly:

submit("rev-k4d7", [
  {"index": 0, "verdict": "partially_holds", "justification": "True in general, but we can scope the rewrite to the two stateless services first, which is effectively a strangler migration."},
  {"index": 1, "verdict": "holds", "justification": "No honest rebuttal: nobody on the team has shipped Rust, and the hiring market for it is thin in our region."},
  {"index": 2, "verdict": "holds", "justification": "The profiling data is real — the p99 latency lives in Postgres queries. A Rust rewrite does not touch that."}
])

All phases are complete, so get_verdict("rev-k4d7") compiles the report:

{
  "review_id": "rev-k4d7",
  "claim": "We should rewrite our backend in Rust",
  "mode": "devils_advocate",
  "risk_score": {"value": 2, "scale": "0-3 counterarguments that held after rebuttal"},
  "assessment": "claim refuted",
  "assessment_reason": "Of 3 rebutted counterargument(s): 2 hold, 1 partially hold, 0 refuted; a severity-5 counterargument holds. Rules: refuted if >=2 hold or any severity-5 holds; ..."
}

export_report("rev-k4d7") then returns the decision memo shown in the demo below, with a Next actions checklist the team can paste into an issue.

Assistant: The claim did not survive scrutiny. Two counterarguments held, including a severity-5 one: our bottleneck is the database, not CPU, so a Rust rewrite attacks the wrong problem — and we have no Rust experience in-house. Recommendation: fix the query layer first; if CPU ever becomes the bottleneck, migrate one stateless service as a pilot.

CLI

The same command works on the review files without an LLM client and without importing the MCP SDK:

--data-dir DIR (before or after the command) overrides DEVILS_ADVOCATE_DIR. python -m mcp_devils_advocate ... is equivalent. For demo, --data-dir keeps the review so that list/show/report can open it afterwards. Otherwise it runs in a temporary directory and deletes it at the end. Try demo --mode gauntlet --memo-only for the full four-lens memo.

This is the real output of mcp-devils-advocate demo. A test checks it against the code, so it cannot go stale:

Offline demo: no LLM, no network. A scripted client plays the model and runs a
'devils_advocate' review through the real ReviewStore.

start_review(claim='We should rewrite our backend in Rust', mode='devils_advocate', context=...)
  -> rev-ess4, phases: counterarguments -> rebuttals

== Phase 1/2: counterarguments ==
Goal: Attack the claim as a devil's advocate. Generate the strongest, most specific counterarguments you can — no strawmen.
Item format:
  text: str, >= 30 characters — the counterargument, concrete and falsifiable
  category: one of: evidence, incentives, base_rates, alternatives, second_order, scope
  severity: int 1-5 — how damaging to the claim if true (5 = fatal)
Rules:
  - Submit at least 3 counterarguments (send them all in one call).
  - Use at least 2 distinct categories.
  - Every counterargument with severity >= 3 must be rebutted in the next phase — assign severity honestly.
Quality checks:
  - 'text' needs at least 4 distinct content words (stopwords and filler like 'good', 'bad', 'really' do not count), and once it has 6+ content words no single word may make up more than 50% of them — junk padding is rejected.
  - Each counterargument must be distinct: a Jaccard word-overlap of >= 0.8 with another counterargument in this phase is rejected as a near-duplicate.
  - A counterargument that only restates the claim (>= 80% of its content words from the claim, or fewer than 2 new ones) is rejected.

A lazy client pastes one counterargument under three categories:
  REJECTED — Invalid submission — nothing was saved. Fix these problems and resubmit:
  - item 1: 'text' is a near-duplicate of item 0 (similarity 1.00) — each counterargument must make a distinct point
  - item 2: 'text' is a near-duplicate of item 0 (similarity 1.00) — each counterargument must make a distinct point
The real client does the thinking instead.

submit(rev-ess4, 4 item(s)):
  [base_rates, severity 4] Full rewrites of working systems fail or massively overrun far more often than they succeed; in…
  [evidence, severity 4] The team has zero production Rust experience; hiring and ramp-up costs will land exactly while …
  [alternatives, severity 5] Profiling shows the bottleneck is the database layer, not CPU — a rewrite optimizes the part th…
  [incentives, severity 2] Engineers pushing the rewrite are the ones who want Rust on their CV — incentives are not align…
  -> phase_complete; next phase: rebuttals

== Phase 2/2: rebuttals ==
Goal: Rebut each high-severity counterargument honestly. If you cannot refute one, admit that it holds.
Targets: 3 (#0, #1, #2)
Item format:
  index: int — index of the counterargument being rebutted (see 'targets')
  verdict: one of: holds, partially_holds, refuted — does the counterargument survive your rebuttal?
  justification: str, >= 20 characters — why that verdict
Rules:
  - Provide exactly one rebuttal per target (3 pending).
  - 'holds' means the counterargument stands and damages the claim. Do not mark 'refuted' without an actual refutation.
Quality checks:
  - 'justification' needs at least 3 distinct content words (stopwords and filler like 'good', 'bad', 'really' do not count), and once it has 6+ content words no single word may make up more than 50% of them — junk padding is rejected.
  - Each rebuttal must be distinct: a Jaccard word-overlap of >= 0.8 with another rebuttal in this phase is rejected as a near-duplicate.
  - A rebuttal that only restates the claim (>= 80% of its content words from the claim, or fewer than 2 new ones) is rejected.
  - A rebuttal that parrots the counterargument it answers (>= 80% of its content words copied, or fewer than 2 new ones) is rejected.

submit(rev-ess4, 3 item(s)):
  #0 partially_holds: True in general, but we can scope the rewrite to the two stateless services first, which is eff…
  #1 holds: No honest rebuttal: nobody on the team has shipped Rust, and the hiring market for it is thin i…
  #2 holds: The profiling data is real — the p99 latency lives in Postgres queries. A Rust rewrite does not…
  -> complete: All phases complete. Call get_verdict('rev-ess4') to compile the final report.

get_verdict(rev-ess4) -> claim refuted (risk score 2: 0-3 counterarguments that held after rebuttal)

export_report(rev-ess4, format='markdown'):

# Decision memo: We should rewrite our backend in Rust

- **Review:** `rev-ess4` · mode `devils_advocate` (Devil's advocate)
- **Assessment:** **claim refuted**
- **Risk score:** 2 — 0-3 counterarguments that held after rebuttal
- **Started:** 2026-01-15T09:00:00+00:00 · **Completed:** 2026-01-15T09:02:00+00:00

**Context:** 9-year-old Python monolith, 14 engineers, enterprise customers complaining about p99 latency.

## Verdict

**claim refuted** — Of 3 rebutted counterargument(s): 2 hold, 1 partially hold, 0 refuted; a severity-5 counterargument holds. Rules: refuted if >=2 hold or any severity-5 holds; revision if exactly 1 holds or >=2 partially hold; survives otherwise.

## Counterarguments

- **#0 · base_rates · severity 4** — Full rewrites of working systems fail or massively overrun far more often than they succeed; incremental strangler migrations have much better base rates.
  - Rebuttal (*partially holds*): True in general, but we can scope the rewrite to the two stateless services first, which is effectively a strangler migration.
- **#1 · evidence · severity 4** — The team has zero production Rust experience; hiring and ramp-up costs will land exactly while feature delivery is frozen.
  - Rebuttal (*holds*): No honest rebuttal: nobody on the team has shipped Rust, and the hiring market for it is thin in our region.
- **#2 · alternatives · severity 5** — Profiling shows the bottleneck is the database layer, not CPU — a rewrite optimizes the part that isn't slow.
  - Rebuttal (*holds*): The profiling data is real — the p99 latency lives in Postgres queries. A Rust rewrite does not touch that.
- **#3 · incentives · severity 2** — Engineers pushing the rewrite are the ones who want Rust on their CV — incentives are not aligned with the business case.
  - _severity below 3 — no rebuttal required_

## Next actions

- [ ] Resolve counterargument #0 (partially holds, severity 4): Full rewrites of working systems fail or massively overrun far more often than they succeed; incremental strangler migrations have much better base rates.
- [ ] Resolve counterargument #1 (holds, severity 4): The team has zero production Rust experience; hiring and ramp-up costs will land exactly while feature delivery is frozen.
- [ ] Resolve counterargument #2 (holds, severity 5): Profiling shows the bottleneck is the database layer, not CPU — a rewrite optimizes the part that isn't slow.

<sub>Generated by mcp-devils-advocate from review `rev-ess4`.</sub>

Compatibility

The dependency is mcp>=1.5.0,<3. SDKs before 1.3 write stdout in the Windows code page, so any non-ASCII text (an em dash, a Spanish accent) corrupts the JSON-RPC stream. The 1.3/1.4 stdio client hangs on Windows, which is why 1.5.0 is the lowest version the end-to-end suite can verify. The upper bound keeps a future 3.x from breaking installs silently, as 2.0 did to 0.1.0.

Development

git clone https://github.com/AleBrito124356/mcp-devils-advocate
cd mcp-devils-advocate
python -m venv .venv && .venv/bin/pip install -e ".[dev]"   # Windows: .venv\Scripts\pip
.venv/bin/python -m pytest

Run the server from the source tree with python -m mcp_devils_advocate (or python -m mcp_devils_advocate.server).

The suite has 140+ tests, all offline:

• test_core.py: phase machine, validation, verdict rules and persistence. It also covers the lifecycle fixes: complete reviews can't be abandoned, malformed files are skipped, same-second ordering is stable, context reaches every phase, and 0.1.0 files still load.
• test_quality.py: every quality check on both sides of its threshold, the four audit exploits, and the README example as a regression test.
• test_gauntlet.py: the nine-phase flow, auto-skipped phases, and every branch of the combined rule.
• test_memo.py: the memo has every section in all five modes, and Next actions matches the open work.
• test_cli.py: demo, list, show, report and error exits, run as real subprocesses.
• test_server_e2e.py: starts the real server (module and console script) and drives it over stdio with the official SDK client. It covers tools, resources, the prompt, a full flow, a gauntlet, and error messages reaching the model word for word.

mcp_devils_advocate/core.py, quality.py and memo.py use only the standard library, and every test except test_server_e2e.py runs without mcp (that file is skipped when the SDK is missing). To check the other SDK generation, install it into a second venv: pip install -e . "mcp<2" pytest, then run python -m pytest.

Related MCP servers

Part of a family of small, dependency-light MCP servers:

• mcp-decision-lab — weighted decision matrices with sensitivity analysis
• mcp-secret-sentinel — scan code for exposed secrets, always redacted
• mcp-git-historian — churn hotspots, blame summaries, bus factor
• mcp-memory-vault — persistent memory with SQLite FTS5 search

License

MIT