qa-mcp-server

Production-ready MCP (Model Context Protocol) governance server for Playwright BDD automation.

Acts as a central gatekeeper that prevents duplicate test creation, enforces BDD structure, reuses existing steps, blocks invalid file creation, and detects duplicate logic using AST analysis.

Table of Contents

1. Project Structure
2. Installation
3. Configuration
4. Running the Server
5. Connecting to Claude
6. Tools Reference
7. How the Registry Works
8. Governance Rules
9. Extending the Server

Project Structure

qa-mcp-server/
├── server.py                  ← MCP entry point, all tools registered here
├── tools/
│   ├── feature_duplicate.py   ← Gherkin duplicate detection (Jaccard ≥ 0.85)
│   ├── step_mapper.py         ← Step definition lookup in .ts files
│   ├── ast_detector.py        ← Structural spec similarity (tree-sitter)
│   ├── file_validator.py      ← File creation gatekeeper
│   ├── structure_validator.py ← BDD 3-file completeness check
│   └── intent_parser.py       ← User story → structured JSON intent
├── utils/
│   ├── gherkin_parser.py      ← Lightweight .feature file parser
│   ├── ast_parser.py          ← tree-sitter / regex fingerprint extractor
│   ├── similarity.py          ← Jaccard similarity functions
│   └── file_utils.py          ← Shared file I/O helpers
├── registry/
│   └── test_registry.json     ← JIRA → Feature → Test mapping store
├── config/
│   ├── rules.py               ← Naming and creation rules
│   └── settings.py            ← Paths and similarity thresholds
├── logs/
│   └── mcp.log                ← Append-only decision log
└── requirements.txt

Installation

# 1. Clone / copy the project
cd qa-mcp-server

# 2. Create and activate a virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

Python 3.11+ is required.

Configuration

All configuration lives in config/settings.py. The most important settings can be overridden via environment variables:

Example:

export QA_PROJECT_ROOT=/home/ci/playwright-bdd
export MCP_PORT=9000
python server.py

Similarity Thresholds

Edit config/settings.py to tune detection sensitivity:

FEATURE_DUPLICATE_THRESHOLD = 0.85   # Feature Jaccard — mark as duplicate
AST_BLOCK_THRESHOLD          = 0.90  # Spec AST Jaccard — block creation
AST_WARN_THRESHOLD           = 0.75  # Spec AST Jaccard — warn

Running the Server

# From the qa-mcp-server directory
python server.py

Output:

2025-01-01 12:00:00  INFO      qa_mcp.server — Starting qa-mcp-server on 0.0.0.0:8000 (transport=streamable-http)

The server listens on http://0.0.0.0:8000/mcp by default.

Connecting to Claude

Claude Desktop (claude_desktop_config.json)

Add the following entry to your Claude Desktop MCP configuration:

{
  "mcpServers": {
    "qa-mcp-server": {
      "type": "streamable-http",
      "url": "http://localhost:8000/mcp",
      "description": "QA governance — Playwright BDD gatekeeper"
    }
  }
}

Config file locations:

Claude Code / API

import anthropic

client = anthropic.Anthropic()

# The MCP server is registered as a remote tool server
response = client.beta.messages.create(
    model="claude-opus-4-5",
    max_tokens=4096,
    mcp_servers=[
        {
            "type": "url",
            "url": "http://localhost:8000/mcp",
            "name": "qa-mcp-server",
        }
    ],
    messages=[{"role": "user", "content": "Check if this feature already exists: ..."}],
)

Tools Reference

check_feature_duplicate

Detects semantically duplicate .feature files using Jaccard similarity on normalised BDD steps.

Inputs:

Output:

{
  "allowed": false,
  "status": "duplicate",
  "matches": [
    { "file": "features/login.feature", "similarity": 0.912 }
  ],
  "message": "Feature 'user_login.feature' is too similar to existing features..."
}

find_existing_step

Searches all .ts step definition files for a matching step definition.

Inputs:

Output:

{
  "found": true,
  "file": "steps/login_steps.ts",
  "matched_step": "user clicks the login button",
  "similarity": 1.0,
  "message": "Exact step match found. Do NOT create a new step."
}

check_ast_duplicate

Structural comparison of .spec.ts files using tree-sitter AST fingerprinting.

Inputs:

Output:

{
  "decision": "block",
  "allowed": false,
  "matches": [
    { "file": "login.spec.ts", "similarity": 0.94, "decision": "block" }
  ],
  "message": "BLOCKED: 'new.spec.ts' duplicates existing test logic..."
}

Decision thresholds:

validate_file_creation

Final gate before any file write. Validates directory, extension, naming convention, and uniqueness.

Inputs:

Output:

{
  "allowed": true,
  "reason": "All checks passed. 'checkout.feature' can be created in 'features'.",
  "checks": [
    "PASS: Extension is not globally blocked.",
    "PASS: Directory 'features' is allowed.",
    "PASS: Extension is allowed in 'features'.",
    "PASS: Naming convention satisfied.",
    "PASS: No duplicate filename found."
  ]
}

validate_test_structure

Checks that all three BDD artefacts exist for a given feature name.

Inputs:

Output:

{
  "complete": false,
  "missing": ["spec"],
  "present": ["feature", "steps"],
  "expected_files": {
    "feature": "playwright-project/features/user_login.feature",
    "steps":   "playwright-project/steps/user_login_steps.ts",
    "spec":    "playwright-project/tests/user_login.spec.ts"
  },
  "message": "Incomplete test structure for 'user_login'. Missing: ['spec']."
}

parse_user_story_intent

Converts a plain-English user story into a structured test intent and checks the registry for duplicates.

Inputs:

Output:

{
  "feature": "login",
  "scenario": "login_valid_credentials",
  "test_name": "TC_login_valid_credentials",
  "jira_id": "QA-42",
  "valid": true,
  "duplicate": false,
  "message": "Intent parsed successfully."
}

How the Registry Works

registry/test_registry.json stores a mapping of JIRA tickets to test artefacts:

{
  "version": "1.0",
  "mappings": {
    "QA-42": {
      "feature": "login",
      "test_name": "TC_login_valid_credentials"
    }
  }
}

• parse_user_story_intent with register=true writes to this file.
• validate_file_creation reads it to block duplicate filenames.
• The registry is append-only — existing entries are never overwritten.

Governance Rules

Extending the Server

Adding a new tool

1. Create tools/my_tool.py with the business logic function.
2. Import and register in server.py:

from tools.my_tool import my_function

@mcp.tool()
def my_tool_name(param: str) -> dict:
    """Docstring shown to the AI agent."""
    return my_function(param)

Adding new rules

Add rule functions to config/rules.py following the (bool, str) return convention, then call them from the relevant tool.

Adjusting thresholds

Edit config/settings.py or set environment variables — no code changes needed.

Logging

All tool decisions are appended to logs/mcp.log:

2025-01-01 12:00:01  INFO      qa_mcp.server — TOOL check_feature_duplicate called — file='checkout.feature'
2025-01-01 12:00:01  WARNING   qa_mcp.tools.feature_duplicate — Duplicate feature detected: 'checkout.feature' ~ 'features/cart_checkout.feature' (similarity=0.912)
2025-01-01 12:00:01  INFO      qa_mcp.server — check_feature_duplicate → allowed=False status=duplicate

Log levels: DEBUG (file only) · INFO · WARNING · ERROR (file + console).