qa-mcp-server

Production-ready MCP (Model Context Protocol) governance server for Playwright BDD automation.

Acts as a central gatekeeper that prevents duplicate test creation, enforces BDD structure, reuses existing steps, blocks invalid file creation, and detects duplicate logic using AST analysis.

---

Table of Contents

1. Project Structure
2. Installation
3. Configuration
4. Running the Server
5. Connecting to Claude
6. Tools Reference
7. How the Registry Works
8. Governance Rules
9. Extending the Server

---

Project Structure

qa-mcp-server/
├── server.py                  ← MCP entry point, all tools registered here
├── tools/
│   ├── feature_duplicate.py   ← Gherkin duplicate detection (Jaccard ≥ 0.85)
│   ├── step_mapper.py         ← Step definition lookup in .ts files
│   ├── ast_detector.py        ← Structural spec similarity (tree-sitter)
│   ├── file_validator.py      ← File creation gatekeeper
│   ├── structure_validator.py ← BDD 3-file completeness check
│   └── intent_parser.py       ← User story → structured JSON intent
├── utils/
│   ├── gherkin_parser.py      ← Lightweight .feature file parser
│   ├── ast_parser.py          ← tree-sitter / regex fingerprint extractor
│   ├── similarity.py          ← Jaccard similarity functions
│   └── file_utils.py          ← Shared file I/O helpers
├── registry/
│   └── test_registry.json     ← JIRA → Feature → Test mapping store
├── config/
│   ├── rules.py               ← Naming and creation rules
│   └── settings.py            ← Paths and similarity thresholds
├── logs/
│   └── mcp.log                ← Append-only decision log
└── requirements.txt

---

Installation

# 1. Clone / copy the project
cd qa-mcp-server

# 2. Create and activate a virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

Python 3.11+ is required.

---

Configuration

All configuration lives in config/settings.py. The most important settings can be overridden via environment variables:

Environment Variable	Default	Description
QA_PROJECT_ROOT	./playwright-project	Root path of the Playwright BDD project being governed
QA_REGISTRY_PATH	./registry/test_registry.json	Path to the test registry file
QA_LOG_PATH	./logs/mcp.log	Path to the log file
MCP_HOST	0.0.0.0	Host for the MCP HTTP server
MCP_PORT	8000	Port for the MCP HTTP server

Example:

export QA_PROJECT_ROOT=/home/ci/playwright-bdd
export MCP_PORT=9000
python server.py

Similarity Thresholds

Edit config/settings.py to tune detection sensitivity:

FEATURE_DUPLICATE_THRESHOLD = 0.85   # Feature Jaccard — mark as duplicate
AST_BLOCK_THRESHOLD          = 0.90  # Spec AST Jaccard — block creation
AST_WARN_THRESHOLD           = 0.75  # Spec AST Jaccard — warn

---

Running the Server

# From the qa-mcp-server directory
python server.py

Output:

2025-01-01 12:00:00  INFO      qa_mcp.server — Starting qa-mcp-server on 0.0.0.0:8000 (transport=streamable-http)

The server listens on http://0.0.0.0:8000/mcp by default.

---

Connecting to Claude

Claude Desktop (claude_desktop_config.json)

Add the following entry to your Claude Desktop MCP configuration:

{
  "mcpServers": {
    "qa-mcp-server": {
      "type": "streamable-http",
      "url": "http://localhost:8000/mcp",
      "description": "QA governance — Playwright BDD gatekeeper"
    }
  }
}

Config file locations:

OS	Path
macOS	~/Library/Application Support/Claude/claude_desktop_config.json
Windows	%APPDATA%\Claude\claude_desktop_config.json
Linux	~/.config/Claude/claude_desktop_config.json

Claude Code / API

import anthropic

client = anthropic.Anthropic()

# The MCP server is registered as a remote tool server
response = client.beta.messages.create(
    model="claude-opus-4-5",
    max_tokens=4096,
    mcp_servers=[
        {
            "type": "url",
            "url": "http://localhost:8000/mcp",
            "name": "qa-mcp-server",
        }
    ],
    messages=[{"role": "user", "content": "Check if this feature already exists: ..."}],
)

---

Tools Reference

check_feature_duplicate

Detects semantically duplicate .feature files using Jaccard similarity on normalised BDD steps.

Inputs:

Field	Type	Description
feature_text	string	Raw Gherkin content of the proposed feature
feature_filename	string	Intended filename (e.g. user_login.feature)

Output:

{
  "allowed": false,
  "status": "duplicate",
  "matches": [
    { "file": "features/login.feature", "similarity": 0.912 }
  ],
  "message": "Feature 'user_login.feature' is too similar to existing features..."
}

---

find_existing_step

Searches all .ts step definition files for a matching step definition.

Inputs:

Field	Type	Description
step_text	string	Plain-text Gherkin step

Output:

{
  "found": true,
  "file": "steps/login_steps.ts",
  "matched_step": "user clicks the login button",
  "similarity": 1.0,
  "message": "Exact step match found. Do NOT create a new step."
}

---

check_ast_duplicate

Structural comparison of .spec.ts files using tree-sitter AST fingerprinting.

Inputs:

Field	Type	Description
spec_source	string	TypeScript source of the proposed spec
spec_filename	string	Intended filename

Output:

{
  "decision": "block",
  "allowed": false,
  "matches": [
    { "file": "login.spec.ts", "similarity": 0.94, "decision": "block" }
  ],
  "message": "BLOCKED: 'new.spec.ts' duplicates existing test logic..."
}

Decision thresholds:

Similarity	Decision
≥ 0.90	block — creation refused
0.75–0.90	warn — review recommended
< 0.75	allow — safe to create

---

validate_file_creation

Final gate before any file write. Validates directory, extension, naming convention, and uniqueness.

Inputs:

Field	Type	Description
target_directory	string	One of: features, steps, tests
filename	string	Proposed filename with extension

Output:

{
  "allowed": true,
  "reason": "All checks passed. 'checkout.feature' can be created in 'features'.",
  "checks": [
    "PASS: Extension is not globally blocked.",
    "PASS: Directory 'features' is allowed.",
    "PASS: Extension is allowed in 'features'.",
    "PASS: Naming convention satisfied.",
    "PASS: No duplicate filename found."
  ]
}

---

validate_test_structure

Checks that all three BDD artefacts exist for a given feature name.

Inputs:

Field	Type	Description
feature_name	string	Snake-case feature name (e.g. user_login)

Output:

{
  "complete": false,
  "missing": ["spec"],
  "present": ["feature", "steps"],
  "expected_files": {
    "feature": "playwright-project/features/user_login.feature",
    "steps":   "playwright-project/steps/user_login_steps.ts",
    "spec":    "playwright-project/tests/user_login.spec.ts"
  },
  "message": "Incomplete test structure for 'user_login'. Missing: ['spec']."
}

---

parse_user_story_intent

Converts a plain-English user story into a structured test intent and checks the registry for duplicates.

Inputs:

Field	Type	Description
user_story	string	Plain-English story
jira_id	string	Optional JIRA ID (e.g. QA-123)
register	bool	Save to registry if true

Output:

{
  "feature": "login",
  "scenario": "login_valid_credentials",
  "test_name": "TC_login_valid_credentials",
  "jira_id": "QA-42",
  "valid": true,
  "duplicate": false,
  "message": "Intent parsed successfully."
}

---

How the Registry Works

registry/test_registry.json stores a mapping of JIRA tickets to test artefacts:

{
  "version": "1.0",
  "mappings": {
    "QA-42": {
      "feature": "login",
      "test_name": "TC_login_valid_credentials"
    }
  }
}

• parse_user_story_intent with register=true writes to this file.
• validate_file_creation reads it to block duplicate filenames.
• The registry is append-only — existing entries are never overwritten.

---

Governance Rules

Rule	Enforced by
No duplicate .feature files	check_feature_duplicate
No duplicate step definitions	find_existing_step
No duplicate spec logic	check_ast_duplicate
Only features/, steps/, tests/ directories	validate_file_creation
No .md or .txt files	validate_file_creation
snake_case.feature naming	validate_file_creation
snake_case_steps.ts naming	validate_file_creation
snake_case.spec.ts naming	validate_file_creation
TC_feature_scenario test names	parse_user_story_intent
Complete 3-file BDD structure	validate_test_structure
No duplicate JIRA/business logic	parse_user_story_intent

---

Extending the Server

Adding a new tool

1. Create tools/my_tool.py with the business logic function.
2. Import and register in server.py:

from tools.my_tool import my_function

@mcp.tool()
def my_tool_name(param: str) -> dict:
    """Docstring shown to the AI agent."""
    return my_function(param)

Adding new rules

Add rule functions to config/rules.py following the (bool, str) return convention, then call them from the relevant tool.

Adjusting thresholds

Edit config/settings.py or set environment variables — no code changes needed.

---

Logging

All tool decisions are appended to logs/mcp.log:

2025-01-01 12:00:01  INFO      qa_mcp.server — TOOL check_feature_duplicate called — file='checkout.feature'
2025-01-01 12:00:01  WARNING   qa_mcp.tools.feature_duplicate — Duplicate feature detected: 'checkout.feature' ~ 'features/cart_checkout.feature' (similarity=0.912)
2025-01-01 12:00:01  INFO      qa_mcp.server — check_feature_duplicate → allowed=False status=duplicate

Log levels: DEBUG (file only) · INFO · WARNING · ERROR (file + console).