Test Coach
This project implements an AI-powered testing and code-fixing server using FastMCP and OpenAI. It provides tools to generate tests, run them under coverage, and suggest code fixes.
Ask AI about Test Coach
Powered by Claude · Grounded in docs
I know everything about Test Coach. Ask me about installation, configuration, usage, or troubleshooting.
0/500
Reviews
Documentation
test-coach
A new project currently under development.
Overview
This project implements an AI-powered testing and code-fixing server using FastMCP and OpenAI. It provides tools to generate tests, run them under coverage, and suggest code fixes.
Setup
- Create and activate a Python virtual environment:
python -m virtualenv .venv source .venv/bin/activate # On Windows: `.\\.venv\\Scripts\\activate` - Install dependencies:
pip install -r requirements.txt - Copy the example environment file and configure your variables:
cp .env.example .env # On Windows: `copy .env.example .env` # Then edit .env and set your values - (Optional) Remove any embedded credentials from Git config:
git remote set-url origin https://github.com/$GITHUB_USER/test-coach.git
Environment Variables
Store all sensitive keys and tokens in the .env file. See .env.example for required names.
How it works
-
Server (
fastmcp_server) exposes three tools via standard I/O:generate_tests(module: str) -> str
Generates pytest tests for a given Python module using the OpenAI ChatCompletion API.run_tests() -> dict
Runs tests under coverage, produces a JSON coverage report, and returns it.fix_code(failing_test: str, error: str) -> str
Prompts OpenAI to suggest code or test fixes so that pytest passes.
-
Pipe (
fastmcp_server/pipe.py) handles communication with the OpenAI API:- Loads
OPENAI_API_KEYand optionalOPENAI_MODELfrom.env. - Defines
codex_query(prompt, temperature)wrappingopenai.ChatCompletion.create.
- Loads
-
Tools (
fastmcp_server/tools.py) implement the logic of each tool, decorated with@mcp.tool(). -
Server entrypoint (
fastmcp_server/server.py):- Imports and registers all tools, then calls
mcp.run()to start listening on stdio.
- Imports and registers all tools, then calls
-
Sample library (
sample_lib/calculator.py) provides simple functionsadd(a, b)anddivide(a, b)(with a deliberateZeroDivisionErroronb=0) to demonstrate the workflow.
Usage Example
- Start the FastMCP server:
python -m fastmcp_server.server - Generate tests for the sample module in another terminal:
mcp-cli call generate_tests \ --server stdio://$(pgrep -f fastmcp_server.server) \ --json '{"module":"sample_lib.calculator"}' - Run tests and retrieve coverage:
mcp-cli call run_tests --server stdio://$(pgrep -f fastmcp_server.server)
