Cognitive Coverage

Like test coverage, but for understanding.

Cognitive Coverage is a skill for AI coding agents that generates a measurement and learning system for any project — codebases, research papers, documentation wikis, or general knowledge domains. It produces coordinated artifacts that help you track, teach, verify, and quickly open genuine system understanding.

Works with any AI coding agent that supports custom instructions, including:

• GitHub Copilot (as a Copilot skill)
• Claude Code (as a custom command or CLAUDE.md instructions)
• OpenAI Codex (as a system prompt)
• Any LLM-powered agent with file access and markdown instruction support

The Problem

Technical debt is well understood. But there's a parallel form of debt that's harder to measure: cognitive debt — the gap between what a system does and what the people responsible for it actually understand.

Generative AI has accelerated this dramatically:

When a developer writes code from scratch, even messy code, the friction means they build at least a partial mental model. When an AI generates that same code, the developer may accept it without building the same understanding. At scale, this creates an accumulation of not knowing. The code works, but the mental models are missing.

Cognitive Coverage closes that gap.

What It Produces

By default, all generated outputs are placed in a cognitive-coverage/ folder in the target project. The launcher is the entry point for opening the artifact set.

Reliable Artifact Writing

The skill includes shell-safe writing guidance so agents do not stall while creating large HTML files. In particular, agents are told not to use Bash heredocs such as python - <<'PY' in PowerShell, to avoid giant one-line shell strings, and to fall back to small deterministic chunks or short temporary writer scripts when native file-write tools are unavailable.

Large Codebases and Knowledge Bases

For large projects, Cognitive Coverage switches from a single exhaustive pass to Large Corpus Mode:

1. Inventory the full corpus first
2. Cluster sources into areas and focused learning modules
3. Prioritize critical flows, entry points, security/data boundaries, and highly referenced material
4. Generate cognitive-coverage/learning-guide.html as the overview map
5. Generate optional learning-guides/<module-id>.html pages for deeper modules
6. Keep uncovered areas visible as explicit dashboard gaps

This keeps the first run honest and useful: high-priority understanding is taught immediately, while lower-priority areas remain tracked instead of being hidden inside an oversized guide.

Three Coverage Axes

Each axis has three status levels (e.g., uncovered → read → understood). Status terminology adapts automatically to the domain.

Quick Start

Option 1: GitHub Copilot

# Unix / macOS / WSL
git clone https://github.com/ryannadel/cognitive-coverage.git
cd cognitive-coverage
bash install.sh

# Windows PowerShell
git clone https://github.com/ryannadel/cognitive-coverage.git
cd cognitive-coverage
.\install.ps1

This copies the skill definition to ~/.copilot/skills/cognitive-coverage/SKILL.md.

Then open any project with Copilot and say:

"Generate cognitive coverage for this project"

Option 2: Claude Code

Claude Code has native skill support. Install as a project or personal skill:

# Project skill (this project only)
mkdir -p .claude/skills/cognitive-coverage
cp skill/SKILL.md .claude/skills/cognitive-coverage/SKILL.md

# Personal skill (all your projects)
mkdir -p ~/.claude/skills/cognitive-coverage
cp skill/SKILL.md ~/.claude/skills/cognitive-coverage/SKILL.md

Then invoke with /cognitive-coverage or just ask Claude to "generate cognitive coverage" — it will auto-detect the skill from the description.

See the Claude Code skills docs for more on skill placement and configuration.

Option 3: OpenAI Codex

Codex has native skill support via the .agents/skills/ directory:

# Project skill (this repo only)
mkdir -p .agents/skills/cognitive-coverage
cp skill/SKILL.md .agents/skills/cognitive-coverage/SKILL.md

# Personal skill (all your projects)
mkdir -p ~/.agents/skills/cognitive-coverage
cp skill/SKILL.md ~/.agents/skills/cognitive-coverage/SKILL.md

Then invoke with -coverage or let Codex auto-detect it from the description. Works in Codex CLI, IDE extension, and Codex app.

See the Codex skills docs for more on skill placement and configuration.

Option 4: Any Other Agent

The skill is just a markdown file (skill/SKILL.md) that tells an AI agent how to analyze a project and generate the artifact set. If your agent can:

1. Read files in a project directory
2. Follow markdown instructions
3. Write output files

...then it can run Cognitive Coverage. Copy skill/SKILL.md into whatever instruction mechanism your agent supports.

Manual Install

If you prefer not to clone, just copy skill/SKILL.md to the appropriate location for your agent:

MCP Integration (optional)

Cognitive Coverage also ships an optional MCP stdio server for agents that can call tools during a session. The skill still works without MCP; the server only exposes an existing cognitive-coverage.json manifest.

uv --directory /path/to/cognitive-coverage/mcp run python server.py \
  --manifest /path/to/your/project/cognitive-coverage.json

The server provides hierarchy-aware tools:

Example host config:

{
  "mcpServers": {
    "cognitive-coverage": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/cognitive-coverage/mcp",
        "run",
        "python",
        "server.py",
        "--manifest",
        "/path/to/your/project/cognitive-coverage.json"
      ]
    }
  }
}

Once installed, ask your agent:

"Use coverage_summary, then list_uncovered for concepts."

See mcp/README.md for Claude Code, Codex, and Cursor setup.

How It Works

  ┌──────────────────────┐
  │ Artifact Launcher    │
  │ cognitive-coverage   │
  │ /cognitive-coverage  │
  │      -open.html      │
  └──────────┬───────────┘
             │ links
             ▼
  ┌──────────────────────┐
  │   Teaching Guide     │──── quiz answers ────┐
  │  learning-guide.html │                      │
  └──────────┬───────────┘                      │
             │ "Learn" links                    ▼
  ┌──────────┴───────────┐              localStorage
  │  Coverage Dashboard  │◄──── reads ──────────┘
  │ cognitive-coverage   │
  │       .html          │
  └──────────┬───────────┘
             │ reads
  ┌──────────┴───────────┐
  │  Coverage Manifest   │
  │ cognitive-coverage   │
  │       .json          │
  └──────────────────────┘

1. Artifact Launcher opens automatically after generation and links to every output
2. Dashboard loads the manifest and renders coverage status
3. Gap report shows uncovered items with "Launch Teaching" buttons
4. Clicking a button opens the teaching guide at the relevant section
5. Answering quiz questions correctly writes results to localStorage
6. Returning to the dashboard syncs quiz results and upgrades coverage
7. You can also manually set status via buttons on each card
8. Export the updated manifest as JSON anytime

See docs/HOW-IT-WORKS.md for the full deep dive.

Domains

The skill auto-detects the project domain and adapts terminology:

• codebase — Source code projects (TypeScript, Python, Rust, Go, Java, etc.)
• research — Paper collections, literature reviews, datasets
• documentation — Wikis, knowledge bases, runbooks, guides
• knowledge — General-purpose (anything else)

Examples

See the examples/ directory for sample manifests:

• examples/codebase/ — REST API project
• examples/research/ — Transformer architecture paper review
• examples/documentation/ — Platform engineering wiki

Schema

The manifest format is defined by a JSON Schema:

schemas/cognitive-coverage.schema.json

Use it to validate manifests programmatically or in CI.

Project Structure

cognitive-coverage/
├── README.md                          # This file
├── LICENSE                            # MIT
├── CONTRIBUTING.md                    # How to contribute
├── install.sh                         # Copilot installer (Unix/macOS/WSL)
├── install.ps1                        # Copilot installer (Windows)
├── skill/
│   └── SKILL.md                       # The skill instructions (works with any agent)
├── mcp/
│   ├── server.py                      # Optional MCP stdio server
│   ├── test_server.py                 # MCP smoke tests
│   └── pyproject.toml                 # MCP server dependencies
├── schemas/
│   └── cognitive-coverage.schema.json # JSON Schema for manifests
├── examples/
│   ├── codebase/                      # TypeScript REST API example
│   ├── research/                      # ML paper review example
│   ├── documentation/                 # Platform wiki example
│   ├── mcp-config-claude-code.json    # Claude Code MCP config snippet
│   └── mcp-config-codex.json          # Codex MCP config snippet
├── docs/
│   └── HOW-IT-WORKS.md               # Detailed system documentation
└── .github/
    ├── ISSUE_TEMPLATE/
    │   ├── bug_report.md
    │   └── feature_request.md
    └── pull_request_template.md

Contributing

Contributions welcome! See CONTRIBUTING.md for guidelines.

Ideas for contributions:

• New domain vocabularies (legal, medical, academic)
• Improved quiz question patterns
• Dashboard visualization improvements
• Additional example manifests
• Install scripts for other agents
• Integrations with other tools

License

MIT

Built to fight cognitive debt — one concept at a time.