OWASP WSTG RAG

A Retrieval-Augmented Generation (RAG) system that indexes the OWASP Web Security Testing Guide (WSTG) into a vector database, providing instant access to security testing methodologies via REST API and MCP (Model Context Protocol) for Claude Code integration.

Features

• Complete WSTG Coverage - All 12 WSTG testing categories indexed and searchable
• Semantic Search - Find relevant testing methodologies using natural language queries
• MCP Integration - Direct integration with Claude Code for AI-assisted penetration testing
• REST API - HTTP endpoints for programmatic access
• WSTG ID Lookup - Retrieve complete test cases by WSTG identifier (e.g., WSTG-INPV-05)

WSTG Categories

Quick Start

1. Install Dependencies

cd RAG_runner
pip install -r requirements.txt

2. Build the Database

python3 build_database.py

This will:

• Parse all OWASP WSTG HTML files
• Create semantic chunks for retrieval
• Build the ChromaDB vector database

3. Start the Server

python3 -m server.http_server

Server runs on http://localhost:5004

4. Test the API

# Health check
curl http://localhost:5004/health

# Search for SQL injection testing
curl -X POST http://localhost:5004/search \
  -H "Content-Type: application/json" \
  -d '{"query": "SQL injection testing methodology"}'

# Get specific WSTG test case
curl http://localhost:5004/wstg/WSTG-INPV-05

REST API Endpoints

Search Request Body

{
  "query": "SQL injection testing",
  "n_results": 5,
  "category": "input_validation",
  "wstg_id": "WSTG-INPV-05"
}

Claude Code Integration (MCP)

Add to ~/.claude.json:

{
  "mcpServers": {
    "owasp-wstg-rag": {
      "command": "python3",
      "args": ["/path/to/OWASP_WSTG_Rag/RAG_runner/server/mcp_client.py"],
      "env": {
        "WSTG_RAG_URL": "http://localhost:5004"
      }
    }
  }
}

MCP Tools

Example Usage in Claude Code

# Search for SQL injection testing methodology
search_wstg("SQL injection testing methodology")

# Get specific test case
get_wstg_test_case("WSTG-INPV-05")

# Search within a category
search_wstg("authentication bypass", category_filter="authentication")

# Get test objectives for IDOR
search_test_objectives("IDOR insecure direct object reference")

Project Structure

OWASP_WSTG_Rag/
├── README.md
├── CLAUDE.md                    # Claude Code project guide
├── raw_data/                    # OWASP WSTG HTML source files
│   ├── 01-Information_Gathering/
│   ├── 02-Configuration_and_Deployment_Management_Testing/
│   ├── 03-Identity_Management_Testing/
│   ├── 04-Authentication_Testing/
│   ├── 05-Authorization_Testing/
│   ├── 06-Session_Management_Testing/
│   ├── 07-Input_Validation_Testing/
│   ├── 08-Testing_for_Error_Handling/
│   ├── 09-Testing_for_Weak_Cryptography/
│   ├── 10-Business_Logic_Testing/
│   ├── 11-Client-side_Testing/
│   └── 12-API_Testing/
└── RAG_runner/
    ├── build_database.py        # Main build pipeline
    ├── requirements.txt
    ├── parsers/
    │   └── wstg_parser.py       # HTML parser for WSTG
    ├── chunking/
    │   └── chunker.py           # Semantic chunking
    ├── server/
    │   ├── vector_store.py      # ChromaDB wrapper
    │   ├── http_server.py       # REST API server
    │   └── mcp_client.py        # MCP tools for Claude Code
    └── data/
        ├── processed/           # Intermediate JSON files
        └── chroma_db/           # Vector database

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    OWASP WSTG HTML Files                        │
│                      (raw_data/*.html)                          │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                     wstg_parser.py                              │
│              Parse HTML → Structured JSON                       │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                       chunker.py                                │
│              Create Semantic Chunks for RAG                     │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                   ChromaDB Vector Store                         │
│                 (data/chroma_db/)                               │
└────────────────────────────┬────────────────────────────────────┘
                             │
              ┌──────────────┴──────────────┐
              ▼                             ▼
┌──────────────────────────┐   ┌──────────────────────────┐
│    http_server.py        │   │    mcp_client.py         │
│    REST API :5004        │   │    MCP for Claude Code   │
│                          │   │                          │
│  GET  /health            │   │  search_wstg()           │
│  GET  /info              │   │  get_wstg_test_case()    │
│  GET  /wstg/{id}         │   │  search_test_methodology │
│  POST /search            │   │  list_wstg_categories()  │
└──────────────────────────┘   └──────────────────────────┘

Use Cases

AI-Assisted Penetration Testing

Integrate with Claude Code to get instant access to OWASP testing methodologies during security assessments:

User: "How do I test for SQL injection?"

Claude: [Queries WSTG RAG]
→ Returns WSTG-INPV-05 methodology with:
  - Test objectives
  - Step-by-step testing procedures
  - Example payloads
  - Tools to use

Automated Security Testing

Use the REST API to integrate WSTG methodologies into automated security pipelines:

import requests

# Get testing methodology for current test
response = requests.post('http://localhost:5004/search', json={
    'query': 'session fixation testing',
    'n_results': 3
})
methodology = response.json()['results']

Security Training

Quick reference for security testing methodologies during training or CTF challenges.

Requirements

• Python 3.8+
• ChromaDB
• BeautifulSoup4
• httpx
• MCP SDK (for Claude Code integration)

License

This project uses content from the OWASP Web Security Testing Guide, which is licensed under Creative Commons Attribution-ShareAlike 4.0.

Related Projects

• OWASP WSTG - Source material
• Claude Code - AI coding assistant with MCP support
• ChromaDB - Vector database for embeddings