Rogue — AI Agent Evaluator & Red Team Platform

Stress-test your AI agents before attackers do.

Discord Community · Quick Start · Documentation

Two Ways to Harden Your Agent

🎯 Automatic Evaluation

Test your agent against business policies and expected behaviors.

• Define scenarios & expected outcomes
• Verify compliance with business rules
• Watch live conversations as Rogue probes your agent
• Get detailed pass/fail reports with reasoning

Best for: Regression testing, behavior validation, policy compliance

🔴 Red Teaming

Simulate adversarial attacks to find security vulnerabilities.

• 75+ vulnerabilities across 12 security categories
• 20 attack techniques (encoding, social engineering, injection)
• CVSS-based risk scoring
• 8 compliance frameworks (OWASP, MITRE, NIST, GDPR, EU AI Act)

Best for: Security audits, penetration testing, compliance reporting

Architecture

Rogue operates on a client-server architecture with multiple interfaces:

https://github.com/user-attachments/assets/b5c04772-6916-4aab-825b-6a7476d77787

Supported Protocols

See examples in examples/ for reference implementations.

Python Entrypoint

For agents implemented as Python functions without A2A or MCP:

1. Create a Python file with a call_agent function:

def call_agent(messages: list[dict]) -> str:
    """
    Process conversation and return response.

    Args:
        messages: List of {"role": "user"|"assistant", "content": "..."}

    Returns:
        Agent's response as a string
    """
    # Your agent logic here
    latest = messages[-1]["content"]
    return f"Response to: {latest}"

1. Run Rogue with Python protocol:

uvx rogue-ai cli \
  --protocol python \
  --python-entrypoint-file ./my_agent.py \
  --judge-llm openai/gpt-4o-mini

Or via TUI: select "Python" as the protocol and enter the file path.

See examples/python_entrypoint_stub.py for a complete example.

🔥 Quick Start

Prerequisites

• uvx — Install uv
• Python 3.10+
• LLM API key (OpenAI, Anthropic, or Google)

Installation

# TUI (recommended)
uvx rogue-ai

# CLI / CI/CD
uvx rogue-ai cli

Try It With the Example Agent

# All-in-one: starts both Rogue and a sample T-shirt store agent
uvx rogue-ai --example=tshirt_store

Configure in the UI:

• Agent URL: http://localhost:10001
• Mode: Choose Automatic Evaluation or Red Teaming

Running Modes

Server Options

uvx rogue-ai server --host 0.0.0.0 --port 8000 --debug

CLI Options

uvx rogue-ai cli \
  --evaluated-agent-url http://localhost:10001 \
  --judge-llm openai/gpt-4o-mini \
  --business-context-file ./.rogue/business_context.md

Red Teaming

Scan Types

Compliance Frameworks

• OWASP LLM Top 10 — Prompt injection, sensitive data exposure, excessive agency
• MITRE ATLAS — Adversarial threat landscape for AI systems
• NIST AI RMF — AI risk management framework
• ISO/IEC 42001 — AI management system standard
• EU AI Act — European AI regulation compliance
• GDPR — Data protection requirements
• OWASP API Top 10 — API security best practices

Attack Categories

Risk Scoring (CVSS-based)

Each vulnerability receives a 0-10 risk score based on:

• Impact — Severity if exploited
• Exploitability — Success rate likelihood
• Human Factor — Manual exploitation potential
• Complexity — Attack difficulty

Reproducible Scans

# Use random seeds for reproducible results
uvx rogue-ai cli --random-seed 42

Perfect for regression testing and validating security fixes.

Configuration

Environment Variables

OPENAI_API_KEY="sk-..."
ANTHROPIC_API_KEY="sk-..."
GOOGLE_API_KEY="..."

Config File (.rogue/user_config.json)

{
  "evaluated_agent_url": "http://localhost:10001",
  "judge_llm": "openai/gpt-4o-mini"
}

Key Features

Documentation

• Quick Reference — One-page cheat sheet
• Red Team Workflow — Technical deep-dive
• Implementation Status — Feature breakdown
• Attack Mapping — Vulnerability coverage

Contributing

1. Fork the repository
2. Create a branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push (git push origin feature/amazing-feature)
5. Open a Pull Request

License

Licensed under a proprietary license — see LICENSE.

Free for personal and internal use. Commercial hosting requires licensing. Contact: hello@rogue.security