🛡️ AgentAudit

Security scanner for AI agent packages — CLI + MCP server

Scan MCP servers, AI skills, and packages for vulnerabilities, prompt injection, and supply chain attacks. Powered by regex static analysis and deep LLM audits.

📑 Table of Contents

• What is AgentAudit?
• Quick Start
• Commands Reference
• Quick Scan vs Deep Audit
• MCP Server
• What It Detects
• How the 3-Pass Audit Works
• CI/CD Integration
• Dashboard & Community
• Configuration
• Requirements
• FAQ
• Related Links
• License

What is AgentAudit?

AgentAudit is a security scanner purpose-built for the AI package ecosystem. It works in two modes:

1. CLI tool — Run agentaudit in your terminal to discover and scan MCP servers installed in your AI editors
2. MCP server — Add to Claude Desktop, Cursor, or Windsurf so your AI agent can audit packages on your behalf

It checks packages against the AgentAudit Trust Registry — a shared, community-driven database of security findings — and can perform local scans ranging from fast regex analysis to deep LLM-powered 3-pass audits.

🚀 Quick Start

Option A: CLI (recommended)

# Install globally (or use npx agentaudit)
npm install -g agentaudit

# Discover MCP servers configured in your AI editors
agentaudit

# Quick scan — clones repo, checks code with regex patterns (~2s)
agentaudit scan https://github.com/owner/repo

# Deep audit — clones repo, sends code to LLM for 3-pass analysis (~30s)
agentaudit audit https://github.com/owner/repo

# Registry lookup — check if a package has been audited before (no cloning)
agentaudit lookup fastmcp

Example output:

  ◆ AgentAudit v3.13.4  │  my-scanner · #3 · 280pts · 19 audits

  Discovering MCP servers in your AI editors...

•  Scanning Cursor  ~/.cursor/mcp.json    found 3 servers

├──  tool   supabase-mcp              ✔ ok
│   SAFE  Risk 0  https://agentaudit.dev/packages/supabase-mcp
├──  tool   browser-tools-mcp         ✔ ok
│   ⚠ not audited  Run: agentaudit audit https://github.com/nichochar/browser-tools-mcp
└──  tool   filesystem                ✔ ok
│   SAFE  Risk 0  https://agentaudit.dev/packages/filesystem

  Looking for general package scanning? Try `pip audit` or `npm audit`.

Enhanced banner: When logged in, the banner shows your agent name, rank, points, and audit count. Run agentaudit setup to create an account.

Option B: MCP Server in your AI editor

Add AgentAudit as an MCP server — your AI agent can then discover, scan, and audit packages using its own LLM. No extra API key needed.

{
  "mcpServers": {
    "agentaudit": {
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  }
}

{
  "mcpServers": {
    "agentaudit": {
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  }
}

{
  "mcpServers": {
    "agentaudit": {
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  }
}

{
  "servers": {
    "agentaudit": {
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  }
}

{
  "mcpServers": [
    {
      "name": "agentaudit",
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  ]
}

{
  "context_servers": {
    "agentaudit": {
      "command": {
        "path": "npx",
        "args": ["-y", "agentaudit", "--stdio"]
      }
    }
  }
}

Then ask your agent: "Check which MCP servers I have installed and audit any unaudited ones."

📋 Commands Reference

Scan & Audit

Community

Configuration

Global Flags

Exit Codes

⚖️ Quick Scan vs Deep Audit

Tip: Use agentaudit scan <url> --deep to run a deep audit via the scan command.

🔌 MCP Server

When running as an MCP server, AgentAudit exposes the following tools to your AI agent:

Workflow

User asks agent to install a package
         │
         ▼
Agent calls check_registry(package_name)
         │
    ┌────┴────┐
    │         │
  Found    Not Found
    │         │
    ▼         ▼
 Return    Agent calls audit_package(repo_url)
 score        │
              ▼
         LLM analyzes code (3-pass)
              │
              ▼
         Agent calls submit_report(findings)
              │
              ▼
         Return findings + risk score

🎯 What It Detects

🧠 How the 3-Pass Audit Works

The deep audit (agentaudit audit) uses a structured 3-phase LLM analysis — not a single-shot prompt, but a rigorous multi-pass process:

Why 3 passes? Single-pass analysis is the #1 cause of false positives. By separating understanding → detection → classification:

• Phase 1 prevents flagging core functionality as suspicious (e.g., SQL execution in a database tool)
• Phase 2 ensures evidence is collected without severity bias
• Phase 3 catches false positives before they reach the report

This architecture achieved 0% false positives on our 11-package test set, down from 42% in v2.

Adversarial Verification Pass (v3.14+)

After the 3-pass audit, a verification pass re-examines each finding against the actual source code. Verification is auto-enabled when uploading to the registry to ensure data quality. For local-only scans, use --no-verify to skip it (this also disables registry upload).

# Verification runs automatically when uploading (default behavior)
agentaudit audit https://github.com/owner/repo

# Explicit verification mode
agentaudit audit https://github.com/owner/repo --verify cross

# Skip verification + upload (local-only, fast)
agentaudit audit https://github.com/owner/repo --no-verify

Each finding goes through a 5-point checklist:

1. Code Existence — Does the cited code actually exist in the file?
2. Context Accuracy — Is the code used in the way described?
3. Execution Model — Can an attacker actually trigger this?
4. Severity Calibration — Is the severity appropriate?
5. Fabrication Check — Are there hallucinated details?

Verdicts: verified (confirmed real), demoted (severity reduced), rejected (false positive removed).

Why require verification for uploads? LLMs can hallucinate code that doesn't exist or overstate severity. Without verification, false positives enter the public registry and unfairly flag packages as unsafe. The verification pass catches these before they become permanent records.

Model Accuracy (Real-World Data)

We benchmarked multiple LLMs on the Top 20 most popular MCP servers (62+ reports):

Key insight: Model choice dramatically affects audit quality. We recommend Claude Opus 4 or Claude Sonnet 4 for production audits. Use --models to run multiple models and compare results via consensus.

🔄 CI/CD Integration

AgentAudit is designed for CI pipelines with proper exit codes and JSON output:

# GitHub Actions example
- name: Scan MCP servers
  run: |
    npx agentaudit scan https://github.com/org/mcp-server --json --quiet > results.json
    # Exit code 1 = findings detected → fail the build

# Shell scripting
agentaudit scan https://github.com/owner/repo --json --quiet 2>/dev/null
if [ $? -eq 1 ]; then
  echo "Security findings detected!"
  exit 1
fi

JSON Output Examples

# Scan with JSON output
agentaudit scan https://github.com/owner/repo --json

{
  "slug": "repo",
  "url": "https://github.com/owner/repo",
  "findings": [
    {
      "severity": "high",
      "title": "Command injection risk",
      "file": "src/handler.js",
      "line": 42,
      "snippet": "exec(`git ${userInput}`)"
    }
  ],
  "fileCount": 15,
  "duration": "1.8s"
}

# Registry lookup with JSON
agentaudit lookup fastmcp --json

Coming soon: --fail-on <severity> flag to set minimum severity threshold for non-zero exit (e.g., --fail-on high ignores low/medium findings).

📊 Dashboard & Community

AgentAudit includes a full-screen interactive dashboard and standalone community commands.

Interactive Dashboard

agentaudit dashboard    # or: agentaudit dash

5-tab TUI with keyboard navigation (←→ tabs, ↑↓ scroll, 1-5 jump, q quit). Overview tab includes interactive Quick Actions — select and launch audits, consensus views, or remote scans directly from the dashboard:

Standalone Commands

All community commands work without the dashboard (pipe-friendly, supports --json):

agentaudit leaderboard              # Top contributors
agentaudit leaderboard --tab monthly --json   # Monthly rankings as JSON
agentaudit benchmark                # Model comparison
agentaudit activity                 # Your recent audits & findings
agentaudit search fastmcp           # Search registry by name/ASF-ID
agentaudit search fastmcp --json    # Machine-readable search results

⚙️ Configuration

Credentials

AgentAudit stores credentials in ~/.config/agentaudit/credentials.json (or $XDG_CONFIG_HOME/agentaudit/credentials.json).

Run agentaudit setup to sign in with GitHub or paste an API key, or set via environment:

export AGENTAUDIT_API_KEY=asf_your_key_here

LLM Providers (13 supported)

AgentAudit supports 13 LLM providers for deep audits. Set one API key — the CLI auto-detects it. Use agentaudit model to choose provider + model interactively, or agentaudit status to check your setup.

Other Environment Variables

Provider priority: Set preferred_provider via agentaudit model, or the CLI picks the first available key. Override per-run with --model <name>.

📦 Requirements

• Node.js ≥ 18.0.0
• Git (for cloning repositories during scan/audit)

❓ FAQ

How do I set up AgentAudit?

npm install -g agentaudit
agentaudit setup

Or use without installing: npx agentaudit

Do I need an API key?

• Quick scan (scan): No API key needed — runs locally with regex
• Deep audit (audit): Needs an LLM API key (see below)
• Registry lookup (lookup): No key needed for reading; key needed for uploading reports
• MCP server: No extra key needed — uses the host editor's LLM

Setting up your LLM key for deep audits

The audit command supports 13 LLM providers. Set one API key and AgentAudit auto-detects it:

# Set any one of these (Anthropic recommended)
export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...
export GEMINI_API_KEY=...
export DEEPSEEK_API_KEY=...
# ... or any of the 13 supported providers (see Configuration section)

Interactive setup:

agentaudit model     # 2-step menu: pick provider → pick model
agentaudit status    # check which keys are set + current config

Override per-run:

agentaudit audit https://github.com/owner/repo --model gpt-4o

Troubleshooting: If you see API error: Incorrect API key, double-check your key is valid and has credits. Use --debug to see the full API response.

What data is sent externally?

• Registry lookups: Package name/slug is sent to agentaudit.dev to check for existing audits
• Report uploads: Audit findings are uploaded to the public registry (requires API key)
• Deep audits: Source code is sent to Anthropic or OpenAI for LLM analysis
• Quick scans: Everything stays local — no data leaves your machine

Can I use it offline?

Quick scans (agentaudit scan) work fully offline after cloning. Registry lookups and deep audits require network access.

Can I use it as an MCP server without the CLI?

Yes! npx agentaudit starts the MCP server when invoked by an editor. The CLI and MCP server are the same package — behavior is determined by how it's called.

How does discover know which editors I use?

It checks standard config file locations for Claude Desktop, Cursor, VS Code, and Windsurf. It also checks the current working directory for project-level .cursor/mcp.json and .vscode/mcp.json.

🔗 Related

📄 License

AGPL-3.0 — Free for open source use. Commercial license available for proprietary integrations.