🛡️ AgentAudit

Security scanner for AI agent packages — CLI + MCP server

Scan MCP servers, AI skills, and packages for vulnerabilities, prompt injection, and supply chain attacks. Powered by regex static analysis and deep LLM audits.

📑 Table of Contents

• What is AgentAudit?
• Quick Start
• Commands Reference
• Quick Scan vs Deep Audit
• MCP Server
• What It Detects
• How the 3-Pass Audit Works
• CI/CD Integration
• Dashboard & Community
• Configuration
• Requirements
• FAQ
• Related Links
• License

What is AgentAudit?

AgentAudit is a security scanner purpose-built for the AI package ecosystem. It works in two modes:

1. CLI tool — Run agentaudit in your terminal to discover and scan MCP servers installed in your AI editors
2. MCP server — Add to Claude Desktop, Cursor, or Windsurf so your AI agent can audit packages on your behalf

It checks packages against the AgentAudit Trust Registry — a shared, community-driven database of security findings — and can perform local scans ranging from fast regex analysis to deep LLM-powered 3-pass audits.

🚀 Quick Start

Option A: CLI (recommended)

# Install globally (or use npx agentaudit)
npm install -g agentaudit

# Discover MCP servers configured in your AI editors
agentaudit

# Quick scan — clones repo, checks code with regex patterns (~2s)
agentaudit scan https://github.com/owner/repo

# Deep audit — clones repo, sends code to LLM for 3-pass analysis (~30s)
agentaudit audit https://github.com/owner/repo

# Registry lookup — check if a package has been audited before (no cloning)
agentaudit lookup fastmcp

Example output:

  ◆ AgentAudit v3.13.4  │  my-scanner · #3 · 280pts · 19 audits

  Discovering MCP servers in your AI editors...

•  Scanning Cursor  ~/.cursor/mcp.json    found 3 servers

├──  tool   supabase-mcp              ✔ ok
│   SAFE  Risk 0  https://agentaudit.dev/packages/supabase-mcp
├──  tool   browser-tools-mcp         ✔ ok
│   ⚠ not audited  Run: agentaudit audit https://github.com/nichochar/browser-tools-mcp
└──  tool   filesystem                ✔ ok
│   SAFE  Risk 0  https://agentaudit.dev/packages/filesystem

  Looking for general package scanning? Try `pip audit` or `npm audit`.

Enhanced banner: When logged in, the banner shows your agent name, rank, points, and audit count. Run agentaudit setup to create an account.

Option B: MCP Server in your AI editor

Add AgentAudit as an MCP server — your AI agent can then discover, scan, and audit packages using its own LLM. No extra API key needed.

{
  "mcpServers": {
    "agentaudit": {
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  }
}

{
  "mcpServers": {
    "agentaudit": {
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  }
}

{
  "mcpServers": {
    "agentaudit": {
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  }
}

{
  "servers": {
    "agentaudit": {
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  }
}

{
  "mcpServers": [
    {
      "name": "agentaudit",
      "command": "npx",
      "args": ["-y", "agentaudit", "--stdio"]
    }
  ]
}

{
  "context_servers": {
    "agentaudit": {
      "command": {
        "path": "npx",
        "args": ["-y", "agentaudit", "--stdio"]
      }
    }
  }
}

Then ask your agent: "Check which MCP servers I have installed and audit any unaudited ones."

📋 Commands Reference

Scan & Audit

Community

Configuration

Global Flags

Exit Codes

⚖️ Quick Scan vs Deep Audit

Tip: Use agentaudit scan <url> --deep to run a deep audit via the scan command.

🔌 MCP Server

When running as an MCP server, AgentAudit exposes the following tools to your AI agent:

Workflow

User asks agent to install a package
         │
         ▼
Agent calls check_registry(package_name)
         │
    ┌────┴────┐
    │         │
  Found    Not Found
    │         │
    ▼         ▼
 Return    Agent calls audit_package(repo_url)
 score        │
              ▼
         LLM analyzes code (3-pass)
              │
              ▼
         Agent calls submit_report(findings)
              │
              ▼
         Return findings + risk score

🎯 What It Detects

🧠 How the 3-Pass Audit Works

The deep audit (agentaudit audit) uses a structured 3-phase LLM analysis — not a single-shot prompt, but a rigorous multi-pass process:

Why 3 passes? Single-pass analysis is the #1 cause of false positives. By separating understanding → detection → classification:

• Phase 1 prevents flagging core functionality as suspicious (e.g., SQL execution in a database tool)
• Phase 2 ensures evidence is collected without severity bias
• Phase 3 catches false positives before they reach the report

This architecture achieved 0% false positives on our 11-package test set, down from 42% in v2.

Adversarial Verification Pass (v3.13+)

After the 3-pass audit, an optional verification pass re-examines each finding against the actual source code:

agentaudit audit https://github.com/owner/repo --verify self

Each finding goes through a 5-point checklist:

1. Code Existence — Does the cited code actually exist in the file?
2. Context Accuracy — Is the code used in the way described?
3. Execution Model — Can an attacker actually trigger this?
4. Severity Calibration — Is the severity appropriate?
5. Fabrication Check — Are there hallucinated details?

Verdicts: verified (confirmed real), demoted (severity reduced), rejected (false positive removed).

Model Accuracy (Real-World Data)

We benchmarked multiple LLMs on the Top 20 most popular MCP servers (62+ reports):

Key insight: Model choice dramatically affects audit quality. We recommend Claude Opus 4 or Claude Sonnet 4 for production audits. Use --models to run multiple models and compare results via consensus.

🔄 CI/CD Integration

AgentAudit is designed for CI pipelines with proper exit codes and JSON output:

# GitHub Actions example
- name: Scan MCP servers
  run: |
    npx agentaudit scan https://github.com/org/mcp-server --json --quiet > results.json
    # Exit code 1 = findings detected → fail the build

# Shell scripting
agentaudit scan https://github.com/owner/repo --json --quiet 2>/dev/null
if [ $? -eq 1 ]; then
  echo "Security findings detected!"
  exit 1
fi

JSON Output Examples

# Scan with JSON output
agentaudit scan https://github.com/owner/repo --json

{
  "slug": "repo",
  "url": "https://github.com/owner/repo",
  "findings": [
    {
      "severity": "high",
      "title": "Command injection risk",
      "file": "src/handler.js",
      "line": 42,
      "snippet": "exec(`git ${userInput}`)"
    }
  ],
  "fileCount": 15,
  "duration": "1.8s"
}

# Registry lookup with JSON
agentaudit lookup fastmcp --json

Coming soon: --fail-on <severity> flag to set minimum severity threshold for non-zero exit (e.g., --fail-on high ignores low/medium findings).

📊 Dashboard & Community

AgentAudit includes a full-screen interactive dashboard and standalone community commands.

Interactive Dashboard

agentaudit dashboard    # or: agentaudit dash

5-tab TUI with keyboard navigation (←→ tabs, ↑↓ scroll, 1-5 jump, q quit). Overview tab includes interactive Quick Actions — select and launch audits, consensus views, or remote scans directly from the dashboard:

Standalone Commands

All community commands work without the dashboard (pipe-friendly, supports --json):

agentaudit leaderboard              # Top contributors
agentaudit leaderboard --tab monthly --json   # Monthly rankings as JSON
agentaudit benchmark                # Model comparison
agentaudit activity                 # Your recent audits & findings
agentaudit search fastmcp           # Search registry by name/ASF-ID
agentaudit search fastmcp --json    # Machine-readable search results

⚙️ Configuration

Credentials

AgentAudit stores credentials in ~/.config/agentaudit/credentials.json (or $XDG_CONFIG_HOME/agentaudit/credentials.json).

Run agentaudit setup to sign in with GitHub or paste an API key, or set via environment:

export AGENTAUDIT_API_KEY=asf_your_key_here

LLM Providers (13 supported)

AgentAudit supports 13 LLM providers for deep audits. Set one API key — the CLI auto-detects it. Use agentaudit model to choose provider + model interactively, or agentaudit status to check your setup.

Other Environment Variables

Provider priority: Set preferred_provider via agentaudit model, or the CLI picks the first available key. Override per-run with --model <name>.

📦 Requirements

• Node.js ≥ 18.0.0
• Git (for cloning repositories during scan/audit)

❓ FAQ

How do I set up AgentAudit?

npm install -g agentaudit
agentaudit setup

Or use without installing: npx agentaudit

Do I need an API key?

• Quick scan (scan): No API key needed — runs locally with regex
• Deep audit (audit): Needs an LLM API key (see below)
• Registry lookup (lookup): No key needed for reading; key needed for uploading reports
• MCP server: No extra key needed — uses the host editor's LLM

Setting up your LLM key for deep audits

The audit command supports 13 LLM providers. Set one API key and AgentAudit auto-detects it:

# Set any one of these (Anthropic recommended)
export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...
export GEMINI_API_KEY=...
export DEEPSEEK_API_KEY=...
# ... or any of the 13 supported providers (see Configuration section)

Interactive setup:

agentaudit model     # 2-step menu: pick provider → pick model
agentaudit status    # check which keys are set + current config

Override per-run:

agentaudit audit https://github.com/owner/repo --model gpt-4o

Troubleshooting: If you see API error: Incorrect API key, double-check your key is valid and has credits. Use --debug to see the full API response.

What data is sent externally?

• Registry lookups: Package name/slug is sent to agentaudit.dev to check for existing audits
• Report uploads: Audit findings are uploaded to the public registry (requires API key)
• Deep audits: Source code is sent to Anthropic or OpenAI for LLM analysis
• Quick scans: Everything stays local — no data leaves your machine

Can I use it offline?

Quick scans (agentaudit scan) work fully offline after cloning. Registry lookups and deep audits require network access.

Can I use it as an MCP server without the CLI?

Yes! npx agentaudit starts the MCP server when invoked by an editor. The CLI and MCP server are the same package — behavior is determined by how it's called.

How does discover know which editors I use?

It checks standard config file locations for Claude Desktop, Cursor, VS Code, and Windsurf. It also checks the current working directory for project-level .cursor/mcp.json and .vscode/mcp.json.

🔗 Related

📄 License

AGPL-3.0 — Free for open source use. Commercial license available for proprietary integrations.