AI DevOps Actions

🔴 Live example: Open PR #1 (canonical demo) or any recent PR to see the full suite running on itself — context summary, quality score, and root cause hints, all from GITHUB_TOKEN.

The full CI/CD layer for AI-native development — 8 GitHub Actions covering PR quality, safety, cost, infra, and behavioral testing.

AI-native repos have problems that standard CI/CD doesn't solve. PRs flooded with AI slop. Unchecked LLM spend. Sensitive data leaking through AI outputs. MCP servers shipped without validation. Action tags silently compromised. Agent skills published without schema checks. Behavioral regressions invisible until production.

This suite covers the full stack — eight GitHub Actions that work independently or as a pipeline.

🧠 The AI Reliability Loop

Most CI tells you something broke. This system tells you why — and what to do next.

Three things this loop does that standard CI/CD can't:

Example output when your AI pipeline breaks:

❌ Eval failed

Root Cause Analysis:
→ Knowledge drift (HIGH confidence)
→ RAG corpus changed 2 commits ago
→ Eval suite: 3/12 assertions failed

Suggested fix:
→ Re-run evals with updated embeddings
→ Check: ai-workflow-evals + llm-cost-tracker

AI systems don't fail like code — they degrade silently, drift with data, and pass tests while producing worse outputs. This is the CI layer that catches it.

⚡️ 1-minute setup

1. Copy this into .github/workflows/ai-hygiene.yml in any repo:

name: AI PR Hygiene
on:
  pull_request:
    types: [opened, synchronize, reopened]
jobs:
  hygiene:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: ollieb89/pr-context-enricher@v1.0.0
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
      - uses: ollieb89/ai-pr-guardian@v1.0.0
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          threshold: 60
      - uses: ollieb89/ai-root-cause-hints@v1.0.0
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

1. Open a PR.

2. You'll see:

🔍 PR Context Summary
Author: @you | Base: main ← feature/my-change
5 files changed (+120/-30) | Risk: low | Complexity: 3/10
Related issues: #42

✅ PR Quality Score: 78/100

🟢 No correlated failure patterns detected.

No API keys. No external services. Just GITHUB_TOKEN.

Start here

New to the suite? Pick your entry point:

Each action works standalone. The full pipeline shows how they compose.

The Suite

🔍 PR Quality & Context

🛡️ Safety & Security

🧪 Testing & Behavioral Validation

💰 Infrastructure & Cost

Usage paths — start with your problem

Don't know which action to use? Pick your problem:

→ Detailed "when to use which action" guide

Full Pipeline

jobs:
  ai-devops:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      # Enrich PR with context for AI reviewers
      - id: context
        uses: ollieb89/pr-context-enricher@v1.0.0
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

      # Gate AI-generated / low-quality PRs before review
      - uses: ollieb89/ai-pr-guardian@v1.0.0
        with:
          threshold: 60
          on-low-quality: comment

      # Scan AI outputs for secrets and PII before they ship
      - uses: ollieb89/ai-output-redacter@v1.0.0
        with:
          path: ./outputs
          mode: enforce

      # Enforce SHA pinning on all workflow action references
      - uses: ollieb89/actions-lockfile-generator@v1.0.0
        with:
          mode: enforce
          github-token: ${{ secrets.GITHUB_TOKEN }}

      # Catch behavioral regressions in prompts and agents
      - uses: ollieb89/ai-workflow-evals@v1.0.0
        with:
          eval-suite: ./evals
          fail-on: regression

      # Track what this run cost in LLM calls
      - uses: ollieb89/llm-cost-tracker@v1.0.0
        with:
          provider: anthropic
          budget-limit: '1.00'

      # Validate MCP server didn't regress
      - uses: ollieb89/mcp-server-tester@v1.0.0
        with:
          transport: stdio
          server-command: "node dist/server.js"

      # Validate agent skills before publish
      - uses: ollieb89/agent-skill-validator@v1.0.0
        with:
          ecosystem: auto
          fail-on: errors

Install any action independently

# PR Quality & Context
uses: ollieb89/ai-pr-guardian@v1.0.0
uses: ollieb89/pr-context-enricher@v1.0.0

# Safety & Security
uses: ollieb89/ai-output-redacter@v1.0.0
uses: ollieb89/actions-lockfile-generator@v1.0.0

# Testing & Behavioral Validation
uses: ollieb89/ai-workflow-evals@v1.0.0
uses: ollieb89/mcp-server-tester@v1.0.0
uses: ollieb89/agent-skill-validator@v1.0.0

# Infrastructure & Cost
uses: ollieb89/llm-cost-tracker@v1.0.0

All actions are MIT licensed, independently versioned, and production-ready.

Suite stats

Related tools

• workflow-guardian — Workflow health monitoring and linting
• ghact — CLI toolkit: lint workflows, audit security, check for updates
• workflow-linter-vscode — VS Code extension for real-time workflow linting