KAOS: K8s Agent Orchestration System

KAOS is a Kubernetes-native framework for deploying and orchestrating AI agents with tool access, multi-agent coordination, and seamless LLM integration.

Features

Quick Start

Prerequisites

• Kubernetes cluster
• kubectl configured
• helm installed

Option 1: KAOS CLI/UI

For CLI/UI documentation, see the CLI Guide.

# Install the CLI
pip install kaos-cli

# Install KAOS in your cluster
kaos system install

# Create a few resources
kaos modelapi deploy my-api --mode Hosted --model "smollm2:135m"
kaos agent deploy my-agent --modelapi my-api --model "smollm2:135m"

# Send a request
kaos agent invoke my-agent --message "Hello"
# 📤 Response:
# Hello! How can I assist you today? If you have any questions, feel free to share them! 

# Open the UI
kaos ui

This command opens axsaucedo.github.io/kaos-ui and runs a kubectl proxy.

You can also open the UI with telemetry enabled:

kaos ui --monitoring-enabled

Make sure you enable observability.

Option 2: Helm/kubectl

# Add the Helm repository
helm repo add kaos https://axsaucedo.github.io/kaos/charts
helm repo update

# Install the operator
helm install kaos kaos/kaos-operator -n kaos-system --create-namespace

Deploy Your First Agent

# simple-agent.yaml
apiVersion: kaos.tools/v1alpha1
kind: ModelAPI
metadata:
  name: ollama
spec:
  mode: Hosted
  hostedConfig:
    model: "smollm2:135m"

---
apiVersion: kaos.tools/v1alpha1
kind: MCPServer
metadata:
  name: echo-tools
spec:
  type: python-runtime
  config:
    tools:
      fromString: |
        def echo(message: str) -> str:
            """Echo back the message."""
            return f"Echo: {message}"

---
apiVersion: kaos.tools/v1alpha1
kind: Agent
metadata:
  name: assistant
spec:
  modelAPI: ollama
  mcpServers:
    - echo-tools
  config:
    description: "AI assistant with echo tools"
    instructions: "You are a helpful assistant."
    env:
      - name: MODEL_NAME
        value: "ollama/smollm2:135m"

kubectl apply -f simple-agent.yaml

# Wait for pods to be ready
kubectl wait --for=condition=ready pod -l agent=assistant --timeout=120s

# Port-forward and test
kubectl port-forward svc/agent-assistant 8000:8000
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "assistant", "messages": [{"role": "user", "content": "Hello!"}]}'

Multi-Agent Systems

KAOS supports hierarchical multi-agent systems where a coordinator delegates tasks to specialist agents:

apiVersion: kaos.tools/v1alpha1
kind: Agent
metadata:
  name: coordinator
spec:
  modelAPI: ollama
  config:
    description: "Coordinator that delegates to specialists"
    instructions: "Delegate research to researcher, calculations to analyst."
  agentNetwork:
    access:
      - researcher
      - analyst

See operator/config/samples/ for complete multi-agent examples.

Architecture

flowchart TB
    subgraph operator["KAOS Operator"]
        ac["Agent Controller"]
        mc["MCPServer Controller"]
        mac["ModelAPI Controller"]
    end
    
    subgraph resources["Managed Resources"]
        agent["Agent Pod<br/>Agent Runtime"]
        mcp["MCP Server Pod<br/>MCP Tools"]
        model["ModelAPI Pod<br/>Ollama/LiteLLM"]
    end
    
    ac --> agent
    mc --> mcp
    mac --> model
    agent --> mcp
    agent --> model

Agent Runtime: Pydantic AI Server (PAIS)

KAOS agents are powered by PAIS — an enterprise server wrapper for Pydantic AI. PAIS adds OpenAI-compatible HTTP API, distributed memory, multi-agent delegation, A2A discovery, health probes, and OpenTelemetry instrumentation on top of Pydantic AI agents.

Documentation

Development

# Python tests
cd python && uv sync && uv run pytest tests/ -v

# Go tests  
cd operator && make test

# E2E tests (requires kind)
cd operator && make kind-create
cd operator && make kind-e2e-run-tests

Sample Configurations

See operator/config/samples/ for examples:

1. Simple Agent - Single agent with echo MCP tool
2. Multi-Agent - Coordinator with worker agents
3. Hierarchical - Multi-level agent hierarchy
4. Custom Tools - Dynamic tool creation with tools.fromString

License

Apache 2.0