CopilotLocalRouter

What is this?

GitHub Copilot, Claude Code, and Cursor send every request to an expensive cloud model — even trivial ones like "write a method to add two integers" or "explain what this loop does." CopilotLocalRouter sits between your AI assistant and the cloud, intercepting simple tasks and handling them locally with Ollama.

Complex tasks (multi-file refactors, architecture decisions) are passed through to the cloud unchanged.

Your AI Assistant (Copilot / Claude / Cursor)
         │
         ▼  MCP stdio transport
  ┌─────────────────────┐
  │  CopilotLocalRouter │
  │  ┌───────────────┐  │
  │  │  Classifier   │  │  ← scores prompt complexity
  │  └──────┬────────┘  │
  └─────────┼───────────┘
            │
     Simple / Medium ──────────► Ollama (local LLM, ~50ms)
            │
          Complex
            │
            ▼
       [skip] signal ──────────► Cloud model (unchanged)

Features

• 5 MCP tools — generate, explain, refactor, review, and test generation
• Automatic routing — heuristic classifier scores every prompt; no manual tagging required
• Transparent fallback — returns [skip] so your AI assistant silently falls back to the cloud
• Circuit breaker — if Ollama goes down, requests fall through to cloud immediately
• Prompt cache — identical prompts served from an LRU cache (SHA256-keyed, 60 min TTL)
• Cost tracking — logs estimated token savings every N requests
• OTel metrics — System.Diagnostics.Metrics compatible; connect any OTel collector
• Zero config default — works out of the box with qwen2.5-coder on localhost:11434

Table of Contents

• Prerequisites
• Installation
• Quick Start
• Supported AI Clients
• Available Tools
• Recommended Models
• Configuration
• How Routing Works
• Architecture
• Resilience
• Telemetry
• Contributing
• Troubleshooting
• Changelog
• License

Prerequisites

Installation

There is nothing to install. dnx is a tool execution script included with the .NET 10 SDK that works like npx — it downloads and runs a .NET tool on demand. Add the config block for your AI client (see Quick Start) and dnx handles the rest automatically.

Pinning a version: Use CopilotLocalRouter@0.1.0 in the args array to lock to a specific release. Omitting the version always uses the latest.

Build from Source

git clone https://github.com/michaelstonis/CopilotLocalRouter.git
cd CopilotLocalRouter
dotnet build
dotnet test

Quick Start

1. Start Ollama and pull a model:

ollama pull qwen2.5-coder

2. Add to your AI client's MCP config (no prior install needed — dnx downloads the tool on first run):

{
  "servers": {
    "copilot-local-router": {
      "command": "dnx",
      "args": ["CopilotLocalRouter"],
      "env": {
        "OLLAMA_BASE_URL": "http://localhost:11434",
        "OLLAMA_MODEL": "qwen2.5-coder"
      }
    }
  }
}

{
  "mcpServers": {
    "copilot-local-router": {
      "command": "dnx",
      "args": ["CopilotLocalRouter"],
      "env": {
        "OLLAMA_BASE_URL": "http://localhost:11434",
        "OLLAMA_MODEL": "qwen2.5-coder"
      }
    }
  }
}

claude mcp add copilot-local-router dnx -- CopilotLocalRouter

{
  "mcpServers": {
    "copilot-local-router": {
      "command": "dnx",
      "args": ["CopilotLocalRouter"],
      "env": {
        "OLLAMA_BASE_URL": "http://localhost:11434",
        "OLLAMA_MODEL": "qwen2.5-coder"
      }
    }
  }
}

3. Restart your AI client and verify:

Ask your assistant: "What MCP tools are available?"

You should see: local_code_generate, local_code_explain, local_code_refactor, local_code_review, local_test_generate.

Supported AI Clients

Available Tools

Recommended Models

Tip: qwen2.5-coder is the recommended default — it offers the best balance of speed, quality, and memory footprint for code tasks.

Configuration

All configuration is done via environment variables in your MCP client config, or as appsettings.json overrides when building from source.

Environment Variables

Routing Thresholds

Control how aggressively tasks are routed locally by setting these in appsettings.json (source builds) or by contributing an env-var override:

More aggressive local routing (accept more tasks locally, at lower quality confidence):

"SimpleConfidenceThreshold": 0.60,
"MediumConfidenceThreshold": 0.40

More conservative (only high-confidence simple tasks handled locally):

"SimpleConfidenceThreshold": 0.90,
"MediumConfidenceThreshold": 0.70

Cost Tracking

"CostTracking": {
  "Enabled": true,
  "CloudInputTokenRate": 0.003,
  "CloudOutputTokenRate": 0.015,
  "Currency": "USD",
  "LogSummaryEveryNRequests": 100
}

Rates are per 1,000 tokens. The estimator uses conservative sizing based on average Ollama response sizes.

How Routing Works

Every prompt is scored by a heuristic classifier across four signals:

1. Keyword analysis — task verbs (generate, explain, refactor) and complexity indicators (across all, entire codebase, architecture)
2. Token count — prompts over 500 tokens are unlikely to be simple tasks
3. Structural signals — multi-file references, import counts, line counts
4. Task type match — each tool has a baseline complexity expectation

Scores above SimpleConfidenceThreshold → local. Scores above MediumConfidenceThreshold → local (medium tier). Everything else → [skip].

Architecture

src/
  CopilotLocalRouter.Core/        # Domain logic
    Classification/               # Heuristic task classifier
    Routing/                      # Tiered request router
    Agents/                       # AgentManager — Ollama conversation executor
    Resilience/                   # Circuit breaker, LRU prompt cache, normalizer
    Telemetry/                    # Metrics, cost estimator, quality scorer
    Configuration/                # RouterOptions, ModelProfile
    Interfaces/                   # IRequestRouter, IAgentManager, ITaskClassifier
  CopilotLocalRouter.Ollama/      # OllamaSharp IChatClient integration
  CopilotLocalRouter.McpTools/    # MCP tool definitions (5 tools)
  CopilotLocalRouter.Host/        # Startup, DI, appsettings, health check

tests/
  CopilotLocalRouter.Core.Tests/         # Unit tests — classifier, resilience, telemetry
  CopilotLocalRouter.McpTools.Tests/     # Integration tests — MCP tool end-to-end

benchmarks/
  CopilotLocalRouter.Benchmarks/         # BenchmarkDotNet — classification + cache key perf

Key dependencies:

Resilience

Telemetry

OTel-compatible metrics emitted via System.Diagnostics.Metrics (meter name: CopilotLocalRouter):

Connect any OpenTelemetry collector (Prometheus, Jaeger, Grafana, etc.) to the standard OTel endpoint.

Contributing

Contributions are welcome. Please open an issue first if you're planning a significant change.

git clone https://github.com/michaelstonis/CopilotLocalRouter.git
cd CopilotLocalRouter
dotnet restore
dotnet build
dotnet test

Guidelines:

• All new routing logic should have classifier unit tests
• MCP tool changes require integration tests in CopilotLocalRouter.McpTools.Tests
• Keep tool descriptions tightly tuned — they directly affect AI agent tool selection

Troubleshooting

See docs/troubleshooting.md for common issues including:

• Ollama not reachable / tools not appearing in assistant
• All tasks returning [skip] (threshold tuning)
• Circuit breaker stuck open
• Model not found errors

Changelog

See CHANGELOG.md for release history.

License

MIT — see LICENSE for details.