LiteLLM Agent MCP Server

Give your AI agent access to 100+ LLMs.

This MCP server lets AI agents (Claude Code, Cursor, etc.) call any LLM through LiteLLM's unified API. Stop being limited to one model — use the right model for each task.

Why?

AI agents are typically stuck on a single model. With this MCP server, your agent can:

• 🔀 Call any model — GPT-4, Claude, Gemini, Mistral, and 100+ more
• ⚖️ Compare outputs — Get responses from multiple models and pick the best
• 🎯 Use the right tool — Code tasks → GPT-4, writing → Claude, long docs → Gemini
• 💰 Save costs — Route simple queries to cheaper models

Tools

Tool	Description
call	Call any LLM model (OpenAI chat completions format)
responses	Use OpenAI Responses API format (stateful, tools, structured output)
messages	Use Anthropic Messages API format (native Claude format)
generate_content	Use Google generateContent format (native Gemini format)
compare	Compare responses from multiple models
models	List available models and their strengths
recommend	Get model recommendation for a task type

Installation

Claude Desktop / Cursor

Add to your MCP config:

{
  "mcpServers": {
    "litellm": {
      "command": "python",
      "args": ["-m", "litellm_agent_mcp"],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-..."
      }
    }
  }
}

From PyPI

pip install litellm-agent-mcp

From Source

git clone https://github.com/BerriAI/litellm-agent-mcp
cd litellm-agent-mcp
pip install -e .

Usage Examples

Call a specific model

Use the `call` tool:
- model: "gpt-4o"  
- prompt: "Explain this code: [code here]"

Compare multiple models

Use the `compare` tool:
- models: ["gpt-4o", "claude-sonnet-4-20250514"]
- prompt: "What's the best approach to implement caching?"

Get a recommendation

Use the `recommend` tool:
- task_type: "code"

→ Returns: gpt-4o (Strong at code generation, debugging, and review)

Environment Variables

Set API keys for the providers you want to use:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-...
GEMINI_API_KEY=...
MISTRAL_API_KEY=...

Or point to a LiteLLM proxy:

LITELLM_API_BASE=https://your-proxy.com
LITELLM_API_KEY=sk-...

Supported Models

Provider	Models
OpenAI	gpt-4o, gpt-4o-mini, o1-preview, o1-mini
Anthropic	claude-sonnet-4, claude-opus-4
Google	gemini-1.5-pro, gemini-1.5-flash
Mistral	mistral-large-latest
+ 100 more	See LiteLLM docs

License

MIT

Links

• LiteLLM Docs
• LiteLLM GitHub
• MCP Spec