Beyond MCP

It's time to push beyond MCP Servers... Right?

Let's breakdown real engineering trade offs between MCP, CLI, File System Scripts, and Skills based approaches for building reusable toolsets for your AI Agents.

Watch the full video breakdown here: Beyond MCP

Purpose of this Repo

• MCP Servers are the standard way to build reusable toolsets for your AI Agents. But they are not the only way.
• MCP Servers come with a massive cost - instant context loss.
• When you have a single, or a few MCP Servers, this is not a big deal. But as you scale to many agents, many tools, and many contexts - this cost quickly becomes a bottleneck.
• So what are the alternatives that big players are using to build powerful, reusable, context preserving toolsets for their AI Agents?

Here we explore 4 concrete approaches in this repo, all implementing access to Kalshi prediction market data.

The 4 Approaches

apps/1_mcp_server/ - MCP Server

apps/2_cli/ - CLI

apps/3_file_system_scripts/ - File System Scripts

apps/4_skill/ - Skill

Quick Start

1. MCP Server

cp .mcp.testing .mcp.json

claude --mcp-config .mcp.json

prompt: "kalshi: get exchange status"

2. CLI

# or by agent
claude

prompt: "/prime_kalshi_cli_tools"

prompt: "kalshi: Get exchange status"

prompt: "kalshi: List events"

prompt: "kalshi: List events in JSON"

prompt: "kalshi: List events in JSON, limit 100"

# or by hand
cd apps/2_cli
uv sync
uv run kalshi status
uv run kalshi events
uv run kalshi events --json
uv run kalshi events --json --limit 100

3. File System Scripts

# by agent
claude

prompt: "/prime_file_system_scripts"

prompt: "kalshi: Get exchange status"

prompt: "kalshi: List events"

...

# or by hand
cd apps/3_file_system_scripts/scripts

uv run status.py

uv run *.py

4. Skill

cd apps/4_skill/

claude

prompt: "kalshi markets: Get exchange status"

prompt: "kalshi markets: search for events about 'best ai'" # Note this will trigger the cache build on first run which will take several minutes

...

The 4 Approaches In Detail

• apps/1_mcp_server/ - MCP Server
• apps/2_cli/ - CLI
• apps/3_file_system_scripts/ - File System Scripts
• apps/4_skill/ - Skill

1. MCP Server (apps/1_mcp_server/)

Classic Model Context Protocol implementation

• ✅ Standardized integration - Works with any MCP-compatible client
• ✅ Tool discovery - Auto-exposes 15 tools to LLMs
• ✅ Clean abstractions - MCP protocol handles complexity
• ❌ Instant context loss - Every tool call loses conversational context
• ❌ Wrapper overhead - Delegates to CLI via subprocess

Architecture:

Claude/LLM → MCP Protocol → MCP Server → subprocess → CLI → Kalshi API

Key files:

• server.py - FastMCP server with 15 tool definitions
• Wraps CLI commands in MCP tool interface
• Each tool call is stateless

When to use: Building tools for multiple LLM clients, need standardized protocol, context loss is acceptable.

---

2. CLI (apps/2_cli/)

Direct HTTP API access via command-line interface

• ✅ Single source of truth - Direct API calls, no wrappers
• ✅ Dual output modes - Human-readable or pure JSON
• ✅ Smart caching - Pandas-based search with 6-hour TTL
• ✅ Minimal overhead - Direct httpx calls, no SDK
• ✅ Improved Context - Agent reads ~half as much context as the MCP Server

Architecture:

Claude → subprocess → CLI (13 commands) → Direct HTTP → Kalshi API

Key files:

• kalshi_cli/cli.py - All 13 commands (552 lines)
• kalshi_cli/modules/client.py - HTTP client & search cache
• kalshi_cli/modules/formatting.py - Output formatters

When to use: Need direct API control, want both CLI and programmatic access, caching important, okay with subprocess overhead.

---

3. File System Scripts (apps/3_file_system_scripts/)

Progressive disclosure via standalone scripts

• ✅ Progressive disclosure - Only load scripts you need (~200-300 lines each)
• ✅ Complete isolation - Each script is fully self-contained
• ✅ Zero dependencies - HTTP client embedded in each script
• ✅ Context efficient - Agent only reads relevant scripts
• ⚠️ Code duplication - HTTP client repeated in each script
• ⚠️ No shared state - Cache and utilities duplicated

Architecture:

Claude → Read tool → Individual script → Embedded HTTP client → Kalshi API

Available scripts (10):

• status.py - Exchange operational status
• markets.py - Browse markets with filters
• market.py - Detailed market information
• orderbook.py - Bid/ask depth
• trades.py - Recent trading activity
• search.py - Keyword search (with caching)
• events.py - List event collections
• event.py - Event details
• series_list.py - Browse all ~6900 series
• series.py - Series information

When to use: Context preservation critical, want progressive disclosure, okay with code duplication, need standalone portability.

---

4. Skill (apps/4_skill/.claude/skills/kalshi-markets/)

Claude Code Agent Skills with embedded scripts

• ✅ Model-invoked - Claude autonomously decides when to use
• ✅ Progressive disclosure - Same scripts as approach #3
• ✅ Team sharing - Commit to git for team access
• ✅ Discovery - Description triggers automatic activation
• ✅ Context preservation - Agent reads only what's needed
• ⚠️ Claude Code specific - Only works in Claude Code
• ⚠️ Learning curve - Requires understanding Skill system

Architecture:

Claude (detects trigger) → Loads SKILL.md → Runs scripts → Kalshi API

Structure:

.claude/skills/kalshi-markets/
├── SKILL.md (concise description & instructions)
└── scripts/ (copies of all 10 file system scripts)

When to use: Using Claude Code, want automatic skill discovery, team collaboration via git, need context preservation with progressive disclosure.

---

My Approach (IndyDevDan)

External Tools

1. 80% Just use MCP servers. Don't overthink it.
2. 15% CLI - If you need modify, extend, or control tools and context.
3. 5% Scripts or Skills - For serious context preservation, portability or ecosystem reuse

New Tools

1. 80% Just use CLI + Prime Prompt (works for you, your team and your agents).
2. 10% Wrap in MCP Server when I need multiple agents at scale - and don't want to add 'another' thing for my agents to focus on.
3. 10% Scripts or Skills - For serious context preservation, portability or ecosystem reuse.

Key Technical Details

API Access:

• Base URL: https://api.elections.kalshi.com/trade-api/v2
• No authentication required (read-only public data)
• ~6900 market series available

Search Caching:

The Kalshi API doesn't provide a native search endpoint, which creates a challenge for finding markets by keyword. Our solution: intelligent local caching.

• The Problem: No API search endpoint means we'd need to paginate through thousands of markets on every search
• The Solution: Build a complete local cache once, then search instantly using pandas
• First run: 2-5 minutes to fetch all ~6900 markets and build cache
• Subsequent searches: Instant (searches cached pandas DataFrame)
• Cache location: .kalshi_cache/ at project root (shared across CLI and scripts)
• TTL: 6 hours (auto-refresh when stale)
• Search scope: Searches titles, subtitles, tickers, series names, and descriptions

Why the delay matters:

• First search in a session will take 2-5 minutes while the cache builds
• Users will see progress messages during cache building
• After initial build, searches are instant for 6 hours
• This trade-off enables comprehensive keyword search across ALL markets instead of just the first 100-500 results from paginated API calls

Path Resolution:

• All scripts use absolute path resolution via Path(__file__).resolve()
• Works correctly when invoked from any directory
• Cache always resolves to project root

Trade-off Comparison

	MCP	CLI	Scripts	Skills
Agent Invoked	Yes	No	No	Yes
Context Window Consumption	High	Medium (Depends)	Low (w/incr)	Low (w/incr)
Customizable	No (unless you own)	Yes	Yes	Yes
Portability	Low	Medium	High	High
Composability	Yes (MCP Prompts)	Yes but requires local prompts	Yes but requires local prompts	Yes but requires local prompts
Simplicity	High	Medium	Medium	Medium
Engineering Investment	Low if external, Medium if custom	Medium	Medium	Low if external, Medium if custom
Feature Set	Tools, Resources, Prompts, Elicitation, Completion, Sampling, Logging, Auth. etc	Whatever you build	Whatever you build	Whatever you build

Key Insights

Context Window Consumption:

• MCP & CLI consume full context on every tool call
• Scripts & Skills use progressive disclosure - only load what's needed

Agent Invoked:

• MCP & Skills are automatically triggered by Claude based on context
• CLI & Scripts require explicit agent decision to use

Customizable:

• MCP is locked unless you own/fork the server
• CLI, Scripts, Skills are fully under your control

Portability:

• Scripts & Skills are most portable (just Python files)
• CLI requires installation but works anywhere
• MCP needs MCP-compatible client setup

When to Use Each Approach

Choose MCP Server if:

• Building for multiple LLM clients (not just Claude)
• Need standardized tool protocol
• Context loss per call is acceptable
• Want automatic tool discovery across clients
• Using external MCP servers you don't control

Choose CLI if:

• Need both human CLI and programmatic access
• Want single source of truth for API logic
• Direct HTTP control is important
• Willing to accept subprocess overhead
• Building general-purpose tooling

Choose File System Scripts if:

• Context preservation is critical
• Want maximum portability (just Python + httpx)
• Need progressive disclosure (minimize token usage)
• Okay with code duplication for isolation
• Building one-off integrations

Choose Skill if:

• Using Claude Code (and the ecosystem) specifically
• Want autonomous skill discovery
• Team collaboration via git is important
• Need context preservation + progressive disclosure
• Building reusable team capabilities

Project Structure

beyond-mcp/
├── apps/
│   ├── 1_mcp_server/          # MCP Server implementation
│   │   ├── server.py           # 15 MCP tools wrapping CLI
│   │   └── README.md
│   ├── 2_cli/                  # CLI implementation
│   │   ├── kalshi_cli/
│   │   │   ├── cli.py          # 13 commands (552 lines)
│   │   │   └── modules/        # HTTP client, cache, formatters
│   │   └── README.md
│   ├── 3_file_system_scripts/  # Progressive disclosure scripts
│   │   ├── scripts/            # 10 standalone scripts
│   │   │   ├── status.py
│   │   │   ├── markets.py
│   │   │   ├── market.py
│   │   │   ├── orderbook.py
│   │   │   ├── trades.py
│   │   │   ├── search.py
│   │   │   ├── events.py
│   │   │   ├── event.py
│   │   │   ├── series_list.py
│   │   │   └── series.py
│   │   └── README.md
│   └── 4_skill/                # Claude Code Skill
│       └── .claude/skills/kalshi-markets/
│           ├── SKILL.md        # Skill description & instructions
│           └── scripts/        # Same 10 scripts as #3
└── .kalshi_cache/              # Shared cache directory (CLI & scripts)

Resources

• MCP Python SDK
• Claude Code Skills Documentation
• Kalshi API Documentation
• FastMCP Framework

Master Agentic Coding

Prepare for the future of software engineering

Learn tactical agentic coding patterns with Tactical Agentic Coding.

Follow the IndyDevDan YouTube channel to improve your agentic coding advantage.