codetect

Fast, token-efficient codebase search for every LLM.

A local MCP server that brings Cursor-like performance to Claude Code and any LLM tool through intelligent codebase indexing.

Why codetect?

LLM coding assistants face two core problems: every question requires multiple slow searches and file reads, and complex queries burn through token limits, killing sessions mid-task.

Cursor solves this through upfront codebase indexing—their speed and token efficiency comes from knowing code structure before answering questions. That capability has been locked to their platform.

codetect brings the same approach to any LLM through intelligent codebase indexing. By building symbol graphs, embeddings, and call relationships, it enables:

• 85.7% accuracy on codebase questions (vs 81.4% without MCP tools)
• Zero token overhead (1.5% fewer tokens than baseline)
• Zero latency overhead (0.3% faster than baseline)
• 4 focused tools instead of multi-step grep/find workflows
• Works with any LLM (Claude, OpenAI, local models via MCP protocol)

See CHANGELOG.md for version history and Migration Guide for upgrade instructions.

Features

• search_keyword - Fast regex search powered by ripgrep, with detail levels (minimal/standard/rich)
• get_file - File reading with optional line-range slicing
• symbols - Symbol lookup and file definition listing (mode=find or mode=list)
• hybrid_search_v2 - Combined keyword + semantic search with reranking and detail levels

Quick Start

# Clone and run interactive installer
git clone https://github.com/brian-lai/codetect.git
cd codetect
./install.sh

The installer will:

• ✓ Check for required dependencies (Go, ripgrep)
• ✓ Guide you through Ollama setup for semantic search (with prominent warnings if missing)
• ✓ Build and install globally to ~/.local/bin
• ✓ Configure your shell PATH automatically

Then in any project:

cd /path/to/your/project
codetect init      # Creates .mcp.json
codetect index     # Index symbols + generate embeddings
claude             # Start Claude Code

Excluding files: Create .codetectignore in your project root to exclude files from indexing (generated code, minified files, test fixtures, etc.). For patterns that apply to all your projects, use the global file at ~/.config/codetect/ignore. Both files use standard .gitignore syntax and are independent of .gitignore. See the .codetectignore Guide for full docs.

See Installation Guide for detailed setup instructions.

Requirements

Dependency	Required	Purpose
Go 1.21+	Yes	Building from source
ripgrep	Yes	Keyword search
Ollama	No	Semantic search (local embeddings)

Note: v3 uses ast-grep for symbol extraction. No external ctags dependency required.

CLI Commands

All commands are subcommands of the single codetect binary:

Main Commands

codetect init        # Create .mcp.json in current directory
codetect index       # Index symbols + generate embeddings (if Ollama available)
codetect embed       # Re-embed with different provider/model settings
codetect doctor      # Check dependencies and configuration
codetect stats       # Show index statistics
codetect help        # Show all commands
codetect version     # Print version

Index flags:

• --force, -f — Re-chunk all files and update locations (embedding cache still used)
• --clear-cache — Clear embedding cache before indexing (forces re-embedding)
• --force --clear-cache — Nuclear option: re-chunk everything AND re-embed from scratch

Indexer features:

• codetect index handles both symbol indexing and embedding in one step
• Incremental indexing with Merkle tree change detection (~2s vs ~30s)
• AST-based chunking preserves semantic boundaries
• Content-addressed caching (95% cache hit rate)

Daemon Commands

codetect daemon start    # Start background indexing daemon
codetect daemon stop     # Stop daemon
codetect daemon status   # Show daemon status

Registry Commands

codetect registry list     # List registered projects
codetect registry add      # Add current project to registry
codetect registry remove   # Remove a project from registry
codetect registry stats    # Show aggregate statistics

Migration (SQLite → PostgreSQL)

codetect migrate-to-postgres   # Migrate embeddings to PostgreSQL

Deprecation Notice (v3.8.0)

The following standalone binaries are deprecated in v3.8.0 and will be removed in v4.0.0. Shim scripts are installed alongside codetect to preserve backward compatibility:

Old binary	Replacement
codetect-index <cmd>	codetect <cmd>
codetect-daemon <cmd>	codetect daemon <cmd>
migrate-to-postgres	codetect migrate-to-postgres

See MIGRATION.md for upgrade instructions.

Evaluation Commands

codetect-eval run --repo <path>     # Run performance evaluation
codetect-eval report                # Show latest results
codetect-eval list                  # List available test cases

MCP Tools

search_keyword

Search for patterns using ripgrep:

{"query": "func main", "top_k": 5, "detail": "standard"}

Parameters: query (required), top_k (default: 10), detail (minimal/standard/rich)

get_file

Read file contents with optional line range:

{"path": "main.go", "start_line": 10, "end_line": 20}

symbols

Find symbols by name or list all definitions in a file:

{"mode": "find", "name": "Server", "kind": "struct", "limit": 20}
{"mode": "list", "path": "internal/mcp/server.go"}

Parameters: mode (find/list, required), name (for find), kind (for find), path (for list), limit (default: 20)

hybrid_search_v2

Combined keyword + semantic search with reranking:

{"query": "authentication", "top_k": 10, "detail": "standard"}

Parameters: query (required), top_k (default: 10), detail (minimal/standard/rich)

Tip: Use bge-m3 embedding model for 47% better retrieval quality. See Embedding Model Comparison.

Configuration

Embedding Provider

Configure embedding provider and model:

# Use Ollama (default)
export CODETECT_EMBEDDING_PROVIDER=ollama

# Recommended: Use bge-m3 for best quality
ollama pull bge-m3
export CODETECT_EMBEDDING_MODEL=bge-m3
export CODETECT_VECTOR_DIMENSIONS=1024

# Or use default (smaller, lower quality)
ollama pull nomic-embed-text
export CODETECT_EMBEDDING_MODEL=nomic-embed-text
export CODETECT_VECTOR_DIMENSIONS=768

# Or use LiteLLM/OpenAI
export CODETECT_EMBEDDING_PROVIDER=litellm
export CODETECT_LITELLM_API_KEY=sk-...

See Embedding Model Comparison for detailed model selection guidance.

Database Backend

codetect supports two database backends for vector search:

Backend	Best For	Performance	Setup
SQLite (default)	Small-medium projects (< 10K files)	Fast for small datasets	Zero config
PostgreSQL + pgvector	Large projects (> 10K files)	60x faster at scale	Docker or manual install

Quick Start with PostgreSQL:

# Start PostgreSQL with Docker
docker-compose up -d

# Configure codetect
export CODETECT_DB_TYPE=postgres
export CODETECT_DB_DSN="postgres://codetect:codetect@localhost:5432/codetect?sslmode=disable"

# Index and embed as usual
codetect index
codetect embed

Performance Comparison:

Dataset Size	SQLite	PostgreSQL	Speedup
100 vectors	77 μs	603 μs	0.13x (slower)
1,000 vectors	1.19 ms	745 μs	1.6x faster
10,000 vectors	58.1 ms	963 μs	60x faster

For large codebases, PostgreSQL + pgvector provides massive performance improvements through HNSW indexing. See PostgreSQL Setup Guide for detailed installation and migration instructions.

Privacy

For enterprise/cloud scenarios where a centralized index could be shared, enable path hashing to store file paths as opaque SHA-256 hashes in the database:

export CODETECT_HASH_PATHS=true
codetect index --clear-cache --force  # Re-index with hashed paths

A local sidecar file (path_map.json) maintains the reverse mapping so search results still return real paths.

See Installation Guide for all configuration options.

Performance Evaluation

codetect includes an evaluation tool to measure the performance improvement of MCP tools vs. standard CLI tools (grep, find, etc.) when working with Claude Code.

# Run evaluations on any repository
codetect-eval run --repo /path/to/project

# View results
codetect-eval report

Eval cases are stored in ~/.codetect/projects/<name>-<hash>/evals/cases/ (centralized storage) to keep project directories clean.

See Evaluation Guide for detailed documentation on creating test cases, understanding metrics, and best practices.

Documentation

• Installation Guide - Detailed setup and configuration
• Embedding Model Comparison - Choosing the best embedding model for code search
• PostgreSQL Setup Guide - PostgreSQL + pgvector for scalable vector search
• Benchmarks - Vector search performance benchmarks and methodology
• Evaluation Guide - Performance testing and benchmarking
• Architecture - Internal design and data flow
• MCP Compatibility - Supported tools and multi-tool roadmap

Compatibility

codetect uses MCP (Model Context Protocol), an open standard for LLM tool integration.

Tool	Support
Claude Code	Fully supported
Cursor	Should work
Cline / Continue	Should work
Zed	Should work

See MCP Compatibility for details and roadmap for non-MCP tools.

Roadmap

• MCP stdio server
• Keyword search via ripgrep
• Symbol indexing (AST-based via ast-grep)
• Semantic search via Ollama
• Hybrid search with reranking
• Global installation
• Background indexing daemon
• Project registry
• Evaluation framework
• PostgreSQL + pgvector support for scalable vector search
• Token-efficient tool design (detail levels, response budgeting)
• Connection pooling for shared DB/embedding connections
• HTTP API for non-MCP tools
• CLI query mode

License

MIT