Ragex

Hybrid Retrieval-Augmented Generation for Multi-Language Codebases

Ragex is an MCP (Model Context Protocol) server that analyzes codebases using compiler output and language-native tools to build comprehensive knowledge graphs. It enables natural language querying of code structure, relationships, and semantics.

Features

Planned Features

• [✓] Streaming RAG responses
• [✓] MCP streaming notifications
• [✓] MetaAST-enhanced retrieval
• [✓] Code quality analysis
• [✓] Impact analysis and risk assessment
• [✓] CLI improvements (interactive wizards, dashboard, completions, man pages)
• [✗] CI tasks
• [✗] Provider health checks and auto-failover
• [✗] Production optimizations
• [✗] Additional language support
• [✗] Cross-language refactoring via Metastatic
• [✗] Enhanced editor integrations

Architecture

graph TD
    MCP["MCP Server (stdio)<br/>28 Tools + 6 Resources + 6 Prompts"]
    
    MCP --> Tools["Tools Handler"]
    MCP --> Resources["Resources Handler"]
    MCP --> Prompts["Prompts Handler"]
    MCP --> Analyzers["Analyzers<br/>(Elixir, Erlang, Metastatic)"]
    MCP --> Graph["Graph Store<br/>(ETS Knowledge Graph)"]
    MCP --> Vector["Vector Store<br/>(Cosine Similarity)"]
    MCP --> Bumblebee["Bumblebee Embedding<br/>(all-MiniLM-L6-v2)"]
    
    Tools <--> Analyzers
    Analyzers <--> Graph
    Resources --> Graph
    Resources --> Vector
    Resources --> Bumblebee
    Prompts --> Tools
    
    Tools --> Hybrid["Hybrid Retrieval (RRF)<br/>Semantic + Graph + Fusion"]
    Graph --> Hybrid
    Vector --> Hybrid
    
    Tools --> RAG["RAG Pipeline<br/>Cache → Context → Prompts → AI"]
    Hybrid --> RAG
    RAG --> Cache["AI Cache<br/>(TTL + LRU)"]
    RAG --> Usage["Usage Tracker<br/>(Costs + Limits)"]
    RAG --> AIProvider["AI Providers<br/>(OpenAI, Anthropic, DeepSeek, Ollama)"]
    
    style MCP fill:#e1f5ff,color:#01579b,stroke:#01579b,stroke-width:2px
    style Hybrid fill:#f3e5f5,color:#4a148c,stroke:#4a148c,stroke-width:2px
    style Graph fill:#e8f5e9,color:#1b5e20,stroke:#1b5e20,stroke-width:2px
    style Vector fill:#fff3e0,color:#e65100,stroke:#e65100,stroke-width:2px
    style Bumblebee fill:#fce4ec,color:#880e4f,stroke:#880e4f,stroke-width:2px
    style Resources fill:#e0f2f1,color:#004d40,stroke:#004d40,stroke-width:2px
    style Prompts fill:#fff9c4,color:#f57f17,stroke:#f57f17,stroke-width:2px
    style RAG fill:#ffebee,color:#b71c1c,stroke:#b71c1c,stroke-width:2px
    style AIProvider fill:#e8eaf6,color:#1a237e,stroke:#1a237e,stroke-width:2px
    style Cache fill:#e0f7fa,color:#006064,stroke:#006064,stroke-width:2px
    style Usage fill:#fff8e1,color:#f57c00,stroke:#f57c00,stroke-width:2px

Use as MCP Server

The only MCP client currently supported is LunarVim (technically, any NeoVim, but I never tested it.)

To enable Ragex support in LunarVim, copy files from lvim.cfg/lua/user/ to where your LunarVim configs are (typically, it’s ~/.config/lvim/lua/user/) and amend your config.lua as shown below.

-- Ragex integration
local ragex = require("user.ragex")
local ragex_telescope = require("user.ragex_telescope")

-- Setup Ragex with configuration
ragex.setup({
  ragex_path = vim.fn.expand("~/Proyectos/Ammotion/ragex"),
  enabled = true,
  debug = false,
})

-- Ragex keybindings (using "r" prefix for Ragex)
lvim.builtin.which_key.mappings["r"] = {
  name = "Ragex",
  s = { function() ragex_telescope.ragex_search() end, "Semantic Search" },
  w = { function() ragex_telescope.ragex_search_word() end, "Search Word" },
  f = { function() ragex_telescope.ragex_functions() end, "Find Functions" },
  m = { function() ragex_telescope.ragex_modules() end, "Find Modules" },
  a = { function() ragex.analyze_current_file() end, "Analyze File" },
  d = { function() ragex.analyze_directory(vim.fn.getcwd()) end, "Analyze Directory" },
  c = { function() ragex.show_callers() end, "Find Callers" },
  r = {
    function()
      vim.ui.input({ prompt = "New name: " }, function(name)
        if name then
          ragex.rename_function(name)
        end
      end)
    end,
    "Rename Function",
  },
  R = {
    function()
      vim.ui.input({ prompt = "Old module: " }, function(old_name)
        if old_name then
          vim.ui.input({ prompt = "New module: " }, function(new_name)
            if new_name then
              ragex.rename_module(old_name, new_name)
            end
          end)
        end
      end)
    end,
    "Rename Module",
  },
  g = { 
    function()
      local result = ragex.graph_stats()
      if result and result.result then
        -- Unwrap MCP response
        local stats = result.result
        if stats.content and stats.content[1] and stats.content[1].text then
          local ok, parsed = pcall(vim.fn.json_decode, stats.content[1].text)
          if ok then
            stats = parsed
          end
        end
        
        -- Format stats for display
        local lines = {
          "# Graph Statistics",
          "",
          string.format("**Nodes**: %d", stats.node_count or 0),
          string.format("**Edges**: %d", stats.edge_count or 0),
          string.format("**Average Degree**: %.2f", stats.average_degree or 0),
          string.format("**Density**: %.4f", stats.density or 0),
          "",
          "## Node Types",
        }
        
        if stats.node_counts_by_type then
          for node_type, count in pairs(stats.node_counts_by_type) do
            table.insert(lines, string.format("- %s: %d", node_type, count))
          end
        end
        
        if stats.top_by_degree and #stats.top_by_degree > 0 then
          table.insert(lines, "")
          table.insert(lines, "## Top by Degree")
          for i, node in ipairs(stats.top_by_degree) do
            if i > 10 then break end
            table.insert(lines, string.format("- %s (in:%d, out:%d, total:%d)",
              node.node_id or "unknown",
              node.in_degree or 0,
              node.out_degree or 0,
              node.total_degree or 0))
          end
        end
        
        ragex.show_in_float("Ragex Graph Statistics", lines)
      else
        vim.notify("No graph statistics available", vim.log.levels.WARN)
      end
    end,
    "Graph Stats"
  },
  W = { function() ragex.watch_directory(vim.fn.getcwd()) end, "Watch Directory" },
  t = { function() ragex.toggle_auto_analyze() end, "Toggle Auto-Analysis" },
  -- Advanced Graph Algorithms
  b = { function() ragex.show_betweenness_centrality() end, "Betweenness Centrality" },
  o = { function() ragex.show_closeness_centrality() end, "Closeness Centrality" },
  n = { function() ragex.show_communities("louvain") end, "Detect Communities (Louvain)" },
  l = { function() ragex.show_communities("label_propagation") end, "Detect Communities (Label Prop)" },
  e = { 
    function()
      vim.ui.select({ "graphviz", "d3" }, {
        prompt = "Export format:",
      }, function(format)
        if format then
          local ext = format == "graphviz" and "dot" or "json"
          vim.ui.input({
            prompt = "Save as: ",
            default = vim.fn.getcwd() .. "/graph." .. ext,
          }, function(filepath)
            if filepath then
              ragex.export_graph_to_file(format, filepath)
            end
          end)
        end
      end)
    end,
    "Export Graph"
  },
}

-- Register Telescope commands for Ragex
vim.api.nvim_create_user_command("RagexSearch", ragex_telescope.ragex_search, {})
vim.api.nvim_create_user_command("RagexFunctions", ragex_telescope.ragex_functions, {})
vim.api.nvim_create_user_command("RagexModules", ragex_telescope.ragex_modules, {})
vim.api.nvim_create_user_command("RagexSearchWord", ragex_telescope.ragex_search_word, {})
vim.api.nvim_create_user_command("RagexToggleAuto", function() ragex.toggle_auto_analyze() end, {})

-- Add Ragex status to lualine
local function ragex_status()
  if ragex.config.enabled then
    return "  Ragex"
  end
  return ""
end

This should result in the following <leader>r update:

Installation

Prerequisites

• Elixir 1.19 or later
• Erlang/OTP 28 or later
• Python 3.x (optional, for Python code analysis)
• Node.JS (optional, for Javascript code analysis)
• ~500MB RAM for embedding model (first run downloads ~90MB)

Build

cd ragex
mix deps.get
mix compile

Note: First compilation will take longer due to ML dependencies. The embedding model (~90MB) will download on first run and be cached at ~/.cache/huggingface/.

Demo

A comprehensive demo showcasing all Ragex features is available in examples/product_cart/.

The demo uses an intentionally mediocre e-commerce cart application to demonstrate:

• Security vulnerability scanning (8+ issues detected)
• Code complexity analysis (cyclomatic, cognitive, Halstead metrics)
• Code smell detection (long functions, deep nesting, magic numbers)
• Code duplication detection (Type I-IV clones)
• Dead code analysis (4 unused functions)
• Dependency and coupling analysis
• Impact analysis and refactoring suggestions
• AI-enhanced features (ValidationAI, AIPreview, AIRefiner, AIAnalyzer, AIInsights)

Quick Start:

cd examples/product_cart
./run_demo.sh

The demo generates 11 detailed reports showing:

• 8 security vulnerabilities (2 critical, 3 high)
• 18 code smells across 5 types
• 52 lines of duplicated code (10% of codebase)
• 28 lines of dead code (7% of codebase)
• 8 prioritized refactoring suggestions
• Expected improvement: 65% better maintainability

See Produce Cart’s README for full details and Product Cart’s DEMO for step-by-step walkthrough.

Usage

As an MCP Server

Run the server (it will listen on both stdin and socket):

./start_mcp.sh

Note: The stdio server is validated and production-ready.

Auto-Analyze Directories on Startup

You can configure Ragex to automatically analyze specific directories when it starts. Add to config/config.exs:

config :ragex, :auto_analyze_dirs, [
  "/opt/Proyectos/MyProject",
  "~/workspace/important-lib"
]

This pre-loads your frequently used codebases into the knowledge graph, making them immediately available for querying. See CONFIGURATION for details.

MCP Protocol Example

Initialize the server:

{"jsonrpc":"2.0","method":"initialize","params":{"clientInfo":{"name":"test-client","version":"1.0"}},"id":1}

List available tools:

{"jsonrpc":"2.0","method":"tools/list","id":2}

Analyze a file (with auto-detection):

{
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "name": "analyze_file",
    "arguments": {
      "path": "lib/ragex.ex"
    }
  },
  "id": 3
}

Or specify the language explicitly:

{
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "name": "analyze_file",
    "arguments": {
      "path": "script.py",
      "language": "python"
    }
  },
  "id": 3
}

Query the graph:

{
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "name": "query_graph",
    "arguments": {
      "query_type": "find_module",
      "params": {"name": "Ragex"}
    }
  },
  "id": 4
}

Development

Running Tests

mix test

Interactive Development

RAGEX_NO_SERVER=1 iex -S mix

# Analyze a file
{:ok, content} = File.read("lib/ragex.ex")
{:ok, analysis} = Ragex.Analyzers.Elixir.analyze(content, "lib/ragex.ex")

# Check graph stats, it’s expected to be empty for this single file
Ragex.stats()

MCP Tools Reference

Core Analysis Tools

analyze_file

Analyzes a source file and extracts code structure into the knowledge graph.

Parameters:

• path (string, required): Path to the file
• language (string, optional): Programming language - elixir, erlang, python, javascript, typescript, or auto (default: auto-detect from extension)
• generate_embeddings (boolean, optional): Generate embeddings for semantic search (default: true)

analyze_directory

Batch analyzes all source files in a directory.

Parameters:

• path (string, required): Directory path
• language (string, optional): Language to filter files (default: auto-detect)
• recursive (boolean, optional): Recursively analyze subdirectories (default: true)
• generate_embeddings (boolean, optional): Generate embeddings (default: true)

query_graph

Queries the knowledge graph for code entities and relationships (symbolic search).

Parameters:

• query_type (string, required): Type of query
  • find_module: Find a module by name
  • find_function: Find a function by module and name
  • get_calls: Get function call relationships
  • get_dependencies: Get module dependencies
• params (object, required): Query-specific parameters

list_nodes

Lists all nodes in the knowledge graph with optional filtering.

Parameters:

• node_type (string, optional): Filter by type (module, function, etc.)
• limit (integer, optional): Maximum results (default: 100)

File Watching Tools

watch_directory

Automatically re-index files when they change.

Parameters:

• path (string, required): Directory to watch

unwatch_directory

Stop watching a directory.

Parameters:

• path (string, required): Directory to stop watching

list_watched

List all watched directories.

Parameters: None

Semantic Search Tools

semantic_search

Performs natural language code search using vector embeddings.

Parameters:

• query (string, required): Natural language query (e.g., "function to parse JSON")
• limit (integer, optional): Maximum results (default: 10)
• threshold (number, optional): Minimum similarity score 0.0-1.0 (default: 0.7)
• node_type (string, optional): Filter by type (module, function)
• include_context (boolean, optional): Include caller/callee context (default: false)

Example:

{
  "query": "HTTP request handler",
  "limit": 5,
  "threshold": 0.75,
  "node_type": "function"
}

hybrid_search

Combines symbolic graph queries with semantic search for best results.

Parameters:

• query (string, required): Search query
• strategy (string, optional): Search strategy:
  • fusion (default): RRF fusion of both approaches
  • semantic_first: Semantic search then graph filtering
  • graph_first: Graph query then semantic ranking
• limit (integer, optional): Maximum results (default: 10)
• threshold (number, optional): Minimum similarity (default: 0.7)
• graph_filter (object, optional): Optional symbolic constraints
• include_context (boolean, optional): Include context (default: false)

Example:

{
  "query": "database connection",
  "strategy": "fusion",
  "limit": 10,
  "graph_filter": {"module": "DB"}
}

get_embeddings_stats

Returns ML model and vector store statistics.

Parameters: None

Returns:

• Model information (name, dimensions, status)
• Vector store metrics (total embeddings, by type)
• Graph statistics (nodes, edges)

Code Editing Tools

edit_file

Safely edit a single file with automatic backup, validation, and atomic operations.

Parameters:

• path (string, required): Path to the file to edit
• changes (array, required): List of changes to apply
  • type (string): replace, insert, or delete
  • line_start (integer): Starting line number (1-indexed)
  • line_end (integer): Ending line number (for replace/delete)
  • content (string): New content (for replace/insert)
• validate (boolean, optional): Validate syntax before applying (default: true)
• create_backup (boolean, optional): Create backup before editing (default: true)
• format (boolean, optional): Format code after editing (default: false)
• language (string, optional): Explicit language for validation (auto-detected from extension)

Example:

{
  "path": "lib/my_module.ex",
  "changes": [
    {
      "type": "replace",
      "line_start": 10,
      "line_end": 15,
      "content": "def new_function do\n  :ok\nend"
    }
  ],
  "validate": true,
  "format": true
}

edit_files

Atomically edit multiple files with coordinated rollback on failure.

Parameters:

• files (array, required): List of files to edit
  • path (string): Path to the file
  • changes (array): List of changes (same format as edit_file)
  • validate (boolean, optional): Override transaction-level validation
  • format (boolean, optional): Override transaction-level formatting
  • language (string, optional): Explicit language for this file
• validate (boolean, optional): Validate all files before applying (default: true)
• create_backup (boolean, optional): Create backups for all files (default: true)
• format (boolean, optional): Format all files after editing (default: false)

Example:

{
  "files": [
    {
      "path": "lib/module_a.ex",
      "changes": [{"type": "replace", "line_start": 5, "line_end": 5, "content": "@version \"2.0.0\""}]
    },
    {
      "path": "lib/module_b.ex",
      "changes": [{"type": "replace", "line_start": 10, "line_end": 12, "content": "# Updated"}]
    }
  ],
  "validate": true,
  "format": true
}

validate_edit

Preview validation of changes without applying them.

Parameters:

• path (string, required): Path to the file
• changes (array, required): List of changes to validate
• language (string, optional): Explicit language for validation

rollback_edit

Undo a recent edit by restoring from backup.

Parameters:

• path (string, required): Path to the file to rollback
• backup_id (string, optional): Specific backup to restore (default: most recent)

edit_history

Query backup history for a file.

Parameters:

• path (string, required): Path to the file
• limit (integer, optional): Maximum number of backups to return (default: 10)

refactor_code

Semantic refactoring operations using AST analysis and knowledge graph.

Parameters:

• operation (string, required): Type of refactoring - rename_function or rename_module
• params (object, required): Operation-specific parameters
  • For rename_function:
    • module (string): Module containing the function
    • old_name (string): Current function name
    • new_name (string): New function name
    • arity (integer): Function arity
  • For rename_module:
    • old_name (string): Current module name
    • new_name (string): New module name
• scope (string, optional): module (same file only) or project (all files, default: project)
• validate (boolean, optional): Validate before/after (default: true)
• format (boolean, optional): Format code after (default: true)

Example - Rename Function:

{
  "operation": "rename_function",
  "params": {
    "module": "MyModule",
    "old_name": "old_function",
    "new_name": "new_function",
    "arity": 2
  },
  "scope": "project",
  "validate": true,
  "format": true
}

Example - Rename Module:

{
  "operation": "rename_module",
  "params": {
    "old_name": "OldModule",
    "new_name": "NewModule"
  },
  "validate": true
}

RAG (AI-Powered) Tools

rag_query

Query the codebase using Retrieval-Augmented Generation with AI assistance.

Parameters:

• query (string, required): Natural language query about the codebase
• limit (integer, optional): Maximum number of code snippets to retrieve (default: 10)
• include_code (boolean, optional): Include full code snippets in context (default: true)
• provider (string, optional): AI provider override (deepseek_r1)

Example:

{
  "query": "How does authentication work in this codebase?",
  "limit": 15,
  "include_code": true
}

Returns:

• AI-generated response based on retrieved code context
• Sources count and model information

rag_explain

Explain code using RAG with AI assistance and aspect-focused analysis.

Parameters:

• target (string, required): File path or function identifier (e.g., MyModule.function/2)
• aspect (string, optional): What to explain - purpose, complexity, dependencies, or all (default: all)

Example:

{
  "target": "Ragex.Graph.Store.add_node/3",
  "aspect": "complexity"
}

Returns:

• AI-generated explanation based on code analysis
• Related code context and dependencies

rag_suggest

Suggest code improvements using RAG with AI analysis.

Parameters:

• target (string, required): File path or function identifier
• focus (string, optional): Improvement focus - performance, readability, testing, security, or all (default: all)

Example:

{
  "target": "lib/ragex/editor/core.ex",
  "focus": "performance"
}

Returns:

• AI-generated improvement suggestions
• Code context and rationale

Configuration:

RAG tools require the DEEPSEEK_API_KEY environment variable:

export DEEPSEEK_API_KEY="sk-xxxxxxxxxxxxx"

Documentation

• Algorithms - Algorithms used
• Usage - Tips on how to use Ragex
• Configuration - Embedding model configuration and migration
• Persistence - Embedding cache management and performance
• Analysis - Code analysis features and tools
• Troubleshooting - Common issues and error messages

Cache Management

Ragex automatically caches embeddings for faster startup:

# View cache statistics
mix ragex.cache.stats

# Clear current project cache
mix ragex.cache.clear --current

# Clear all caches
mix ragex.cache.clear --all --force

TODO: Streaming Enhancements

The following streaming improvements are planned but not yet implemented:

• Tool-call delta parsing in providers: Currently, streaming parsers for all four providers (DeepSeek, OpenAI, Anthropic, Ollama) silently skip tool_calls deltas in the SSE stream. Adding index-based function.arguments accumulation would allow real-time thinking tokens even during intermediate tool-call steps in the agent loop. This requires per-provider work (OpenAI/DeepSeek: delta.tool_calls[i]; Anthropic: content_block_start + input_json_delta; Ollama: not supported by API).

• Full MCP streaming protocol: Emit individual JSON-RPC streaming responses (not just notifications) per chunk, allowing MCP clients to render responses incrementally. Includes cancellation support via MCP protocol.

• Stream caching and replay: Cache reconstructed responses from consumed streams so that repeated queries can be served from cache without re-calling the AI provider.

Supported Languages

Language	Extensions	Parser	Status
Elixir	.ex, .exs	Native AST (Code.string_to_quoted)	✓ Full
Erlang	.erl, .hrl	Native AST (:erl_scan, :erl_parse)	✓ Full
Python	.py	Python ast module (subprocess)	✓ Full
JavaScript/TypeScript	.js, .jsx, .ts, .tsx, .mjs	Regex-based	✗ Basic