️ Arachne (n2-arachne)

한국어 | English | 日本語

ARACHNE V4.0.0 TITANIUM EDITION

1GB (1,000,000,000 Bytes) Codebase Search in 0.54 Seconds. Powered by Zero-Marshaling Architecture. 0 Bytes JS Heap Bloat. No OOM Crashes.

Weave your codebase into the perfect context for AI

What's New in v4.0 (Titanium Edition)

Arachne has been completely rewritten from JS to strict TypeScript, with heavy computational paths natively rewritten in C++ SIMD (sqlite-vec) and Rust (napi-rs).

• TypeScript (Strict): 100% type safety with zero runtime regressions.
• Rust Acceleration: BM25 1.3x faster (memchr SIMD + rayon parallel); BatchCosine 19.9x faster (96ms → 4.8ms).
• C++ SIMD Search: sqlite-vec scans 10,000 × 768D vectors in just 25ms natively.
• KV-Cache Integration: Soul-bridge incremental re-indexing enables sub-second history load on start. — like Arachne, the greatest weaver of Greek mythology. ️

The Problem — Why AI Gets Your Code Wrong

Imagine going to a doctor and saying "I have a headache."

• Bad doctor: reads your entire 500-page medical history, gets confused, prescribes the wrong medicine
• Good doctor: looks at relevant records only — recent symptoms, medications, allergies — and nails the diagnosis

AI coding assistants are like that bad doctor.

When your project has 500 files, AI can't read them all. So what happens?

 Your Project (500 files, 2M tokens)
│
├── auth/login.ts        ←  The bug is HERE
├── auth/session.ts      ←  login imports this
├── api/http.ts          ←  session imports this
├── utils/config.ts      ← ️ timeout settings live here
│
├── pages/home.tsx       ←  completely irrelevant
├── pages/about.tsx      ←  completely irrelevant
├── components/Button.tsx ←  completely irrelevant
└── ... 493 more files    ←  all irrelevant

Tokens = units of text AI reads. More tokens = more cost, slower, less accurate. AI has a limited "context window" — like a desk that can only hold so many papers.

Real-World Benchmark (N2 Browser — 3,219 files)

Measured on a real production project. Arachne delivered exactly what AI needed — 333x less data, same accuracy.

️ The Solution — Arachne Picks Exactly What AI Needs

Arachne is a local MCP server that acts like that good doctor. It reads your entire codebase once, understands the structure, and only sends what's relevant to AI.

You: "Fix the login timeout bug"
                │
                ▼
┌──────────────────────────────────────────────────────┐
│  ️ Arachne: "I'll find exactly what you need"      │
│                                                      │
│  L1  Project tree (so AI knows the structure)      │
│  L2  login.ts (the file you're working on)         │
│  L3  http.ts, session.ts (found via search +       │
│        dependency chain: login → session → http)     │
│  L4 ️ config.ts (frequently accessed, has timeout)  │
│                                                      │
│  → 30,000 tokens of perfectly curated context        │
└──────────────────────────────────────────────────────┘
                │
                ▼
        AI generates accurate fix 

No manual file selection. No prompt engineering. Just ask.

Why Arachne?

• 98.5% token savings — 30K instead of 2M tokens. Real money saved on API calls
• Beats "Lost in the Middle" — Smart output ordering (L1→L3→L4→L2) keeps critical code where AI pays attention (research-backed)
• Zero external deps — No Docker, no cloud, no API keys. Just npm install and go
• Blazing fast — 21 files indexed in 12ms. Incremental updates in sub-second
• Ultralight — Only 3 deps: better-sqlite3, sqlite-vec, zod. No bloat
• Free for personal/OSS — Apache-2.0 for individuals and open-source projects. No telemetry
• Plug & play — Add MCP config → done. Zero code changes to your project
• Multi-language — Follows import chains across JS/TS, Python, Rust, Go, Java
• Ollama optional — Works perfectly without Ollama (BM25 search). Add Ollama for bonus semantic search

Arachne in 4 Panels — Titanium Edition

Soul + Arachne Synergy

Key Features

️ Architecture: 4-Layer Context Assembly

┌─────────────────────────────────────────────┐
│              Token Budget (e.g. 30K)        │
├────────────┬────────────────────────────────┤
│ L1: Fixed  │ File tree overview (10%)       │
│ (always)   │ Project structure snapshot     │
├────────────┼────────────────────────────────┤
│ L2: Short  │ Current file + recent (20%)   │
│ (context)  │ What you're working on now     │
├────────────┼────────────────────────────────┤
│ L3: Assoc  │ Search + dependencies (50%)  │
│ (relevant) │ BM25 + semantic + dep chain   │
├────────────┼────────────────────────────────┤
│ L4: Spare  │ Frequently accessed (20%)     │
│ (backup)   │ Files you use most            │
└────────────┴────────────────────────────────┘

Output order: L1 → L3 → L4 → L2  (mitigates "Lost in the Middle")

Semantic Search (Optional, Zero Lock-in)

When Ollama is available, Arachne upgrades from keyword-only to hybrid search:

BM25 Score (keyword) ──┐
                       ├── Weighted Merge (α=0.5) → Best Results
Cosine Similarity ─────┘
(nomic-embed-text 768D)

• sqlite-vec for SIMD-accelerated (AVX2/SSE2/Neon) KNN vector search
• 768-dimensional embeddings via Ollama nomic-embed-text — runs 100% local
• Graceful degradation: No Ollama? Falls back to BM25-only. Zero crashes. Always works.
• Enable in config: embedding.enabled = true
• Vector storage: ~3KB per chunk. 5000 chunks = just 15MB on disk

Benchmark (v4.0)

All benchmarks run on AMD Ryzen 5 5600G, Node v24, Windows x64. Three engines: TypeScript (V8 JIT), Rust (napi-rs), sqlite-vec (C++ SIMD).

Search Performance (10,000 chunks / 768D vectors)

Architecture Pivot: 1GB+ Scale Stability

While Rust BatchCosine achieved blazing speeds (4.9ms), loading millions of vectors across the Node.js FFI boundary triggered massive V8 Garbage Collection (GC) pauses and Heap OOM crashes on 1GB+ codebases.

To achieve 100% stability on large-scale datasets, Arachne v4.0 enforces a Zero-Marshaling Policy:

• Semantic Search: Delegated 100% to sqlite-vec. It takes ~29ms (slightly slower than Rust), but memory overhead is 0 bytes in JS. The Node event loop remains completely unblocked.
• Keyword Search: Powered by the Rust BM25 Cache. Chunk data is cached heavily in the Rust heap, crossing the FFI boundary only for small queries and resulting IDs.

npm run build && node test/bench-hybrid-engine.js   # Raw engine comparison
npm run build && node test/bench-10mb.js             # Memory scale impact

Java Support — Built for Enterprise

Arachne provides first-class Java support, designed for large-scale enterprise codebases (5M+ LOC):

How Large Class Sub-Chunking Works

// 500+ token class → automatically split into methods
public class UserService {       // ← detected as container
    public User findById() {}    // ← sub-chunk 1
    public List<User> findAll()  // ← sub-chunk 2
    public User save() {}        // ← sub-chunk 3
    // ... fields, constructor   // ← remainder chunk
}

// Small class (<500 tokens) → kept as single chunk (no overhead)
public class TinyDTO { ... }     // ← single chunk, efficient

Why this matters for 5M LOC projects: A single Java class can have 50+ methods spanning thousands of lines. Without sub-chunking, AI would receive the entire class as one blob. With Arachne, AI gets individual methods — enabling precise, targeted code generation.

Token Impact: Less Is More

Without sub-chunking:
  AI asks: "Fix the findById bug"
  → BM25 hits UserService class
  → Entire class sent: 6,000 tokens  

With sub-chunking:
  AI asks: "Fix the findById bug"
  → BM25 hits findById() method only
  → Just the method sent: 80 tokens    75x savings!

Sub-chunking doesn't cost extra — it saves tokens by sending only what's relevant instead of entire classes.

️ Stability: 128 Tests, Zero Failures

Arachne is built for production. Every edge case is tested:

Phase 1 (Indexing/Search):    15/15 
Phase 2 (Assembly/Deps):      26/26 
Phase 2 (KV-Cache Bridge):   33/33 
Phase 3 (Semantic/Hybrid):    10/10 
Stability (Reddit-proof):     44/44 
─────────────────────────────────────
Total:                       128/128 

Installation

Pro tip: The best way to install? Just ask your AI agent: "Install n2-arachne for me." It knows what to do. ️

npm install n2-arachne

MCP Config (Claude Desktop / Cursor / etc.)

{
  "mcpServers": {
    "n2-arachne": {
      "command": "node",
      "args": ["/path/to/n2-arachne/dist/index.js"],
      "env": {
        "ARACHNE_PROJECT_DIR": "/path/to/your/project"
      }
    }
  }
}

Configuration

Create config.local.js in the Arachne directory:

module.exports = {
    projectDir: '/path/to/your/project',
    dataDir: './data',

    indexing: {
        autoIndex: true,
        maxFileSize: 512 * 1024,    // 512KB max per file
    },

    // Enable semantic search (requires Ollama)
    embedding: {
        enabled: true,              // default: false
        provider: 'ollama',
        model: 'nomic-embed-text',
        endpoint: 'http://localhost:11434',
    },

    assembly: {
        defaultBudget: 30000,       // tokens
    },
};

Usage (MCP Tool)

Arachne registers a single MCP tool n2_arachne with these actions:

Example: Assemble Context

{
  "action": "assemble",
  "query": "HTTP request timeout error handling",
  "activeFile": "lib/executor.js",
  "budget": 20000
}

Connect with Soul / QLN

Arachne works great standalone, but becomes far more powerful with Soul (session memory) and QLN (tool routing).

Setup is simple — just register them together in your MCP config:

Soul + Arachne Together

{
  "mcpServers": {
    "n2-soul": {
      "command": "node",
      "args": ["/path/to/n2-soul/index.js"]
    },
    "n2-arachne": {
      "command": "node",
      "args": ["/path/to/n2-arachne/dist/index.js"],
      "env": {
        "ARACHNE_PROJECT_DIR": "/path/to/your/project"
      }
    }
  }
}

Zero extra config needed! Register both servers in the same MCP config and AI automatically uses both tools.

• Soul remembers past session work and decisions
• Arachne finds the exact code and delivers it to AI
• Result: AI picks up right where you left off — no "what was I working on?"

Full N2 Stack (Soul + Arachne + QLN)

{
  "mcpServers": {
    "n2-soul": {
      "command": "node",
      "args": ["/path/to/n2-soul/index.js"]
    },
    "n2-arachne": {
      "command": "node",
      "args": ["/path/to/n2-arachne/dist/index.js"],
      "env": {
        "ARACHNE_PROJECT_DIR": "/path/to/your/project"
      }
    },
    "n2-qln": {
      "command": "node",
      "args": ["/path/to/n2-qln/index.js"]
    }
  }
}

Add QLN and even with 100+ MCP tools, AI automatically finds and uses only what it needs via QLN's semantic routing.

N2 Ecosystem — Better Together

Every package works 100% standalone. But when combined, magic happens:

Synergy: How They Work Together

User: "Fix the login timeout bug"
     │
     ▼
┌─── QLN (Router) ──────────────────────────────────────┐
│ 1000+ tools → Semantic routing finds:                 │
│   → n2_arachne.assemble (context)                     │
│   → n2_arachne.search (code search)                   │
│ Token cost: 2 tool defs instead of 1000+              │
└────────────────┬──────────────────────────────────────┘
                 │
                 ▼
┌─── Arachne (Context) ─────────────────────────────────┐
│ L1: Project tree overview                              │
│ L2: auth/login.ts (current file)                       │
│ L3: BM25 + semantic search → timeout-related code      │
│     + dependency chain: login.ts → api.ts → http.ts    │
│ L4: Frequently accessed config files                   │
│ → 30K tokens of perfectly curated context              │
└────────────────┬──────────────────────────────────────┘
                 │
                 ▼
┌─── Soul (Memory) ─────────────────────────────────────┐
│ "Last session, Rose fixed a similar timeout in         │
│  api.ts line 47. Decision: increased to 30s."          │
│ → Past context + decisions + handoff notes             │
│ → KV-Cache: instant session restoration                │
└────────────────┬──────────────────────────────────────┘
                 │
                 ▼
┌─── Ark (Security) ────────────────────────────────────┐
│  No hardcoded credentials in generated code          │
│  Timeout value from config, not magic number         │
│  Error handling follows project conventions           │
│ → Code verification before commit                      │
└───────────────────────────────────────────────────────┘

Solo vs Combined

Real-World Impact

• QLN + Arachne: QLN routes the request to Arachne → Arachne provides perfect context → AI generates accurate code on the first try. No more "which file was that in?"
• Soul + Arachne: Soul remembers what you worked on last session → Arachne indexes those files with higher priority → continuity across sessions
• Ark + Arachne: Arachne provides code context → AI generates code → Ark validates it follows project patterns. Catch bugs before they ship.
• All 4 together: The AI becomes a team member who remembers everything, finds anything, uses the right tools, and follows the rules.

License

This project is dual-licensed:

See LICENSE for full details.

Star History

No coffee? A star is fine too

Arachne — the greatest weaver. Your code, perfectly woven. ️