LibScope

LibScope is a local knowledge base that makes your documentation searchable by AI assistants. Point it at markdown files, URLs, or connect it to Obsidian/Notion/Confluence/Slack, and it chunks, embeds, and indexes everything into a local SQLite database. Your AI tools query it through MCP (Model Context Protocol) or a REST API.

Everything runs on your machine. No cloud services required for basic usage — just npm install and go.

Getting Started

npm install -g libscope

# Set up the database
libscope init

# Index some docs
libscope add ./docs/getting-started.md --library my-lib
libscope add https://docs.example.com/guide
libscope import ./docs/ --library my-lib --extensions .md,.mdx

# Search
libscope search "how to authenticate"

# Start the MCP server so your AI assistant can query it
libscope serve

On first run with the default embedding provider, LibScope downloads the all-MiniLM-L6-v2 model (~80 MB). Subsequent runs use the cached model.

Document Format Support

LibScope supports Markdown (.md, .mdx) and plain text natively. Additional formats are available via optional dependencies:

The pdf-parse, mammoth, epub2, and pizzip packages are listed as optionalDependencies and install automatically when the Node.js version is compatible. Note: binary .ppt files are not supported — only .pptx.

Using with AI Assistants

LibScope exposes an MCP server over stdio. Point your MCP-compatible client at it:

Cursor — add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "libscope": {
      "command": "npx",
      "args": ["-y", "libscope", "serve"]
    }
  }
}

Claude Code:

claude mcp add --scope user libscope -- npx -y libscope serve

Once connected, your assistant can search docs, submit new documents, rate content quality, and ask RAG-powered questions against your knowledge base.

Connectors

LibScope can pull documentation from several platforms. Each connector handles incremental syncing so re-runs only process what changed.

# Obsidian — parses wikilinks, frontmatter, embeds, tags
libscope connect obsidian /path/to/vault
libscope connect obsidian /path/to/vault --sync   # incremental re-sync

# Notion
libscope connect notion --token secret_abc123

# Confluence
libscope connect confluence \
  --url https://acme.atlassian.net \
  --email user@acme.com \
  --token $CONFLUENCE_TOKEN

# Slack — index channel messages and threads
libscope connect slack --token xoxb-... --channels general,engineering

# OneNote — device code auth via Microsoft Graph
export ONENOTE_CLIENT_ID=your-client-id
libscope connect onenote

# GitHub / GitLab repos
libscope add-repo https://github.com/org/repo --branch main --path docs/

# Remove a connector's data
libscope disconnect obsidian /path/to/vault

Search and RAG

# Semantic search
libscope search "authentication best practices"
libscope search "API rate limiting" --library my-lib --topic security --limit 10

# Ask questions (needs an LLM provider configured — see Configuration below)
libscope ask "How do I configure OAuth2?" --library my-lib

# Interactive REPL for iterative searching
libscope repl

Search uses sqlite-vec for vector similarity when available, with FTS5 full-text search as a fallback. Results automatically get a 1.5× title boost when the document title matches query words. You can also pass --diversity 0.5 (0–1) for MMR-based diversity reranking.

Programmatic SDK

LibScope exports two embeddable APIs:

LibScope — full SDK with all features (connectors, topics, packs, etc.):

import { LibScope } from "libscope";

const scope = LibScope.create();
await scope.index({ title: "My Doc", content: "..." });
const results = await scope.search("query");
scope.close();

LibScopeLite — lightweight embeddable class for external applications. No CLI, no MCP server, no connectors. Designed for embedding semantic search directly into other tools (MCP servers, VS Code extensions, CI scripts):

import { LibScopeLite } from "libscope/lite";

const lite = new LibScopeLite({ dbPath: ":memory:" });

// Index documents (or code files via tree-sitter chunking)
await lite.indexBatch(docs, { concurrency: 4 });

// Hybrid vector + FTS5 search
const results = await lite.search("how to authenticate");

// Get RAG context for injection into an external LLM prompt
const context = await lite.getContext("How does auth work?");

lite.close();

Tree-sitter powered code indexing splits TypeScript, JavaScript, and Python files at function/class boundaries:

import { TreeSitterChunker } from "libscope/lite";

const chunker = new TreeSitterChunker();
const chunks = await chunker.chunk(sourceCode, "typescript");
// Each chunk is a complete function or class with 1-based line numbers

See the LibScope Lite guide and API reference for the full documentation.

Organizing Content

Topics give your docs a hierarchy. Tags give them flexible labels. Workspaces give you isolated databases.

# Topics
libscope topics create "backend"
libscope topics create "auth" --parent backend --description "Auth & identity"

# Tags
libscope tag add <doc-id> typescript,api,v2

# Workspaces — separate knowledge bases entirely
libscope workspace create my-project
libscope workspace use my-project
libscope --workspace my-project search "deploy steps"

Documents also keep version history, so you can roll back if a re-index goes wrong:

libscope docs history <doc-id>
libscope docs rollback <doc-id> 3

REST API

For programmatic access outside of MCP:

libscope serve --api --port 3378

OpenAPI 3.0 spec at GET /openapi.json. Key endpoints:

Configuration

LibScope reads config from (highest priority first): environment variables → .libscope.json in your project → ~/.libscope/config.json → built-in defaults.

Embedding Providers

libscope config set embedding.provider ollama
# or
export LIBSCOPE_EMBEDDING_PROVIDER=openai
export LIBSCOPE_OPENAI_API_KEY=sk-...

LLM for RAG

The ask command and ask-question MCP tool need an LLM. Configure one with:

export LIBSCOPE_LLM_PROVIDER=openai   # or ollama, anthropic
export LIBSCOPE_LLM_MODEL=gpt-4o-mini # optional

{
  "embedding": {
    "provider": "local",
    "ollamaUrl": "http://localhost:11434",
    "ollamaModel": "nomic-embed-text",
    "openaiModel": "text-embedding-3-small"
  },
  "llm": {
    "provider": "openai",
    "model": "gpt-4o-mini",
    "anthropicApiKey": "sk-ant-..."
  },
  "database": {
    "path": "~/.libscope/libscope.db"
  },
  "logging": {
    "level": "info"
  },
  "indexing": {
    "allowPrivateUrls": false,
    "allowSelfSignedCerts": false
  }
}

Corporate / Internal Networks

If you're indexing docs from internal servers (Confluence, wikis, etc.), you may need:

# Allow fetching from private/internal IP addresses
libscope config set indexing.allowPrivateUrls true

# Accept self-signed or corporate TLS certificates
libscope config set indexing.allowSelfSignedCerts true

Or via environment variables:

export LIBSCOPE_ALLOW_PRIVATE_URLS=true
export LIBSCOPE_ALLOW_SELF_SIGNED_CERTS=true

Webhooks

LibScope can push events to any HTTP endpoint. Useful for triggering CI pipelines, Slack notifications, or custom workflows whenever documents are created or updated.

libscope serve --api  # webhooks require the REST API

# Create a webhook
curl -X POST http://localhost:3378/api/v1/webhooks \
  -H "Content-Type: application/json" \
  -d '{"url": "https://hooks.example.com/libscope", "events": ["document.created", "document.updated"], "secret": "my-hmac-secret"}'

# Send a test ping
curl -X POST http://localhost:3378/api/v1/webhooks/<id>/test

Webhook payloads are signed with HMAC-SHA256 when a secret is set. The signature is in the X-LibScope-Signature header.

Supported events: document.created, document.updated, document.deleted.

Other Tools

LibScope ships with a few more utilities beyond the core index-and-search loop:

# Watch a directory and auto-reindex on changes
libscope watch ./docs/

# Re-embed everything after switching embedding providers
libscope reindex

# Find duplicate documents
libscope dedupe

# Export / import the whole knowledge base
libscope export ./backup.json
libscope import-backup ./backup.json

# Usage analytics
libscope stats                  # overview
libscope stats popular          # most-searched docs
libscope stats stale --days 90  # docs nobody searches for

# Knowledge packs — portable document bundles
libscope pack create --name "react-docs" --topic react
libscope pack install ./react-docs.json

Pack Registries

Share and discover knowledge packs through git-based registries. A registry is a git repo with a defined folder structure managed by libscope.

# Add a registry
libscope registry add https://github.com/org/libscope-registry.git --name official

# Search for packs across all registries
libscope registry search "react"

# Install a pack by name (resolves from registries)
libscope pack install react-docs
libscope pack install react-docs@1.2.0    # specific version

# Create your own registry
libscope registry create ./my-registry

# Publish a pack file to your registry
libscope registry publish ./my-pack.json -r my-registry --version 1.0.0

# Submit a pack to someone else's registry (creates a feature branch)
libscope registry publish ./my-pack.json -r community --submit

Authentication is delegated to git — SSH keys and HTTPS credential helpers work automatically. Registries cache locally and support offline index lookups. See the Pack Registries guide for full details.

There's also a web dashboard at http://localhost:3377 when you run libscope serve, with search, document browsing, topic navigation, and a knowledge graph visualization at /graph.

How It Works

LibScope stores everything in a local SQLite database (at ~/.libscope/libscope.db by default):

• Documents are split into chunks by heading boundaries
• Each chunk is embedded into a vector using the configured provider
• Vector search is done via sqlite-vec; FTS5 full-text search is used as a fallback
• The MCP server reads from this same database over stdio
• Connectors fetch content from external platforms and feed it through the same indexing pipeline

The stack: better-sqlite3 + sqlite-vec + @modelcontextprotocol/sdk + @xenova/transformers for local embeddings.

Contributing

See CONTRIBUTING.md. The short version:

git clone https://github.com/RobertLD/libscope.git
cd libscope && npm install
npm run dev        # watch mode
npm test           # run tests
npm run typecheck  # type check
npm run lint       # lint

License

Business Source License 1.1 — see LICENSE for full terms.