LibScope

LibScope is a local knowledge base that makes your documentation searchable by AI assistants. Point it at markdown files, URLs, or connect it to Obsidian/Notion/Confluence/Slack, and it chunks, embeds, and indexes everything into a local SQLite database. Your AI tools query it through MCP (Model Context Protocol) or a REST API.

Everything runs on your machine. No cloud services required for basic usage — just npm install and go.

Getting Started

npm install -g libscope

# Set up the database
libscope init

# Index some docs
libscope add ./docs/getting-started.md --library my-lib
libscope add https://docs.example.com/guide
libscope import ./docs/ --library my-lib --extensions .md,.mdx

# Search
libscope search "how to authenticate"

# Start the MCP server so your AI assistant can query it
libscope serve

On first run with the default embedding provider, LibScope downloads the all-MiniLM-L6-v2 model (~80 MB). Subsequent runs use the cached model.

Document Format Support

LibScope supports Markdown (.md, .mdx) and plain text natively. Additional formats are available via optional dependencies:

Format	Extension	Optional Dependency	Node.js Requirement
PDF	.pdf	pdf-parse (v2)	≥ 20.16 or ≥ 22.3
Word	.docx	mammoth	Any
EPUB	.epub	epub2	Any
PowerPoint	.pptx	pizzip	Any
CSV	.csv	Built-in	Any

The pdf-parse, mammoth, epub2, and pizzip packages are listed as optionalDependencies and install automatically when the Node.js version is compatible. Note: binary .ppt files are not supported — only .pptx.

Using with AI Assistants

LibScope exposes an MCP server over stdio. Point your MCP-compatible client at it:

Cursor — add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "libscope": {
      "command": "npx",
      "args": ["-y", "libscope", "serve"]
    }
  }
}

Claude Code:

claude mcp add --scope user libscope -- npx -y libscope serve

Once connected, your assistant can search docs, submit new documents, rate content quality, and ask RAG-powered questions against your knowledge base.

Tool	What it does
search-docs	Semantic search with topic/library/version/rating filters
ask-question	RAG question-answering with source citations
get-document	Retrieve a document by ID
list-documents	List docs with filters
list-topics	Browse the topic hierarchy
submit-document	Index new content (raw text or a URL to fetch)
update-document	Update a document's title, content, or metadata
delete-document	Remove a document
rate-document	Rate a doc 1–5 with optional feedback and corrections
suggest-tags	Auto-suggest tags based on content analysis
save-search	Save a named search query with filters
list-saved-searches	List all saved searches
run-saved-search	Execute a saved search by name or ID
delete-saved-search	Delete a saved search
link-documents	Create a cross-reference between two documents
get-document-links	List all incoming and outgoing links for a document
delete-link	Remove a cross-reference link
reindex-documents	Re-embed chunks (useful after switching providers)
health-check	DB status, doc/chunk counts
sync-obsidian-vault	Sync an Obsidian vault
sync-onenote	Sync OneNote notebooks via Microsoft Graph
sync-notion	Sync Notion pages and databases
sync-confluence	Sync Confluence spaces
sync-slack	Sync Slack channels and threads
install-pack	Install a knowledge pack
list-packs	List installed or registry packs

Connectors

LibScope can pull documentation from several platforms. Each connector handles incremental syncing so re-runs only process what changed.

# Obsidian — parses wikilinks, frontmatter, embeds, tags
libscope connect obsidian /path/to/vault
libscope connect obsidian /path/to/vault --sync   # incremental re-sync

# Notion
libscope connect notion --token secret_abc123

# Confluence
libscope connect confluence \
  --url https://acme.atlassian.net \
  --email user@acme.com \
  --token $CONFLUENCE_TOKEN

# Slack — index channel messages and threads
libscope connect slack --token xoxb-... --channels general,engineering

# OneNote — device code auth via Microsoft Graph
export ONENOTE_CLIENT_ID=your-client-id
libscope connect onenote

# GitHub / GitLab repos
libscope add-repo https://github.com/org/repo --branch main --path docs/

# Remove a connector's data
libscope disconnect obsidian /path/to/vault

Obsidian: --topic-mapping frontmatter, --exclude "templates/*" "daily/*", --sync

Notion: --exclude page-id-1 db-id-2, --sync

Confluence: --spaces ENG,DEVOPS, --exclude-spaces ARCHIVE

Slack: --thread-mode aggregate|separate, --sync

OneNote: --notebook "Work Notes", --sync

GitHub/GitLab: --token, --branch, --path, --extensions .md,.mdx,.rst

Search and RAG

# Semantic search
libscope search "authentication best practices"
libscope search "API rate limiting" --library my-lib --topic security --limit 10

# Ask questions (needs an LLM provider configured — see Configuration below)
libscope ask "How do I configure OAuth2?" --library my-lib

# Interactive REPL for iterative searching
libscope repl

Search uses sqlite-vec for vector similarity when available, with FTS5 full-text search as a fallback. Results automatically get a 1.5× title boost when the document title matches query words. You can also pass --diversity 0.5 (0–1) for MMR-based diversity reranking.

Programmatic SDK

LibScope exports two embeddable APIs:

LibScope — full SDK with all features (connectors, topics, packs, etc.):

import { LibScope } from "libscope";

const scope = LibScope.create();
await scope.index({ title: "My Doc", content: "..." });
const results = await scope.search("query");
scope.close();

LibScopeLite — lightweight embeddable class for external applications. No CLI, no MCP server, no connectors. Designed for embedding semantic search directly into other tools (MCP servers, VS Code extensions, CI scripts):

import { LibScopeLite } from "libscope/lite";

const lite = new LibScopeLite({ dbPath: ":memory:" });

// Index documents (or code files via tree-sitter chunking)
await lite.indexBatch(docs, { concurrency: 4 });

// Hybrid vector + FTS5 search
const results = await lite.search("how to authenticate");

// Get RAG context for injection into an external LLM prompt
const context = await lite.getContext("How does auth work?");

lite.close();

Tree-sitter powered code indexing splits TypeScript, JavaScript, and Python files at function/class boundaries:

import { TreeSitterChunker } from "libscope/lite";

const chunker = new TreeSitterChunker();
const chunks = await chunker.chunk(sourceCode, "typescript");
// Each chunk is a complete function or class with 1-based line numbers

See the LibScope Lite guide and API reference for the full documentation.

Organizing Content

Topics give your docs a hierarchy. Tags give them flexible labels. Workspaces give you isolated databases.

# Topics
libscope topics create "backend"
libscope topics create "auth" --parent backend --description "Auth & identity"

# Tags
libscope tag add <doc-id> typescript,api,v2

# Workspaces — separate knowledge bases entirely
libscope workspace create my-project
libscope workspace use my-project
libscope --workspace my-project search "deploy steps"

Documents also keep version history, so you can roll back if a re-index goes wrong:

libscope docs history <doc-id>
libscope docs rollback <doc-id> 3

REST API

For programmatic access outside of MCP:

libscope serve --api --port 3378

OpenAPI 3.0 spec at GET /openapi.json. Key endpoints:

Method	Endpoint	Description
GET	/api/v1/search?q=...	Semantic search
POST	/api/v1/batch-search	Batch search (up to 20 queries)
POST	/api/v1/ask	RAG question-answering
GET/POST	/api/v1/documents	List or create documents
GET/PATCH/DELETE	/api/v1/documents/:id	Get, update, or delete a document
POST	/api/v1/documents/url	Index from a URL
POST	/api/v1/documents/:id/tags	Add tags
GET	/api/v1/documents/:id/suggest-tags	Auto-suggest tags
GET/POST	/api/v1/documents/:id/links	List or create cross-references
DELETE	/api/v1/links/:id	Delete a cross-reference
GET/POST	/api/v1/topics	List or create topics
GET	/api/v1/tags	List tags
GET/POST	/api/v1/searches	List or create saved searches
POST	/api/v1/searches/:id/run	Run a saved search
DELETE	/api/v1/searches/:id	Delete a saved search
POST	/api/v1/bulk/delete	Bulk delete documents
POST	/api/v1/bulk/retag	Bulk add/remove tags
POST	/api/v1/bulk/move	Bulk move to a topic
GET/POST	/api/v1/webhooks	List or create webhooks
DELETE	/api/v1/webhooks/:id	Delete a webhook
POST	/api/v1/webhooks/:id/test	Send a test ping to a webhook
GET	/api/v1/analytics/searches	Search analytics and knowledge gaps
GET	/api/v1/connectors/status	Connector sync status and history
GET	/api/v1/stats	Usage statistics
GET	/api/v1/health	Health check

Configuration

LibScope reads config from (highest priority first): environment variables → .libscope.json in your project → ~/.libscope/config.json → built-in defaults.

Embedding Providers

Provider	Default?	Requirements	Model
local	Yes	None (~80 MB model download on first run)	all-MiniLM-L6-v2
ollama		Ollama running locally	nomic-embed-text
openai		LIBSCOPE_OPENAI_API_KEY	text-embedding-3-small

libscope config set embedding.provider ollama
# or
export LIBSCOPE_EMBEDDING_PROVIDER=openai
export LIBSCOPE_OPENAI_API_KEY=sk-...

LLM for RAG

The ask command and ask-question MCP tool need an LLM. Configure one with:

export LIBSCOPE_LLM_PROVIDER=openai   # or ollama, anthropic
export LIBSCOPE_LLM_MODEL=gpt-4o-mini # optional

Variable	Description	Default
LIBSCOPE_EMBEDDING_PROVIDER	local, ollama, or openai	local
LIBSCOPE_OPENAI_API_KEY	OpenAI API key	—
LIBSCOPE_OLLAMA_URL	Ollama server URL	http://localhost:11434
LIBSCOPE_LLM_PROVIDER	LLM for RAG (openai / ollama / anthropic)	—
LIBSCOPE_LLM_MODEL	LLM model override	—
LIBSCOPE_ANTHROPIC_API_KEY	Anthropic API key (for Claude models)	—
LIBSCOPE_ALLOW_PRIVATE_URLS	Allow fetching from private/internal IPs	false
LIBSCOPE_ALLOW_SELF_SIGNED_CERTS	Accept self-signed TLS certificates	false
ONENOTE_CLIENT_ID	Microsoft app registration client ID	—
ONENOTE_TENANT_ID	Microsoft tenant ID	common
NOTION_TOKEN	Notion integration token	—
CONFLUENCE_URL	Confluence base URL	—
CONFLUENCE_EMAIL	Confluence user email	—
CONFLUENCE_TOKEN	Confluence API token	—

{
  "embedding": {
    "provider": "local",
    "ollamaUrl": "http://localhost:11434",
    "ollamaModel": "nomic-embed-text",
    "openaiModel": "text-embedding-3-small"
  },
  "llm": {
    "provider": "openai",
    "model": "gpt-4o-mini",
    "anthropicApiKey": "sk-ant-..."
  },
  "database": {
    "path": "~/.libscope/libscope.db"
  },
  "logging": {
    "level": "info"
  },
  "indexing": {
    "allowPrivateUrls": false,
    "allowSelfSignedCerts": false
  }
}

Corporate / Internal Networks

If you're indexing docs from internal servers (Confluence, wikis, etc.), you may need:

# Allow fetching from private/internal IP addresses
libscope config set indexing.allowPrivateUrls true

# Accept self-signed or corporate TLS certificates
libscope config set indexing.allowSelfSignedCerts true

Or via environment variables:

export LIBSCOPE_ALLOW_PRIVATE_URLS=true
export LIBSCOPE_ALLOW_SELF_SIGNED_CERTS=true

Webhooks

LibScope can push events to any HTTP endpoint. Useful for triggering CI pipelines, Slack notifications, or custom workflows whenever documents are created or updated.

libscope serve --api  # webhooks require the REST API

# Create a webhook
curl -X POST http://localhost:3378/api/v1/webhooks \
  -H "Content-Type: application/json" \
  -d '{"url": "https://hooks.example.com/libscope", "events": ["document.created", "document.updated"], "secret": "my-hmac-secret"}'

# Send a test ping
curl -X POST http://localhost:3378/api/v1/webhooks/<id>/test

Webhook payloads are signed with HMAC-SHA256 when a secret is set. The signature is in the X-LibScope-Signature header.

Supported events: document.created, document.updated, document.deleted.

Other Tools

LibScope ships with a few more utilities beyond the core index-and-search loop:

# Watch a directory and auto-reindex on changes
libscope watch ./docs/

# Re-embed everything after switching embedding providers
libscope reindex

# Find duplicate documents
libscope dedupe

# Export / import the whole knowledge base
libscope export ./backup.json
libscope import-backup ./backup.json

# Usage analytics
libscope stats                  # overview
libscope stats popular          # most-searched docs
libscope stats stale --days 90  # docs nobody searches for

# Knowledge packs — portable document bundles
libscope pack create --name "react-docs" --topic react
libscope pack install ./react-docs.json

Pack Registries

Share and discover knowledge packs through git-based registries. A registry is a git repo with a defined folder structure managed by libscope.

# Add a registry
libscope registry add https://github.com/org/libscope-registry.git --name official

# Search for packs across all registries
libscope registry search "react"

# Install a pack by name (resolves from registries)
libscope pack install react-docs
libscope pack install react-docs@1.2.0    # specific version

# Create your own registry
libscope registry create ./my-registry

# Publish a pack file to your registry
libscope registry publish ./my-pack.json -r my-registry --version 1.0.0

# Submit a pack to someone else's registry (creates a feature branch)
libscope registry publish ./my-pack.json -r community --submit

Authentication is delegated to git — SSH keys and HTTPS credential helpers work automatically. Registries cache locally and support offline index lookups. See the Pack Registries guide for full details.

There's also a web dashboard at http://localhost:3377 when you run libscope serve, with search, document browsing, topic navigation, and a knowledge graph visualization at /graph.

Core

Command	Description
libscope init	Initialize the database
libscope add <fileOrUrl>	Index a file or URL
libscope import <directory>	Bulk import from a directory
libscope import-batch <directory>	Parallel batch import
libscope search <query>	Search
libscope ask <question>	RAG question-answering
libscope repl	Interactive search REPL
libscope serve	Start MCP server (--api for REST)

Documents

Command	Description
libscope docs list	List documents
libscope docs show <id>	Show a document
libscope docs update <id>	Update title/content/metadata
libscope docs delete <id>	Delete a document
libscope docs history <id>	Version history
libscope docs rollback <id> <ver>	Roll back to a prior version

Organization

Command	Description
libscope topics list	List topics
libscope topics create <name>	Create a topic
libscope tag add <id> <tags...>	Add tags
libscope tag remove <id> <tag>	Remove a tag
libscope tag list	List tags
libscope workspace create <name>	Create workspace
libscope workspace list	List workspaces
libscope workspace use <name>	Switch workspace
libscope workspace delete <name>	Delete workspace

Saved Searches

Command	Description
libscope searches list	List all saved searches
libscope searches run <name>	Re-run a saved search
libscope searches delete <name>	Delete a saved search
libscope search <q> --save <name>	Save a search while running it

Document Links

Command	Description
libscope link <srcId> <tgtId>	Create a cross-reference
libscope links <docId>	Show all links for a document
libscope unlink <linkId>	Remove a link
libscope prereqs <docId>	Show prerequisite reading chain

Bulk Operations

Command	Description
libscope bulk delete	Delete all matching documents
libscope bulk retag	Add/remove tags on matching docs
libscope bulk move	Move matching docs to a topic

Connectors

Command	Description
libscope connect obsidian <path>	Sync Obsidian vault
libscope connect onenote	Sync OneNote
libscope connect notion	Sync Notion
libscope connect confluence	Sync Confluence
libscope connect slack	Sync Slack
libscope add-repo <url>	Index a GitHub/GitLab repo
libscope disconnect <name>	Remove connector data

Registries

Command	Description
libscope registry add <url> [-n <alias>]	Register a git repo as a pack registry
libscope registry remove <name>	Unregister a registry
libscope registry list	List configured registries
libscope registry sync [<name>]	Sync one or all registries
libscope registry search <query> [-r <name>]	Search registry pack indexes
libscope registry create <path>	Initialize a new registry repo
libscope registry publish <file> -r <name>	Publish a pack file to a registry
libscope registry unpublish <pack> -r <name>	Remove a pack version from a registry

Utilities

Command	Description
libscope watch <dir>	Auto-reindex on file changes
libscope reindex	Re-embed all chunks
libscope dedupe	Find duplicates
libscope export <path>	Export to JSON
libscope import-backup <path>	Import from backup
libscope stats	Usage overview
libscope pack install <name>	Install a knowledge pack
libscope pack create	Create a knowledge pack
libscope config set <key> <val>	Set config
libscope config show	Show config

Global flags: --verbose, --log-level <level>, --workspace <name>

How It Works

LibScope stores everything in a local SQLite database (at ~/.libscope/libscope.db by default):

• Documents are split into chunks by heading boundaries
• Each chunk is embedded into a vector using the configured provider
• Vector search is done via sqlite-vec; FTS5 full-text search is used as a fallback
• The MCP server reads from this same database over stdio
• Connectors fetch content from external platforms and feed it through the same indexing pipeline

The stack: better-sqlite3 + sqlite-vec + @modelcontextprotocol/sdk + @xenova/transformers for local embeddings.

Contributing

See CONTRIBUTING.md. The short version:

git clone https://github.com/RobertLD/libscope.git
cd libscope && npm install
npm run dev        # watch mode
npm test           # run tests
npm run typecheck  # type check
npm run lint       # lint

License

Business Source License 1.1 — see LICENSE for full terms.