OpenMake LLM

Self-hosted AI Assistant Platform with Multi-Model Orchestration

OpenMake LLM is a high-performance, self-hosted AI assistant platform designed for multi-model orchestration and advanced agentic workflows. It provides a lightweight, framework-free frontend paired with a robust TypeScript backend, supporting local and cloud LLM deployments with intelligent routing and semantic caching.

Key Features

• 7 Brand Model Profiles — Default, Pro, Fast, Think, Code, Vision, Auto, each mapped to different LLM engines via environment configuration
• Intelligent Auto-Routing — LLM classifier + 2-layer semantic cache for optimized query handling via openmake_llm_auto
• 100+ Specialized Agents — 18 industry categories with keyword routing, topic analysis, discussion engine, and skill management
• Deep Research Engine — Multi-step autonomous research with topic decomposition, web scraping, content synthesis, and report generation
• MCP (Model Context Protocol) — 9 built-in tools (web search, scraping, vision, filesystem, deep research, sequential thinking, firecrawl, etc.) with tier-based access, user sandbox, and external MCP client support
• A2A (Agent-to-Agent) Multi-Model — Parallel multi-model orchestration across different API keys and providers
• Real-time Streaming — Low-latency WebSocket-based chat with streaming responses
• RAG (Retrieval-Augmented Generation) — Upload your documents and get AI answers grounded in your own data
• OpenAI-Compatible API — Drop-in replacement endpoint for OpenAI API consumers
• Ollama Cluster Management — Multi-node cluster with load balancing and API key pool rotation (up to 5 keys)
• External LLM Providers (BYO Key, 9 providers) — Each user can register their own API keys directly from the unified model selector in the chat input area (no separate page needed). Keys are AES-256-GCM encrypted at rest, billed to the user's own provider account, and managed via inline ⋮ context menu (validate / usage / delete).
  • Anthropic Claude (native SDK): Opus 4.5 / Sonnet 4.6 / Haiku 4.5
  • OpenAI-compatible (8 providers): OpenRouter (300+ routed models), Google Gemini, Groq (LPU), Together AI, Mistral La Plateforme, Cohere, remote Ollama, custom endpoints
  • 34 models with detailed pricing (USD micros) + capability inference (vision/thinking/tool calling/embedding) auto-detected per model ID
  • 90-day usage retention with per-call cost tracking

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Frontend (Vanilla JS SPA)                 │
│              ES Modules · No Framework · Vite Dev            │
└────────────────────────┬────────────────────────────────────┘
                         │ REST + WebSocket
┌────────────────────────▼────────────────────────────────────┐
│                  Backend (Express 5 + TypeScript)            │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌───────────────┐  │
│  │  Routes   │ │  Auth    │ │  MCP     │ │  WebSocket    │  │
│  │  (25+)    │ │  JWT/    │ │  Tools   │ │  Streaming    │  │
│  │          │ │  OAuth   │ │  Router  │ │               │  │
│  └────┬─────┘ └──────────┘ └──────────┘ └───────────────┘  │
│       │                                                      │
│  ┌────▼──────────────────────────────────────────────────┐  │
│  │              Chat Pipeline                             │  │
│  │  Query → Classifier → Semantic Cache → Model Selector  │  │
│  │       → Domain Router → Context Engineering → Stream   │  │
│  └───────────────────────────────────────────────────────┘  │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌───────────────┐  │
│  │ 100+     │ │  Deep    │ │  RAG &   │ │  Monitoring   │  │
│  │ Agents   │ │ Research │ │  Memory  │ │  & Analytics  │  │
│  └──────────┘ └──────────┘ └──────────┘ └───────────────┘  │
└────────────────────────┬────────────────────────────────────┘
                         │
          ┌──────────────┼──────────────┐
          ▼              ▼              ▼
    ┌──────────┐  ┌──────────┐  ┌──────────┐
    │PostgreSQL│  │  Ollama  │  │  Ollama  │
    │          │  │  (Local) │  │  (Cloud) │
    └──────────┘  └──────────┘  └──────────┘

Tech Stack:

• Backend: Express 5, TypeScript (strict mode), CommonJS output, ES2022
• Frontend: Vanilla JS SPA with ES Modules — no framework, no JS build step
• Database: PostgreSQL via pg — raw parameterized SQL, auto-schema on launch, no ORM
• Process Manager: PM2
• CI/CD: GitHub Actions — 4 gates (Bun Test → TS Build → File Size Guard → ESLint)
• Observability: OpenTelemetry

Quick Start

Overview — Clone to first chat in 6 steps:

1. Install prerequisites (Node.js, PostgreSQL, Ollama)
2. Clone the repository and run npm install
3. Copy .env.example to .env and set 5 required variables
4. Pull the local embedding model (ollama pull nomic-embed-text)
5. Start the server (npm run dev)
6. Open http://localhost:52416 and log in

Prerequisites

Required

Optional

• PM2 — Production process manager

  npm install -g pm2
  

• Playwright — Required only for E2E tests

  npx playwright install
  

Setup Guides

brew install node
node -v   # Verify v20.0+
npm -v    # Verify v10.0+

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash
source ~/.zshrc
nvm install 20
node -v

# Install
brew install postgresql@16

# Start service (auto-start on boot)
brew services start postgresql@16

# Verify status
brew services list

# Connect to PostgreSQL
psql postgres

# Run the following SQL (change the password to your own)
CREATE USER openmake WITH PASSWORD 'your_password';
CREATE DATABASE openmake_llm OWNER openmake;
GRANT ALL PRIVILEGES ON DATABASE openmake_llm TO openmake;
\q

# Verify installation
ollama --version

# Start Ollama service (or just launch the Ollama app)
ollama serve

# Node.js (NodeSource)
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs

# PostgreSQL
sudo apt-get install -y postgresql postgresql-contrib
sudo systemctl start postgresql
sudo systemctl enable postgresql

# Create PostgreSQL user and database
sudo -u postgres psql -c "CREATE USER openmake WITH PASSWORD 'your_password';"
sudo -u postgres psql -c "CREATE DATABASE openmake_llm OWNER openmake;"

# Ollama
curl -fsSL https://ollama.com/install.sh | sh
ollama serve &

# In PowerShell (Run as Administrator)
wsl --install -d Ubuntu
# Restart your PC, then open "Ubuntu" from Start menu
# Follow the Linux (Ubuntu/Debian) guide above

# PowerShell
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"

Tested Environment

Installation

# Clone
git clone https://github.com/openmake/openmake_llm.git
cd openmake_llm

# Install dependencies
npm install

# Configure environment
cp .env.example .env

Configure .env

Open the .env file and set the following 5 required variables:

# 1. DATABASE_URL — PostgreSQL connection string (use credentials from setup above)
DATABASE_URL=postgresql://openmake:your_password@localhost:5432/openmake_llm

# 2. JWT_SECRET — Auth token signing key (generate with: openssl rand -hex 32)
JWT_SECRET=paste_generated_64_char_hex_string_here

# 3. API_KEY_PEPPER — API key hashing salt (generate with: openssl rand -hex 32)
API_KEY_PEPPER=paste_generated_64_char_hex_string_here

# 4. ADMIN_PASSWORD — Initial admin account password
#    Must be 8+ chars with uppercase, lowercase, digit, and special character
ADMIN_PASSWORD=YourSecurePassword123!

# 5. OLLAMA_API_KEY_1 — Ollama Cloud API key (required for cloud models)
#    Get your key from https://ollama.com/settings
OLLAMA_API_KEY_1=your_ollama_api_key_here

Tip: Generate secret keys from your terminal (produces a random 64-character hex string):

# macOS / Linux
openssl rand -hex 32

# Windows (PowerShell) — if openssl is not available
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"

Run the command twice — once for JWT_SECRET and once for API_KEY_PEPPER.

Ollama Cloud vs Local — which should I use?

	Cloud Models (:cloud suffix)	Local Models
How it works	Requests are sent to Ollama Cloud servers	Models run on your own machine's CPU/GPU
API key required?	Yes — at least one OLLAMA_API_KEY_*	No
Hardware needed	Minimal (any machine)	GPU with 8GB+ VRAM recommended (varies by model)
Cost	Free tier available — see ollama.com/pricing for limits	Free (uses your electricity)
Setup	Set OLLAMA_API_KEY_1 in .env	ollama pull <model> then update OLLAMA_DEFAULT_MODEL in .env

Default configuration uses Cloud models. All default models use the :cloud suffix (e.g., gemini-3-flash-preview:cloud). To switch to local models, change OLLAMA_DEFAULT_MODEL to a local model (e.g., llama3.2:latest) and run ollama pull llama3.2 first.

Start the Server

# Pull the local embedding model
ollama pull nomic-embed-text

# Start development server
npm run dev

The database schema is automatically created on first launch. When the server starts successfully, you should see output similar to:

[Server] OpenMake LLM server listening on port 52416
[Database] Connected to PostgreSQL
[Database] Schema initialized

First Login

Open http://localhost:52416 in your browser. You can:

• Admin login — Use the email from DEFAULT_ADMIN_EMAIL in your .env (default: admin@example.com) with the ADMIN_PASSWORD you set above.
• Register — Create a new account from the registration tab.
• Guest mode — Click "Continue as Guest" for limited access without an account.

What to Do After Login

1. Start a chat — Type a message in the chat input. The default model is the configured Ollama model.
2. Switch models — Click the 📋 model selector at the right of the input area (next to the send button). Dropdown shows your local Ollama model + any external LLM providers you've registered (Anthropic / OpenRouter / Gemini / Groq / etc.). Pure Manual mode — your selection is never overridden by auto-routing.
3. Register external LLM keys — From the same dropdown, click "+ 새 LLM 키 등록" → choose a provider → enter your API key. Registered models appear immediately in the dropdown.
4. Try an expert agent — Open the Agent panel to select a specialist (e.g., Software Engineer, Financial Analyst) for domain-specific conversations.
5. Explore the Skill Library — Browse available tools and capabilities in the Skill Library tab.
6. Admin settings — If logged in as admin, visit the Admin panel to manage users, models, and system configuration.

Production

# Build (required — compiles TypeScript to JavaScript)
npm run build

# Start with PM2
pm2 start ecosystem.config.js

# Or start directly
npm start

Note: You must run npm run build before npm start or pm2 start. The build step compiles TypeScript source into backend/api/dist/. Update the cwd path in ecosystem.config.js to match your project directory before using PM2.

Configuration

All settings are managed via .env. See .env.example for the full reference.

Essential Variables

Supported Models & Engine Mapping

Each brand profile routes queries to a specialized cloud model via Ollama:

Local Embedding Model

• nomic-embed-text:latest (274 MB) — Used for vector embeddings in semantic search and RAG. Runs locally to keep embedding fast and private.

  ollama pull nomic-embed-text
  

Optional Integrations

• Google OAuth 2.0 — GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET
• Google Custom Search — GOOGLE_API_KEY, GOOGLE_CSE_ID
• Language Policy — DEFAULT_RESPONSE_LANGUAGE (20+ languages supported)

External LLM Providers (BYO Key Workflow)

Each user can register their own API keys directly from the chat input area — no separate page or admin role required. Operators only need to set TOKEN_ENCRYPTION_KEY once.

Workflow:

1. Login → main chat page
2. Click the model selector trigger (📋) at the right of the input area
3. Open the dropdown → "+ 새 LLM 키 등록" section lists all unregistered providers
4. Click any provider (e.g., "+ Anthropic Claude") → key registration modal
5. Enter API key → registered key's models automatically populate the dropdown
6. ⋮ context menu next to any registered model → validate / view usage / delete

Supported providers (catalog):

Pricing & Capability:

• 34 models with built-in USD pricing (1M token 단위, BIGINT micros 누적 정확도)
• Capability auto-inference per model ID — vision (gpt-4o, claude-3+, gemini-, pixtral 등), thinking (claude-opus-4, deepseek-r1, o1/o3), embedding (text-embedding-*)
• Cohere command-r-* 는 native tools 미지원 — 자동 비활성
• /v1/models 빈 응답 시 provider별 fallback 모델 보강 (Gemini 3 / OpenRouter 6 / Groq 2 / Together 2 / Mistral 3 / Cohere 2)

Usage tracking:

• 모든 외부 호출별 토큰/비용/지연 자동 기록 (external_provider_usage 테이블)
• ⋮ → 📊 사용량 모달: 직전 50건 raw 표 + 최근 30일 provider별 누계 박스 (호출수 / 토큰 / 비용 USD)
• GET /api/external-keys/usage/summary?days=N REST endpoint (max 90일)
• 90일 자동 보존 (db-retention cron, 환경변수 EXTERNAL_USAGE_RETENTION_DAYS)

Phase 2 (planned): OpenAI ChatGPT Plus/Pro OAuth (구독 계정 sign-in). 단, OpenAI/Anthropic 모두 표준 third-party OAuth client 등록 미공개 — 실현 가능성은 provider 정책 변경 의존. Anthropic Claude Pro/Max OAuth는 ToS 명시 금지로 영구 제외.

Project Structure

backend/api/src/
├── routes/          # 25+ Express route modules (REST API)
├── services/        # Core: ChatService, DeepResearch, RAG, Memory, Embedding
├── chat/            # Pipeline: classifier, model-selector, domain-router, cache
├── agents/          # 100+ industry agents, keyword router, discussion engine
├── mcp/             # Tool router, tiers, external client, user sandbox
├── auth/            # JWT, OAuth, API keys, RBAC, scope middleware
├── data/            # PostgreSQL repositories, migrations
├── sockets/         # WebSocket streaming handler
├── config/          # Environment, constants, limits, model defaults
├── monitoring/      # Analytics, token tracking
├── ollama/          # Ollama client wrapper
└── cluster/         # Multi-node cluster management

frontend/web/public/
├── js/modules/         # Core modules (chat, auth, state, websocket, sanitize)
│   ├── pages/          # 23 page modules (admin, analytics, research, documents...)
│   └── components/     # Reusable components (model-selector, add-key-modal,
│                       #                       usage-modal, model-action-menu)
└── css/                # Design tokens, components, model-selector styles

Development

npm run dev              # API + Frontend (concurrent)
npm run dev:api          # Backend only
npm run dev:frontend     # Frontend only (Vite)
npm run build            # Full production build
npm run lint             # ESLint

Testing

npm test                 # Jest unit tests
npm run test:e2e         # Playwright E2E (Chromium)
npm run test:e2e:ui      # Playwright interactive UI mode

API

OpenMake LLM provides an OpenAI-compatible endpoint (/api/v1/chat/completions), allowing it to serve as a drop-in replacement for applications using the OpenAI API.

Interactive API documentation is available at http://localhost:52416/api/docs when running in development mode.

Selected Domain Endpoints

/external-keys.html URL 은 폐기됨 — /?openModelSelector=1 로 301 redirect.

Skill Library

Security

• Authentication: JWT (JSON Web Token) access/refresh tokens in HttpOnly cookies
• OAuth: Google OAuth 2.0 social login
• API Keys: HMAC-SHA-256 hashed, scope-based access control
• Authorization: RBAC (Role-Based Access Control) — admin, user, and guest roles
• Rate Limiting: Per-route rate limiting to prevent abuse
• XSS Defense: Content sanitization via sanitize.js
• CORS: Configurable origin whitelist

Contributing

Contributions are welcome! Please ensure:

1. Strict TypeScript — no any types in the backend
2. Vanilla JS only — no frontend frameworks
3. Parameterized SQL — no raw string concatenation in queries
4. Tests — unit tests for new services, E2E for user-facing features
5. File size — source files must stay under 600 lines (CI enforced)

Troubleshooting

Glossary

License

MIT © 2026 OpenMake Contributors