Taking Stock Fast DB
Refactoring the Taking Stock DB for speeeeed
Installation
npx taking-stock-fast-dbAsk AI about Taking Stock Fast DB
Powered by Claude Β· Grounded in docs
I know everything about Taking Stock Fast DB. Ask me about installation, configuration, usage, or troubleshooting.
0/500
Reviews
Documentation
Taking Stock - Fast Database
Analytics backend and AI-powered query interface for Michael Mandiberg's Taking Stock artwork.
Overview
This project migrates the Taking Stock database from MySQL/MongoDB to ClickHouse for fast analytical queries over tens of millions of stock photos. It provides an AI-queryable interface via MCP (Model Context Protocol) and a web chat UI for natural language data exploration.
The MooseStack service runs ClickHouse and exposes an MCP server. The web app provides a chat interface where users ask questions in natural language, Claude generates SQL queries via MCP tools, and results are returned for display.
Architecture
This is a pnpm monorepo with three packages:
packages/moosestack-service/ β ClickHouse-backed data service with MCP server. Handles data ingestion and exposes query tools via API at http://localhost:4000.
packages/web-app/ β Next.js chat interface for AI-powered data exploration. Connects to MooseStack MCP server and uses Anthropic Claude for query generation.
packages/data-faker/ β Test data generator for performance testing. Generates realistic variant data matching the images_analytical schema with checkpoint/resume support. See data-faker README for details.
Getting Started
Clone and install dependencies:
pnpm install
Copy example environment variables
cp packages/moosestack-service/.env.{example,local}
cp packages/web-app/.env.{example,local}
Create API Key authentication tokens
$ cd packages/moosestack-service
moose generate hash-token # use output for the API Key & Token below
# if you get "command not found: moose" try this version instead:
pnpm exec moose generate hash-token
Set environment variables
- Set your API Key in packages/moosestack-service/.env.local to the
ENV API KEYgenerated by moose generate hash-token - Set your API Token in packages/web-app/.env.local to the
Bearer Tokengenerated by moose generate hash-token - Set your Anthropic API key in packages/web-app/.env.local
Start both services:
pnpm dev
Or start services individually:
pnpm dev:moose # Start MooseStack service only
pnpm dev:web # Start web app only
pnpm dev:faker # Run data faker (generates test data)
Data Faker
The data-faker package generates realistic test data for performance testing. It supports checkpoint/resume, graceful shutdown, and generates ~100GB of variant data.
Quick start:
# Quick test (100 rows)
pnpm dev:faker --rows 100
# Check current row count
pnpm dev:faker --count
# Full scale (100GB, ~200M rows)
pnpm dev:faker --rows 200000000
# Resume from checkpoint
pnpm dev:faker
See packages/data-faker/README.md for complete documentation.
Note: The data faker requires MooseStack service to be running (or at least ClickHouse accessible) to insert data. Run pnpm dev:moose first if ClickHouse isn't already running.
MCP Tools Available
query_clickhouse- Execute read-only SQL queries against ClickHouse with automatic result limitingget_data_catalog- Discover available tables and views with their schema information
Learn More
- Data Migration Plan - Strategy for migrating from MySQL/MongoDB to ClickHouse
- Development Guide - Data models and ingestion API
- Security - Security features and production considerations
- MooseStack Documentation
- Model Context Protocol
- MCP SDK (@modelcontextprotocol/sdk)
- WebApp Class Reference
Built with MooseStack by Fiveonefour
