Pixeltable

pxt.Image, pxt.Video, pxt.Audio, pxt.Document, pxt.Json – manage diverse data consistently.

t = pxt.create_table(
    'media',
    {
        'img': pxt.Image,
        'video': pxt.Video,
        'audio': pxt.Audio,
        'document': pxt.Document,
        'metadata': pxt.Json,
    },
)

→ Type System · Tables & Data

Define processing steps once; they run automatically on new/updated data. Supports API calls (OpenAI, Anthropic, Gemini), local inference (Hugging Face, YOLOX, Whisper), vision models, and any Python logic.

# LLM API call
t.add_computed_column(
    summary=openai.chat_completions(
        messages=[{'role': 'user', 'content': t.text}], model='gpt-4o-mini'
    )
)

# Local model inference
t.add_computed_column(
    classification=huggingface.vit_for_image_classification(t.image)
)

# Vision analysis (multimodal)
t.add_computed_column(
    description=openai.chat_completions(
        messages=[{'role': 'user', 'content': [
            {'type': 'text', 'text': 'Describe this image'},
            {'type': 'image_url', 'image_url': t.image},
        ]}],
        model='gpt-4o-mini'
    )
)

→ Computed Columns · AI Integrations · Sample App: Prompt Studio

Create views with iterators to explode one row into many (video→frames, doc→chunks, audio→segments).

from pixeltable.functions.video import frame_iterator
from pixeltable.functions.document import document_splitter

# Document chunking with overlap & metadata
chunks = pxt.create_view(
    'chunks', docs,
    iterator=document_splitter(
        document=docs.doc,
        separators='sentence,token_limit',
        overlap=50, limit=500
    )
)

# Video frame extraction
frames = pxt.create_view(
    'frames', videos,
    iterator=frame_iterator(video=videos.video, fps=0.5)
)

→ Views · Iterators · RAG Pipeline

Add embedding indexes and perform similarity searches directly on tables/views.

t.add_embedding_index(
    'img',
    embedding=clip.using(model_id='openai/clip-vit-base-patch32')
)

sim = t.img.similarity(string='cat playing with yarn')
results = t.order_by(sim, asc=False).limit(10).collect()

→ Embedding Indexes · Semantic Search · Image Search App

Extend Pixeltable with UDFs, reusable queries, batch processing, and custom aggregators.

@pxt.udf
def format_prompt(context: list, question: str) -> str:
    return f'Context: {context}\nQuestion: {question}'

@pxt.query
def search_by_topic(topic: str):
    return t.where(t.category == topic).select(t.title, t.summary)

→ UDFs Guide · Custom Aggregates

Register @pxt.udf, @pxt.query functions, or MCP servers as callable tools. LLMs decide which tool to invoke; Pixeltable executes and stores results.

# Load tools from MCP server, UDFs, and query functions
mcp_tools = pxt.mcp_udfs('http://localhost:8000/mcp')
tools = pxt.tools(get_weather_udf, search_context_query, *mcp_tools)

# LLM decides which tool to call; Pixeltable executes it
t.add_computed_column(
    tool_output=invoke_tools(tools, t.llm_tool_choice)
)

→ Tool Calling Cookbook · Agents & MCP · Pixelbot · Pixelagent

Unlike pandas/polars, Pixeltable persists everything, parallelizes API calls automatically, caches results, and turns your experiment into production with one line change. No separate notebook → pipeline handoff:

# Explore with a familiar DSL — filter, sample, apply UDFs ephemerally
results = (
    t.where(t.score > 0.8)
    .order_by(t.timestamp)
    .select(t.image, score=t.score)
    .limit(10)
    .collect()
)

# Sample 5 rows and test a UDF — nothing stored, API calls parallelized and cached
t.sample(5).select(t.text, summary=summarize(t.text)).collect()

# Happy? One line to commit — runs on full dataset, skips already-cached rows
t.add_computed_column(summary=summarize(t.text))

→ Queries & Expressions · Iterative Workflow · Version Control

All data is automatically stored and versioned. Query any prior version.

t = pxt.get_table('my_table')  # Get a handle to an existing table
t.revert()  # Undo the last modification

t.history()  # Display all prior versions
old_version = pxt.get_table('my_table:472')  # Query a specific version

→ Version Control · Data Sharing

Pixeltable ships with a built-in local dashboard that launches automatically when you start a session. Browse tables, inspect schemas, view media with lightbox navigation, visualize your full data pipeline as a DAG, and track computation errors — all from your browser.

import pixeltable as pxt

# Dashboard launches automatically at http://localhost:22089
pxt.init()

# Disable if needed
pxt.init(config_overrides={'start_dashboard': False})
# Or set environment variable: PIXELTABLE_START_DASHBOARD=false

Highlights: Table browser with sorting & filtering · Media preview (images, video, audio) · Column lineage visualization · Pipeline graph · Per-column error tracking · CSV export · Auto-refresh

No extra dependencies. No setup. It's just there.

Import from any source and export to ML formats.

# Import from files, URLs, S3, Hugging Face
t.insert(pxt.io.import_csv('data.csv'))
t.insert(pxt.io.import_huggingface_dataset(dataset))

# Export to analytics/ML formats
pxt.io.export_parquet(table, 'data.parquet')
pytorch_ds = table.to_pytorch_dataset('pt')  # → PyTorch DataLoader ready
coco_path = table.to_coco_dataset()          # → COCO annotations

# ML tool integrations
pxt.create_label_studio_project(table, label_config)  # Annotation
pxt.export_images_as_fo_dataset(table, table.image)   # FiftyOne

→ Data Import · PyTorch Export · Label Studio · Data Wrangling for ML