MidStream

Real-time LLM streaming with inflight analysis

Treat an LLM token stream as a first-class signal — pattern-match it, score it, intervene on it — while the tokens are still arriving. MidStream is the Rust workspace + WASM bindings + npm shims that make that practical: a nanosecond scheduler, a multi-stream QUIC transport, dynamical-systems analysis, and a self-referential meta-learning loop, all under one consistent feature flag policy and a hard supply-chain gate.

Why MidStream?

Most "LLM tooling" treats the response as a black box that opens at the end. MidStream treats it as a stream of evidence that you can analyze, gate, and steer per chunk. Built in Rust by rUv so the analysis layer adds microseconds, not seconds — and ships the same code path to native, browser, and edge via WASM.

What MidStream Does

One workspace, six published libraries, one binary. The libraries are independently useful (pattern matching, real-time scheduling, attractor analysis, QUIC multi-stream) and compose into a single inflight-analysis pipeline. The binary wires them together with the OpenAI Realtime API and a console dashboard.

Provider stream  ──▶  zero-copy Bytes  ──▶  inflight pipeline  ──▶  decisions
   (OpenAI RT,         (ADR-0006)             │
    Anthropic,                                ├─ temporal-compare    (DTW, LCS, edit-distance)
    custom)                                   ├─ scheduler           (ns-scale priority queue)
                                              ├─ attractor-studio    (Lyapunov, phase-space)
                                              ├─ neural-solver       (LTL + neural reasoning)
                                              ├─ strange-loop        (meta-learning)
                                              └─ quic-multistream    (transport, 0-RTT)
                                                       │
                                              AIMDS safety gate (optional)
                                                       │
                                                       ▼
                                              Dashboard · MCP · WASM · npm

New to MidStream? Start with one crate. temporal-compare and scheduler ship clean tests, run on WASM, and have zero unsafe code — they're the easiest entry points. Move up to strange-loop once you want the full meta-learning loop.

Quick Start

Three different entry points depending on what you need. Pick one:

Path A — One crate

# Pattern matching only
cargo add midstreamer-temporal-compare

# Real-time scheduler only
cargo add midstreamer-scheduler

# Or the full library stack
cargo add midstreamer-temporal-compare midstreamer-scheduler \
          midstreamer-attractor midstreamer-neural-solver \
          midstreamer-strange-loop midstreamer-quic

Path B — Full workspace

git clone https://github.com/ruvnet/midstream.git
cd midstream

# Build everything
cargo build --workspace --release

# Run the binary
cargo run --release --bin midstream -- --help

# Run the bench suite
cargo bench --workspace

See docs/GETTING_STARTED.md for the full zero-to-working-build walkthrough.

Path C — WASM / browser

# In a node project
npm install @midstream/wasm

# Or build from source
cd npm-wasm
npm install
npm run build:wasm

import { TemporalCompare, Scheduler } from '@midstream/wasm';

const cmp = new TemporalCompare();
const distance = cmp.dtw(seriesA, seriesB);

What You Get

Rust Workspace

Six published library crates at the same workspace version (currently 0.2.1). Each one is independently useful and can be added without pulling the others.

Crate	What it does	crates.io
midstreamer-temporal-compare	Sequence comparison: Dynamic Time Warping (DTW), Longest Common Subsequence (LCS), edit distance, with LTL extensions
midstreamer-scheduler	Ultra-low-latency priority queue with strict Ord semantics (priority-major, deadline-minor). Designed for ns-scale arrival rates
midstreamer-attractor	Dynamical-systems analysis: Lyapunov exponents, phase-space reconstruction, attractor classification (fixed-point / periodic / chaotic)
midstreamer-neural-solver	Linear Temporal Logic (LTL) verification fused with a small neural reasoner. Used for inflight safety/intent checks
midstreamer-strange-loop	Self-referential meta-learning: the system can observe and adjust its own analysis policy mid-stream
midstreamer-quic	QUIC multi-stream transport (quinn-backed) for native; thin shim on the browser side. 0-RTT, stream prioritization, secure-by-default TLS

The midstream binary at the workspace root wires these into the inflight pipeline.

Every Rust library that doesn't touch the OS compiles to wasm32-unknown-unknown from the same source tree:

cargo build -p midstreamer-temporal-compare --target wasm32-unknown-unknown --no-default-features
cargo build -p midstreamer-scheduler        --target wasm32-unknown-unknown --no-default-features
cargo build -p midstreamer-strange-loop     --target wasm32-unknown-unknown --no-default-features

The npm-wasm/ package builds with wasm-pack, ships TypeScript declarations, and works in Node ≥ 18, modern browsers, and edge runtimes (Cloudflare Workers, Deno, Bun). Network egress from the WASM sandbox is gated by an allowlist — ADR-0015.

                   ┌──────────────────────────────┐
                   │     LLM provider stream      │
                   │ (OpenAI RT · Anthropic · ...)│
                   └─────────────┬────────────────┘
                                 ▼
                       ┌─────────────────────┐
                       │  midstream binary   │
                       │  (zero-copy Bytes)  │
                       └─────────────┬───────┘
                                 ▼
        ┌─────────────────────────────────────────────────┐
        │              Inflight pipeline                  │
        │                                                 │
        │  temporal-compare ─► scheduler ─► attractor     │
        │           │              │            │         │
        │           ▼              ▼            ▼         │
        │       neural-solver  strange-loop  dashboard    │
        └─────────────────────────────────────────────────┘
                                 │
                  ┌──────────────┼──────────────┐
                  ▼              ▼              ▼
              MCP server     WASM bundle    OTLP traces
              (ADR-0032)     (ADR-0003)     (ADR-0010)

See docs/ARCHITECTURE.md for the long form, and docs/adr/ for the decisions that shaped each layer.

Performance

cargo bench --workspace runs the full criterion suite. Honest benchmarks against real workloads, not mocks — ADR-0009.

Highlights from the current baseline (Ubuntu 24.04, Ryzen 9 7950X):

Full numbers (CSV + plots + methodology) in docs/BENCHMARKS.md.

Documentation

Four canonical docs. Everything else is either an ADR or archived.

Plus the 41 ADRs — every architectural decision, immutable, with status tracked (Proposed → Accepted → Superseded). Browse the index by topic: foundational · perf/SOTA · security · API · transport · TS surface · dashboard · governance.

Development

# Run the workspace tests (skip midstream root + hyprstream; broken under --all-features)
cargo test --workspace --exclude midstream --exclude hyprstream --all-features

# Lint
cargo clippy --workspace --exclude midstream --exclude hyprstream --all-targets --all-features -- -D warnings

# Format check
cargo fmt --all -- --check

# Supply-chain gate (matches CI)
cargo deny --workspace check --hide-inclusion-graph advisories
cargo deny --workspace check --hide-inclusion-graph bans
cargo deny --workspace check --hide-inclusion-graph licenses
cargo deny --workspace check --hide-inclusion-graph sources
cargo audit

# Property tests
cargo test --workspace --exclude midstream --exclude hyprstream --test 'proptest_*'

# Fuzz one target
cargo +nightly fuzz run dtw_does_not_panic -- -runs=100000

The xtask crate (cargo xtask --help) replaces hand-rolled shell scripts — ADR-0037.

Project Conventions

• Modular Cargo workspace. One root, every member is in crates/ or a top-level directory — ADR-0001.
• Off-by-default features, prefixed by domain (backend-, provider-, wasm-, insecure-) — ADR-0025.
• thiserror for libraries, anyhow for binaries — ADR-0018.
• Workspace-wide lints. dbg!, todo!(), unimplemented!(), mem::forget are all deny — ADR-0034.
• Cargo.lock is committed. Standard for binary-shipping workspaces; required for cargo audit and --locked MSRV checks.
• Released via release.yml. Tag vX.Y.Z, the workflow handles cargo set-version, dependency-DAG publish, and the GitHub release — ADR-0017.

Contributing

PRs welcome. The bar:

1. Every change references an ADR (or proposes a new one if it's a real decision).
2. New code has property tests where the surface is parseable / generative; unit tests otherwise.
3. cargo clippy clean and cargo fmt applied — both are CI gates.
4. Security-sensitive changes (TLS, network, deserialization) get an explicit ADR-0011 / ADR-0015 review.

See CONTRIBUTING.md, CODE_OF_CONDUCT.md, and SECURITY.md — all ratified under ADR-0039.

Support

License

Dual-licensed under MIT OR Apache-2.0 — pick whichever is more convenient. The same dual licence applies to every workspace member, every WASM artifact, and every npm package. See LICENSE-MIT and LICENSE-APACHE. ADR-0036 explains why.