🎹 Droplets

Let AI make music in your DAW — Connect Claude, GPT, or any AI to Ableton, Bitwig, Logic, and more!

Droplets is a VST3/CLAP plugin that bridges AI assistants and your DAW. Ask Claude to play a melody, create a drum pattern, or automate your synth — and hear it instantly in your project.

Built by Christophe Poucet (Simply Chris).

---

✨ What Can It Do?

• 🎵 Real-time MIDI — AI sends notes and CC messages directly to your instruments
• 🔁 Looping Patterns — Create "fugues" that sync to your DAW's transport
• 🎛️ Parameter Control — Automate any plugin via MIDI CC or parameter slots
• 📤 MIDI Export — Drag AI-created patterns into your arrangement as clips
• 🎚️ MIDI 2.0 — Per-note pitch bend, pressure, and 16-bit velocity

---

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         YOUR DAW                                │
│  ┌─────────────────┐    MIDI     ┌─────────────────────────┐    │
│  │                 │◄───────────►│    Your Instruments     │    │
│  │    Droplets     │             │                         │    │
│  │     Plugin      │             │  🎹 Synths              │    │
│  │                 │             │  🥁 Drums               │    │
│  └────────┬────────┘             │  🎸 Samplers            │    │
│           │                      └─────────────────────────┘    │
└───────────┼─────────────────────────────────────────────────────┘
            │ MCP (HTTP)
            ▼
┌─────────────────────────────────────────────────────────────────┐
│                        AI ASSISTANT                             │
│                                                                 │
│   🤖 Claude, GPT, or any MCP-compatible AI                      │
│                                                                 │
│   "Play a jazz chord progression"                               │
│   "Create a 4-bar drum loop"                                    │
│   "Sweep the filter cutoff from 20% to 80%"                     │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

---

🚀 Quick Start

1️⃣ Install the Plugin

Build from source (see Building from Source below) and run cargo xtask install — it copies the plugin to the right location automatically.

Platform	VST3 Location	CLAP Location
🍎 macOS	~/Library/Audio/Plug-Ins/VST3/	~/Library/Audio/Plug-Ins/CLAP/
🪟 Windows	C:\Program Files\Common Files\VST3\	C:\Program Files\Common Files\CLAP\
🐧 Linux	~/.vst3/	~/.clap/

2️⃣ Load in Your DAW

Add Droplets as a MIDI effect and route its output to your instruments.

Bitwig: drop Droplets on a Note Effects chain before any instrument; MIDI flows downstream automatically.

Ableton Live: the VST3 build is classified as an instrument (a clap-wrapper limitation — the wrapper can't emit a MIDI-effect-only VST3 category today). Use Ableton's two-track MIDI-routing pattern:

1. Put Droplets on its own MIDI track.
2. Create a second MIDI track with your real instrument (drum rack, synth, sampler, etc.).
3. On the second track, set MIDI From to the Droplets track and select Droplets in the plugin dropdown below it.
4. Arm the second track (or set monitor to In) so it hears the incoming MIDI.

Droplets still appears in the Plug-Ins browser under the Instruments list. Proper MIDI-effect classification is tracked as a post-demo fix.

3️⃣ Connect Your AI

Droplets speaks MCP over streamable HTTP on http://localhost:9999/mcp. MCP clients that speak HTTP directly (e.g. Claude Code, Cursor) can use that URL as-is:

{
  "mcpServers": {
    "droplets": {
      "url": "http://localhost:9999/mcp"
    }
  }
}

Claude Desktop is stdio-only and does not support streamable HTTP. Bridge through mcp-remote, passing --allow-http so it accepts the non-TLS loopback URL:

{
  "mcpServers": {
    "droplets": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "http://localhost:9999/mcp",
        "--allow-http"
      ]
    }
  }
}

4️⃣ Make Music! 🎉

Ask Claude:

"Play middle C for 2 beats, then E and G together"

Or get fancy:

"Create a chill lo-fi beat with swing"

---

🎼 The Fugue System

Fugues are musical sequences that sync perfectly with your DAW's transport. Think of them as AI-generated MIDI clips that play in real-time.

Features

Feature	Description
🎯 Quantized Start	Begin on the next beat, bar, or 4-bar phrase
🔄 Looping	Play once, N times, or forever
🎹 Layering	Stack multiple patterns simultaneously
🏷️ Tags	Group patterns (e.g., cancel all "drums" at once)
📈 CC Automation	Smooth parameter sweeps with interpolation

Compact Format

Fugues use a compact JSON schema designed for LLM token efficiency. Each fugue is a typed block — notes, cc, per_note_pitch_bend, per_note_pressure, or a composite that bundles several together:

{
  "fugues": [
    {
      "type": "notes",
      "tag": "melody",
      "notes": [
        { "beat": 0, "note": "C4", "duration": 1, "velocity": 100 },
        { "beat": 1, "note": "E4", "duration": 1 },
        { "beat": 2, "note": "G4", "duration": 2 }
      ]
    },
    {
      "type": "cc",
      "cc": 74,
      "points": [[0, 20], [4, 100]],
      "interpolation": "exp"
    }
  ],
  "quantize": "bar",
  "loop_mode": "forever"
}

Notes auto-generate their own note-offs at beat + duration. CC points interpolate smoothly on the audio thread.

---

🛠️ MCP Tools

MIDI is fugue-only by design. LLM round-trip latency (~1–10s) is too high for musically-timed one-shot events — notes, CC, and MIDI 2.0 per-note expression all go through queue_fugue, which schedules events on the audio thread with sample-accurate timing. Use a composite fugue to bundle notes + CC automation + per-note pitch bend + pressure into one atomic musical moment.

🎼 Fugues

Tool	Description
queue_fugue	Schedule one or more fugues (notes / cc / pitch_bends / pressures / composite) with tempo-quantized start
list_fugues	List active + pending fugues with timing and loop progress
get_fugue	Read a single fugue's full content back in the same compact shape queue_fugue accepts — enables read-modify-write
import_fugue	Import a base64-encoded .mid as one or more fugues (the return leg of drag-out → edit-in-DAW → hand back)
cancel_fugue	Stop a specific fugue by id
cancel_fugues_by_tag	Stop every fugue sharing a tag (e.g. "melody")
clear_fugues	Emergency stop — cancel every fugue on an instance

🎚️ Transport & context

Tool	Description
get_transport	Current {beat, tempo, playing, time_sig, loop bounds} for reasoning about scheduling
get_project_state	Call first when composing. Summary of connected Droplets instances, each track's primary device, and drum-pad maps (pitch notation + sample names) so the LLM writes correct notes instead of GM conventions

🏷️ Instances & slots

Tool	Description
list_instances	Every connected Droplets plugin process
set_instance_name	Rename an instance (bass, pad, lead) for clearer targeting
list_slots	Parameter slots with their CC mappings and current values

---

🖥️ Plugin UI

Four tabs in the plugin window:

🎼 Sequencer

• Every active fugue renders its own piano-roll grid with a live playhead.
• Drag a row (or the ⇣ Drag all button) onto your DAW to drop a .mid clip of the fugue(s).
• Drop a .mid back onto the sequencer panel to re-queue it as fugues (drag-round-trip).
• Cancel, export, or export-all via the row controls.

🎛️ MIDI Mapping

• Per-instance list of CC slot parameters with their names and mapped CC numbers.
• Wiggle a slot to test the mapping; use MIDI-learn to bind a hardware controller.

🎹 DAW

• Live view of what the host controller extension (Bitwig today) has pushed: tracks, primary devices, drum-pad maps with sample names, per-track Remote Controls.
• Empty state shown when no extension is running (Ableton, Logic, older Bitwig).

⚙️ Settings

• MCP server URL (copy for your AI's config).
• Export folder for manually-saved .mid files.
• Custom instructions appended to the MCP system prompt.

---

📤 MIDI Export

Turn AI compositions into DAW clips:

1. Click ↓ on any fugue in the list
2. File saves to your exports folder
3. Finder/Explorer opens automatically
4. Drag the .mid file into your DAW!

---

🔧 Building from Source

Prerequisites

• 🦀 Rust 1.70+
• 📦 Node.js 18+
• 🎛️ For VST3: a VST3 SDK checkout, with CLAP_WRAPPER_VST3_SDK pointing at it

Build + Install (macOS, Linux, Windows)

git clone https://github.com/poucet/droplets.git
cd droplets

# Build bundles for your platform into target/bundle/
cargo xtask build

# …or build AND copy into your DAW's user plugin folders in one step
cargo xtask install

cargo xtask install detects your OS and copies the CLAP/VST3 bundles to the right per-user location:

Platform	CLAP	VST3
🍎 macOS	~/Library/Audio/Plug-Ins/CLAP/	~/Library/Audio/Plug-Ins/VST3/
🐧 Linux	~/.clap/	~/.vst3/
🪟 Windows	%LOCALAPPDATA%\Programs\Common\CLAP\	%LOCALAPPDATA%\Programs\Common\VST3\

Useful flags:

cargo xtask build --format clap            # skip VST3 if you don't have the SDK
cargo xtask build --profile debug --dev-gui # dev build with WebView devtools
cargo xtask install --skip-build           # just copy existing bundles

---

📁 Project Structure

droplets/
├── 🦀 src/
│   ├── lib.rs              # Plugin entry point
│   ├── fugue/              # 🎼 Fugue sequencer
│   │   ├── sequencer.rs    # Transport-synced playback
│   │   ├── export.rs       # MIDI file export
│   │   └── settings.rs     # Persistent settings
│   ├── mcp/                # 🤖 AI communication
│   │   ├── server.rs       # MCP tool definitions
│   │   └── bridge.rs       # Plugin ↔ server bridge
│   ├── gui/                # 🖥️ UI backend
│   │   ├── api.rs          # REST endpoints
│   │   └── routes.rs       # Request routing
│   └── midi/               # 🎹 MIDI processing
│
├── ⚛️ frontend/            # React UI
│   ├── src/
│   │   ├── App.tsx         # Main app
│   │   └── components/     # UI components
│   └── package.json
│
└── 🔨 xtask/               # Build tools

---

⚙️ Configuration

Ports

Service	Port
MCP Server	9999
GUI Server	9998

Settings Location

Platform	Path
🍎 macOS	~/.droplets/settings.json
🪟 Windows	%APPDATA%\Droplets\settings.json
🐧 Linux	~/.config/droplets/settings.json

Export Folder

Platform	Default Path
🍎 macOS	~/Music/Droplets/Exports/
🪟 Windows	Documents\Droplets\Exports\
🐧 Linux	~/Music/Droplets/Exports/

---

🤝 Contributing

Contributions welcome! Feel free to:

• 🐛 Report bugs
• 💡 Suggest features
• 🔧 Submit pull requests

---

📜 License

MIT License — see LICENSE for details.

---

🙏 Credits

Built with love using:

• clack-plugin — Rust CLAP framework
• rmcp — MCP server
• wry — WebView
• midly — MIDI files
• React — UI

---

Made with 🎵 by Christophe Poucet (Simply Chris)