Chiming into conversations (the magic moment):

👤 Alice: I wonder what the weather will be like tomorrow
👤 Bob: Yeah, we should check before planning the picnic
👤 Alice: Jarvis, what do you think?
  📝 Heard: "What do you think Jarvis?"
  🧠 Intent (wake word): directed → "what do you think about the weather for the picnic"

✨ Working on it: what do you think about the weather for the picnic
  🧰 Tool: getWeather…
  💬 Generating response...

🤖 Jarvis
Tomorrow looks great for a picnic! Sunny with highs around 22°C...

Jarvis understood the entire conversation and gave a contextual answer — no need to repeat the question.

Browser automation via MCP:

  📝 Heard: "Open YouTube Jarvis."
  🧠 Intent (wake word): directed → "open YouTube"

✨ Working on it: open YouTube
  💬 Generating response...

🤖 Jarvis
I have opened YouTube for you.

Health-aware advice:

  📝 Heard: "Should I order pizza tonight considering my health goals Jarvis?"
  🧠 Intent (wake word): directed → "should I order pizza tonight considering my health goals"

✨ Working on it: should I order pizza tonight considering my health goals
  🧰 Tool: fetchMeals…
  💬 Generating response...

🤖 Jarvis
Looking at your week — you mentioned wanting to lose 5kg by June...
Today you've had about 860 calories so far. Given your 1,800 calorie target,
you've got room for pizza! But maybe consider thin crust to save calories.

Personalised news search:

  📝 Heard: "What are some news from today that might interest me Jarvis?"
  🧠 Intent (wake word): directed → "what are some news from today that might interest me"

✨ Working on it: what are some news from today that might interest me
  🧰 Tool: searchMemory…
  🧰 Tool: webSearch…
  💬 Generating response...

🤖 Jarvis
Here's a quick snapshot of today's headlines...

Natural wake word placement (say "Jarvis" anywhere):

  📝 Heard: "Give me a random topic to discuss Jarvis."
  🧠 Intent (wake word): directed → "give me a random topic to discuss"

✨ Working on it: give me a random topic to discuss
  💬 Generating response...

🤖 Jarvis
How about the Fermi Paradox? Given the vast number of stars...

Echo detection (Jarvis ignores its own speech):

🤖 Jarvis
I have opened YouTube for you.

👂 Listening for follow-up (3s)...
  📝 Heard: "I have opened YouTube for you."
  🔇 Heard (echo): "i have opened youtube for you."
💤 Returning to wake word mode

Language Modes

• Multilingual (default, 99 languages): "whisper_model": "medium"
• English Only (slightly better English accuracy): "whisper_model": "medium.en"

Model Sizes

Speed is relative to the original large model. Source

GPU Acceleration (Windows)

If you have an NVIDIA GPU, Jarvis can use CUDA for much faster speech recognition. The Windows installer offers an optional CUDA download during setup. For development:

pip install nvidia-cublas-cu12 nvidia-cudnn-cu12

CUDA is detected automatically — no configuration needed.

Hallucination Filters

Whisper sometimes produces confident but false transcriptions during silence or background noise (e.g. news-show intros, music). Two thresholds filter these out before they reach the intent judge:

• "whisper_min_confidence": 0.3 — drops segments whose avg_logprob-derived confidence falls below this value. Raise if you see low-confidence noise leaking through; lower if real speech is being dropped.
• "whisper_no_speech_threshold": 0.5 — drops any segment whose no_speech_prob is at or above this value, regardless of avg_logprob. Catches the case where Whisper is confident about a hallucinated phrase but its own no-speech signal says the audio was silent. Applies to both the faster-whisper and MLX backends.

Both thresholds are exposed in the Settings window under Whisper.

LLM Intent Judge - Jarvis uses gemma4:e2b for intelligent voice intent classification (echo detection, query extraction, stop commands). This model is automatically installed alongside your chosen chat model during setup. The intent judge cannot be disabled but gracefully falls back to simpler text matching if Ollama is unavailable.

Tool Router - When "tool_selection_strategy": "llm" (the default), Jarvis asks a small LLM to pick which tools are relevant for each query, shrinking the tool catalogue the chat model sees. By default this routing call reuses the intent-judge model — it's already warm and small enough not to stall the turn. Override with "tool_router_model": "<name>" to dedicate a different model to routing. Other strategies: "keyword" (fast, no LLM), "embedding" (nomic-embed-text), "all" (no filtering).

Task-list Planner - Before the agentic loop, Jarvis runs a short planning pass that decomposes multi-step queries into an ordered list of sub-tasks. For small models (gemma4:e2b class), each planned step is directly resolved to a concrete tool call without relying on the chat model to re-plan turn-by-turn. This significantly improves multi-step reliability. Config options:

{
  "planner_enabled": true,          // set to false to disable the planner entirely
  "planner_model": "",              // override which model plans (default: reuses tool_router_model chain)
  "planner_timeout_sec": 6.0        // per-call timeout for plan and step-resolver LLM calls
}

Small chat models (~2B, e.g. gemma4:e2b) degrade sharply as their prompt grows. Jarvis runs two cheap distil passes to keep the prompt tight:

• Memory digest — boils diary + graph recall into a short relevance-filtered note before injecting it as background context.
• Tool-result digest — boils a raw tool payload (especially webSearch UNTRUSTED WEB EXTRACT blocks) into a short attributed fact note before it reaches the main reply model.

Both digest passes auto-enable for small models (≤7B) and stay off for large models. For small models, tool-result digest also prevents large fetch_web_page payloads from blowing the context window. Override in ~/.config/jarvis/config.json:

{
  "memory_digest_enabled": null,          // null = auto-on for SMALL, false to force off, true to force on
  "tool_result_digest_enabled": null,     // null = auto-on for SMALL, false to force off, true to force on
  "llm_digest_timeout_sec": 8.0           // tight ceiling shared by both passes
}

Field logs show 🧩 Memory digest: … and 🧩 Tool digest: … lines when a pass ran, so you can see when the substrate was replaced.

Piper TTS (default) - Neural TTS that auto-downloads on first use (~60MB):

• Works out of the box - no setup required
• High-quality British English male voice (en_GB-alan-medium)
• Fast local synthesis with exact duration tracking

To use different Piper voices, download from HuggingFace and set:

{
  "tts_piper_model_path": "~/.local/share/jarvis/models/piper/en_GB-alan-medium.onnx"
}

Chatterbox - AI voice with emotion control (requires running from source):

{ "tts_engine": "chatterbox" }

Voice cloning with Chatterbox - add a 3-10 second .wav sample:

{
  "tts_engine": "chatterbox",
  "tts_chatterbox_audio_prompt": "/path/to/voice.wav"
}

Jarvis can provide location-aware responses (weather, local time, etc.) using a local GeoLite2 database — no cloud geolocation services are used.

IP detection chain (in order of preference):

1. Manual IP — configure location_ip_address in settings
2. UPnP — queries your local router (no traffic leaves LAN)
3. Socket heuristic — determines which interface routes externally (no data sent)
4. OpenDNS DNS query — single myip.opendns.com lookup to 208.67.222.222 (only external query)

If your ISP uses carrier-grade NAT (CGNAT), Jarvis automatically resolves your true public IP via the same OpenDNS DNS query. This can be disabled:

{
  "location_cgnat_resolve_public_ip": false
}

Setup: Register for a free MaxMind GeoLite2 account, download the City database (MMDB format), and save it to ~/.local/share/jarvis/geoip/GeoLite2-City.mmdb. The setup wizard will guide you through this.

Connect Jarvis to external tools via MCP servers:

{
  "mcps": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": { "GITHUB_TOKEN": "your-token" }
    }
  }
}

Popular integrations:

• Home Assistant - Voice control for smart home
• Google Workspace - Gmail, Calendar, Drive, Docs
• GitHub - Issues, PRs, workflows
• Notion - Knowledge management
• Slack/Discord - Team communication
• Databases - MySQL, PostgreSQL, MongoDB
• Composio - 500+ apps in one integration

See full MCP setup guide below.

1. Add MCP Server integration in Home Assistant (Settings → Devices & services)
2. Expose entities you want to control (Settings → Voice assistants → Exposed entities)
3. Create Long-lived Access Token (Profile → Security → Create token)
4. Install proxy: uv tool install git+https://github.com/sparfenyuk/mcp-proxy
5. Add to config:

{
  "mcps": {
    "home_assistant": {
      "command": "mcp-proxy",
      "args": ["http://localhost:8123/mcp_server/sse"],
      "env": { "API_ACCESS_TOKEN": "YOUR_TOKEN" }
    }
  }
}

"Jarvis, turn on the living room lights" / "set bedroom to 72°" / "run good night scene"

{
  "mcps": {
    "google_workspace": {
      "command": "npx",
      "args": ["-y", "google-workspace-mcp"],
      "env": {
        "GOOGLE_CLIENT_ID": "your-client-id",
        "GOOGLE_CLIENT_SECRET": "your-client-secret"
      }
    }
  }
}

Setup: taylorwilsdon/google_workspace_mcp

{
  "mcps": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": { "GITHUB_TOKEN": "your-token" }
    }
  }
}

Notion:

{ "mcps": { "notion": { "command": "npx", "args": ["-y", "@makenotion/mcp-server-notion"], "env": { "NOTION_API_KEY": "your-token" } } } }

Slack:

{ "mcps": { "slack": { "command": "npx", "args": ["-y", "slack-mcp-server"], "env": { "SLACK_BOT_TOKEN": "xoxb-...", "SLACK_USER_TOKEN": "xoxp-..." } } } }

Discord:

{ "mcps": { "discord": { "command": "npx", "args": ["-y", "discord-mcp-server"], "env": { "DISCORD_BOT_TOKEN": "your-token" } } } }

Databases: bytebase/dbhub (SQL), mongodb-mcp-server (MongoDB)

{
  "mcps": {
    "composio": {
      "command": "npx",
      "args": ["-y", "@composiohq/rube"],
      "env": { "COMPOSIO_API_KEY": "your-key" }
    }
  }
}

Get API key at composio.dev

First startup takes a bit - Jarvis pre-warms the Whisper, chat, and intent-judge models before announcing "Listening!" so the first engagement feels instant. This adds a few seconds on cold start and is bounded at 60 s — if Ollama is slow, Jarvis will start listening anyway and load the models on demand.

Jarvis doesn't hear me - Check microphone permissions, speak clearly after "Jarvis"

Responses are slow - Ensure you have enough VRAM (8GB+ for default model; see System Requirements for other models)

Windows: App won't start - Extract full zip first, check Windows Defender

macOS: "App can't be opened" - Right-click → Open, or System Settings → Privacy & Security → Allow

Linux: No tray icon - sudo apt install libayatana-appindicator3-1

Jarvis keeps deflecting on questions it answered before - small models can record their own past failures into the diary, which then primes future sessions to repeat them. New writes are scrubbed automatically; to clean historical entries, open the Memory Viewer, switch to the Diary tab, and click Clean up deflection narration in the sidebar Maintenance section. Only sentences that narrate the assistant's failures are removed; the rest of each entry stays.

git clone https://github.com/isair/jarvis.git
cd jarvis

# macOS
bash scripts/run_macos.sh

# Windows (with Micromamba)
pwsh -ExecutionPolicy Bypass -File scripts\run_windows.ps1

# Linux
bash scripts/run_linux.sh

Running from source enables Chatterbox TTS (AI voice with emotion/cloning). Piper TTS works in both bundled and source modes.

{
  "web_search_enabled": false,
  "wikipedia_fallback_enabled": false,
  "brave_search_api_key": "",
  "mcps": {},
  "location_auto_detect": false,
  "location_cgnat_resolve_public_ip": false,
  "location_enabled": false
}

Verify: sudo lsof -i -n -P | grep jarvis (should only show 127.0.0.1 to Ollama)

When DuckDuckGo is rate-limited or returns nothing fetchable, Jarvis walks a small fallback chain before giving up rather than confabulating:

1. Brave Search — opt-in, requires brave_search_api_key. Free tier: 2,000 queries/month. Get a key at api.search.brave.com.
2. Wikipedia — zero-config, on by default, uses the Wikipedia host matching the language Whisper auto-detected on the utterance (so a Turkish question gets a Turkish answer). Disable with wikipedia_fallback_enabled: false.
3. Honest failure — if every provider fails, the reply tells you the search was blocked rather than making something up.

The whole chain is bounded by a ~20s wall-clock deadline so a stalled provider can't run out the voice-assistant latency budget.