HestiaClaw

Hestia is a self-hosted smart home AI assistant built around Home Assistant. She runs entirely on your own infrastructure — no cloud middleman, no subscription, no data leaving your server unless you choose it. Talk to her in the browser, ask about your home, trigger automations, and watch her build a persistent memory of your world through a live 3D knowledge graph.

Under the hood: a React + Vite frontend, an Express agent harness, real-time token streaming, a layered memory system (pinned facts + Graphiti long-term graph), MCP tool support, voice I/O, and a skills system for custom behaviors.

Home Environment Systems Technology Intelligence Assistant

Features

• Native agent harness — built-in agent loop with multi-provider LLM support: Anthropic Claude, OpenAI / GPT, or any OpenAI-compatible endpoint (Ollama, LM Studio, etc.)
• Home Assistant integration — 87+ HA tools via ha-mcp; control devices, query entities, trigger automations, and check states directly from chat
• HA Assist voice integration — connect Home Assistant's Assist voice pipeline directly to Hestia via the webhook-conversation integration; voice sessions appear in the sidebar alongside regular chats
• Layered memory system — pinned facts file (MEMORY.md) + per-turn Graphiti recall + long-term knowledge graph; memories are promoted, consolidated, and pruned automatically each night
• MCP tool support — connect any MCP server in agent.config.json; tools are auto-registered and immediately available to the LLM
• Skills system — drop SKILL.md files into skills/ to define custom agent behaviors; invoke with /skill-name or let the model pick them up automatically
• 3D knowledge graph — full-screen force-directed visualization of the Graphiti/Neo4j memory graph, with community coloring, degree sizing, and one-click GDS recompute
• Runtime provider & model switching — swap between Anthropic and OpenAI and pick any model from a live-fetched dropdown without restarting the server; settings persist across restarts
• Agent diagnostics panel — inspect provider, MCP server health, tool metadata, skills, and fully traced recent runs
• Harness controls — browser-mediated approvals for risky tools, bounded context compaction, session forking; toggle approvals with /approvals in chat
• Real-time streaming — responses appear token-by-token; tool call badges show which tools fired and when
• Voice I/O — hold to speak (ElevenLabs realtime STT), spoken replies after mic-originated turns (ElevenLabs TTS)
• Thinking animation — rotating rings, hex grid, pulsing core while the agent is thinking; shows active tool name live
• Multi-conversation sidebar — persistent conversation history sorted by recency, create / switch / delete / search sessions
• Markdown + syntax highlighting — GFM rendering, auto-fenced code blocks, Prism highlighting with copy button

Screenshots

Chat Interface

Knowledge Graph

Install as Home Assistant Add-on

The easiest way to run HestiaClaw is directly inside Home Assistant as an add-on. No separate server required — it runs alongside HA and connects automatically.

Add-ons in this repository

Adding the repository to HA exposes three add-ons:

ha-mcp is bundled inside HestiaClaw — it runs as an embedded subprocess and doesn't appear as a separate add-on. If you have a standalone ha-mcp add-on installed, HestiaClaw won't use it; it connects to its own internal copy instead. You can uninstall the standalone one to avoid running it twice.

If you only want chat + Home Assistant control, install HestiaClaw alone. Add the other two if you want persistent memory and the knowledge graph view.

1. Add the repository

In Home Assistant → Settings → Add-ons → Add-on Store → overflow menu (⋮) → Repositories, add:

https://github.com/cl0ud6uru/HestiaClaw

2. Install HestiaClaw

Find HestiaClaw in the store and click Install.

3. Create a long-lived access token

The add-on needs a Home Assistant token to control your devices:

1. Go to your HA profile (bottom-left avatar) → Security tab
2. Scroll to Long-lived access tokens → Create token
3. Name it HestiaClaw and copy the value

4. Configure the add-on

In the HestiaClaw add-on → Configuration tab:

5. Start and open

Click Start. Once running, the HestiaClaw panel appears in the HA sidebar.

Note: Graphiti / Neo4j are optional. Without them the memory and knowledge graph features are disabled, but chat and all HA tools work normally.

Quickstart

The full stack — Neo4j, Graphiti MCP, ha-mcp, and Hestia — starts with one command.

git clone https://github.com/cl0ud6uru/HestiaClaw
cd HestiaClaw

cp .env.example .env
# Edit .env — set SESSION_SECRET, BOOTSTRAP_ADMIN_PASSWORD, OPENAI_API_KEY,
# NEO4J_PASSWORD, HA_URL, and HA_TOKEN at minimum

cp agent.config.example.json agent.config.json
# Edit agent.config.json — set your provider, model, and MCP servers.
# The core memory + Home Assistant policy is built into the app; per-install
# persona/tone goes in data/SOUL.md (editable from the Hestia settings panel).

docker compose up -d

App runs at http://localhost:3001 · Neo4j browser at http://localhost:7474

Note: Two extra files — graphiti_mcp_server.py and graphiti_config.yaml — must exist in the project root before running docker compose up. They patch the Graphiti container to accept cross-container connections and write memory to Neo4j instead of its embedded FalkorDB. Copy them from a running zepai/knowledge-graph-mcp instance or see CLAUDE.md for details.

Configuration

.env

cp .env.example .env

agent.config.json

The agent harness activates when this file is present. Copy the example and customize:

cp agent.config.example.json agent.config.json

{
  "provider": {
    "type": "openai",
    "model": "gpt-5.4-mini"
  },
  "harness": {
    "approvals": true,
    "approvalTimeoutMs": 60000,
    "compactionEnabled": true,
    "contextMaxMessages": 40,
    "systemPromptLocked": true
  },
  "mcpServers": {
    "home-assistant": {
      "url": "http://ha-mcp:8086/mcp"
    },
    "graphiti": {
      "url": "http://graphiti:8000/mcp"
    }
  }
}

The core memory + Home Assistant policy is built in (locked by default). Per-install persona and tone go in data/SOUL.md, which is editable from the Agent Harness settings panel. Developers can opt out of the lock with harness.systemPromptLocked: false or HESTIA_SYSTEM_PROMPT_LOCKED=false, in which case systemPrompt from this file is used instead.

Supported providers:

MCP servers support both stdio configs (command/args) and remote HTTP configs (url). ${VAR} references in env and headers blocks expand from the environment at startup.

All settings changed through the Agent Harness panel (provider, model, system prompt, context window, etc.) are written back to agent.config.json automatically and persist across container restarts.

Home Assistant Assist Integration

Hestia can serve as the AI backend for Home Assistant's Assist voice pipeline via the webhook-conversation custom integration. Voice commands spoken to an HA voice satellite (e.g. a Voice PE device) are routed through Hestia's full agent harness — including MCP tools, long-term memory, and entity awareness.

Setup

1. Install the Webhook Conversation integration in Home Assistant (HACS → Custom repositories → eulemitkeule/webhook-conversation)

2. Add a new Conversation agent in the integration and configure it:

   Field	Value
   Webhook URL	https://your-hestia-domain/api/webhook/conversation
   Authentication	Basic HTTP authentication
   Username	anything (e.g. hestia) — not validated
   Password	value of WEBHOOK_SECRET in your .env
   Output field name	output

3. Generate a secure webhook secret:

   openssl rand -base64 32
   

   Set it as WEBHOOK_SECRET in .env and restart the container.

4. In Home Assistant → Settings → Voice assistants, select your new Hestia conversation agent.

How it works

• Each HA voice conversation uses its own conversation_id, which maps to a dedicated thread in the HestiaClaw sidebar — browse past voice sessions just like regular chat sessions
• HA's list of exposed entities (with entity IDs, names, states, and areas) is injected into the agent's context each turn — Hestia knows your devices immediately without extra MCP lookups
• Hestia's own system prompt and memory system are used; HA's generic system prompt is ignored
• Write-tool approvals are bypassed for webhook calls (no browser is available during a voice interaction)
• The sidebar polls for new voice sessions every 30 seconds and sorts them to the top as they receive new messages

Memory System

Hestia uses a three-layer memory architecture so she actually remembers things:

1. Pinned facts (data/MEMORY.md) — the highest-confidence durable facts, injected into every turn. The agent can update this file directly (with your approval), and a nightly consolidation job regenerates it from the full Graphiti graph.

2. Active recall — before each reply, Hestia runs a fast semantic search against your Graphiti graph using your message as the query. The top results are silently injected into her context so relevant memories surface without her having to explicitly look them up.

3. Graphiti knowledge graph — all conversations write episodes to Graphiti. Entities and relationships accumulate over time and are visible in the 3D graph view. The nightly consolidation job prunes duplicates and promotes the best facts to the pinned layer.

To trigger consolidation manually: POST /api/agent/consolidate

The pinned memory file lives at data/MEMORY.md inside the hestia_data Docker volume. To inspect or edit it directly:

docker exec hestiaclaw-hestia-1 cat /app/data/MEMORY.md

Skills

Skills are pre-defined agent behaviors stored as SKILL.md files in the skills/ directory.

skills/
  morning-brief/
    SKILL.md
  lights-off/
    SKILL.md

SKILL.md format:

---
name: morning-brief
description: Deliver a morning briefing covering overnight activity and what's on for the day.
user-invocable: true
argument-hint: (optional hint shown in the palette)
disable-model-invocation: false
---

Your instructions for the agent go here. This becomes part of the system prompt
when this skill is active.

Invoking skills:

• Type / in the chat input to open the command palette — navigate with arrow keys, select with Enter or Tab
• Type /morning-brief and send to invoke directly
• Skills with disable-model-invocation: false are injected into the system prompt so the model can invoke them automatically when the context matches

Built-in slash commands:

Knowledge Graph

Click KNOWLEDGE GRAPH in the header to open the full-screen 3D force-directed view. Requires Neo4j running with NEO4J_URI, NEO4J_USER, and NEO4J_PASSWORD set in .env.

• Nodes colored by Louvain community, sized by degree centrality
• Click a node to see its relationships in the info panel
• RECOMPUTE re-runs the full GDS pipeline (community detection + degree centrality) server-side

As Hestia converses and queries Home Assistant, Graphiti extracts entities and builds relationships in the graph automatically — it grows on its own.

Agent Harness

API routes

All routes require authentication except /api/auth/login and /api/webhook/conversation.

Context management

Long conversations use bounded context: recent messages are kept verbatim, older messages are summarized and injected into the effective system prompt. Configurable via harness.contextMaxMessages in agent.config.json or the Agent Harness panel at runtime.

N8N Mode

A legacy N8N streaming webhook integration is still available as a selectable backend. Toggle between N8N and Agent mode from the sidebar. Set N8N_WEBHOOK_URL in .env and configure your N8N node with Response Mode: Streaming.

Built With

• Frontend: React 19 + Vite 8, plain CSS, Orbitron & Rajdhani fonts
• Backend: Express 5, SQLite (better-sqlite3) — auth, sessions, agent history, audit log
• LLM: @anthropic-ai/sdk + openai SDK (Responses API)
• MCP: @modelcontextprotocol/sdk stdio + HTTP client
• Memory: Graphiti + Neo4j + GDS (via Docker)
• Rendering: react-markdown + remark-gfm + react-syntax-highlighter (Prism)
• 3D graph: three.js + 3d-force-graph
• Voice: ElevenLabs realtime STT WebSocket + TTS proxy
• Auth: Argon2 password hashing, express-session, helmet
• Scheduling: node-cron (daily memory consolidation)

Acknowledgments

Hestia stands on the shoulders of some excellent open-source projects:

• Graphiti by Zep AI — the temporal knowledge graph engine that powers Hestia's long-term memory
• ha-mcp by homeassistant-ai — the MCP server that gives Hestia her Home Assistant tools
• webhook-conversation by eulemitkeule — the HA custom integration that bridges Assist voice pipeline to external AI backends
• Home Assistant — the open-source smart home platform this whole thing is built around
• Neo4j — the graph database backing the knowledge graph visualization
• 3d-force-graph + three.js — the 3D graph rendering stack
• ElevenLabs — realtime voice STT and TTS
• Model Context Protocol — the open standard that makes MCP servers plug-and-play with any compliant agent