HestiaClaw

Hestia is a self-hosted smart home AI assistant built around Home Assistant. She runs entirely on your own infrastructure — no cloud middleman, no subscription, no data leaving your server unless you choose it. Talk to her in the browser, ask about your home, trigger automations, and watch her build a persistent memory of your world through a live 3D knowledge graph.

Under the hood: a React + Vite frontend, an Express agent harness, real-time token streaming, a layered memory system (pinned facts + Graphiti long-term graph), MCP tool support, voice I/O, and a skills system for custom behaviors.

Home Environment Systems Technology Intelligence Assistant

---

Features

• Native agent harness — built-in agent loop with multi-provider LLM support: Anthropic Claude, OpenAI / GPT, or any OpenAI-compatible endpoint (Ollama, LM Studio, etc.)
• Home Assistant integration — 87+ HA tools via ha-mcp; control devices, query entities, trigger automations, and check states directly from chat
• HA Assist voice integration — connect Home Assistant's Assist voice pipeline directly to Hestia via the webhook-conversation integration; voice sessions appear in the sidebar alongside regular chats
• Layered memory system — pinned facts file (MEMORY.md) + per-turn Graphiti recall + long-term knowledge graph; memories are promoted, consolidated, and pruned automatically each night
• MCP tool support — connect any MCP server in agent.config.json; tools are auto-registered and immediately available to the LLM
• Skills system — drop SKILL.md files into skills/ to define custom agent behaviors; invoke with /skill-name or let the model pick them up automatically
• 3D knowledge graph — full-screen force-directed visualization of the Graphiti/Neo4j memory graph, with community coloring, degree sizing, and one-click GDS recompute
• Runtime provider & model switching — swap between Anthropic and OpenAI and pick any model from a live-fetched dropdown without restarting the server; settings persist across restarts
• Agent diagnostics panel — inspect provider, MCP server health, tool metadata, skills, and fully traced recent runs
• Harness controls — browser-mediated approvals for risky tools, bounded context compaction, session forking; toggle approvals with /approvals in chat
• Real-time streaming — responses appear token-by-token; tool call badges show which tools fired and when
• Voice I/O — hold to speak (ElevenLabs realtime STT), spoken replies after mic-originated turns (ElevenLabs TTS)
• Thinking animation — rotating rings, hex grid, pulsing core while the agent is thinking; shows active tool name live
• Multi-conversation sidebar — persistent conversation history sorted by recency, create / switch / delete / search sessions
• Markdown + syntax highlighting — GFM rendering, auto-fenced code blocks, Prism highlighting with copy button

---

Screenshots

Chat Interface

Knowledge Graph

---

Install as Home Assistant Add-on

The easiest way to run HestiaClaw is directly inside Home Assistant as an add-on. No separate server required — it runs alongside HA and connects automatically.

Add-ons in this repository

Adding the repository to HA exposes three add-ons:

Add-on	Purpose	Required?
HestiaClaw	The main AI assistant — chat UI, agent harness, voice, skills	✅ Always
HestiaClaw Neo4j	Neo4j Community Edition with GDS plugin	Optional — only needed for the knowledge graph and Graphiti memory
HestiaClaw Graphiti	Graphiti temporal knowledge graph MCP server	Optional — enables long-term memory; requires Neo4j

ha-mcp is bundled inside HestiaClaw — it runs as an embedded subprocess and doesn't appear as a separate add-on. If you have a standalone ha-mcp add-on installed, HestiaClaw won't use it; it connects to its own internal copy instead. You can uninstall the standalone one to avoid running it twice.

If you only want chat + Home Assistant control, install HestiaClaw alone. Add the other two if you want persistent memory and the knowledge graph view.

1. Add the repository

In Home Assistant → Settings → Add-ons → Add-on Store → overflow menu (⋮) → Repositories, add:

https://github.com/cl0ud6uru/HestiaClaw

2. Install HestiaClaw

Find HestiaClaw in the store and click Install.

3. Create a long-lived access token

The add-on needs a Home Assistant token to control your devices:

1. Go to your HA profile (bottom-left avatar) → Security tab
2. Scroll to Long-lived access tokens → Create token
3. Name it HestiaClaw and copy the value

4. Configure the add-on

In the HestiaClaw add-on → Configuration tab:

Option	Required	Description
provider	✅	anthropic or openai
model	✅	Model ID (e.g. claude-opus-4-7, gpt-4o)
anthropic_api_key	Provider	Anthropic API key
openai_api_key	Provider	OpenAI API key
ha_token	✅	Long-lived access token created above
admin_username	✅	Login username for the Hestia UI
admin_password	✅	Login password — auto-generated and logged if left blank
system_prompt	—	Deprecated. Ignored at runtime. The core memory + Home Assistant policy ships built-in; edit data/SOUL.md from the Hestia settings panel for per-install customization
elevenlabs_api_key	Voice	ElevenLabs key for STT + TTS
elevenlabs_default_voice_id	Voice	Default TTS voice ID
graphiti_url	Memory	URL of an external Graphiti MCP server (e.g. http://your-server:8001/mcp)
neo4j_uri	Graph	Bolt URI for an external Neo4j instance
neo4j_user	Graph	Neo4j username, default neo4j
neo4j_password	Graph	Neo4j password
searxng_url	Search	SearXNG instance URL for web search
session_secret	—	Session cookie secret; auto-generated if blank

5. Start and open

Click Start. Once running, the HestiaClaw panel appears in the HA sidebar.

Note: Graphiti / Neo4j are optional. Without them the memory and knowledge graph features are disabled, but chat and all HA tools work normally.

---

Quickstart

The full stack — Neo4j, Graphiti MCP, ha-mcp, and Hestia — starts with one command.

git clone https://github.com/cl0ud6uru/HestiaClaw
cd HestiaClaw

cp .env.example .env
# Edit .env — set SESSION_SECRET, BOOTSTRAP_ADMIN_PASSWORD, OPENAI_API_KEY,
# NEO4J_PASSWORD, HA_URL, and HA_TOKEN at minimum

cp agent.config.example.json agent.config.json
# Edit agent.config.json — set your provider, model, and MCP servers.
# The core memory + Home Assistant policy is built into the app; per-install
# persona/tone goes in data/SOUL.md (editable from the Hestia settings panel).

docker compose up -d

App runs at http://localhost:3001 · Neo4j browser at http://localhost:7474

Note: Two extra files — graphiti_mcp_server.py and graphiti_config.yaml — must exist in the project root before running docker compose up. They patch the Graphiti container to accept cross-container connections and write memory to Neo4j instead of its embedded FalkorDB. Copy them from a running zepai/knowledge-graph-mcp instance or see CLAUDE.md for details.

---

Configuration

.env

cp .env.example .env

Variable	Required	Description
SESSION_SECRET	✅	Long random string for session cookies
BOOTSTRAP_ADMIN_USERNAME	✅	First-run admin username
BOOTSTRAP_ADMIN_PASSWORD	✅	First-run admin password
NEO4J_PASSWORD	✅	Shared password for Neo4j and Graphiti
HA_URL	✅	Home Assistant URL (e.g. https://ha.yourdomain.com)
HA_TOKEN	✅	Home Assistant long-lived access token
OPENAI_API_KEY	Agent / Graphiti	OpenAI key — also used by Graphiti for embeddings
OPENAI_BASE_URL	—	Override for Ollama, LM Studio, etc.
ANTHROPIC_API_KEY	Agent mode	Anthropic key (if using Claude models)
ELEVENLABS_API_KEY	Voice	ElevenLabs key for STT + TTS
ELEVENLABS_DEFAULT_VOICE_ID	Voice	Default TTS voice ID
ELEVENLABS_TTS_MODEL_ID	—	TTS model, default eleven_flash_v2_5
ELEVENLABS_STT_MODEL_ID	—	STT model, default scribe_v2_realtime
WEBHOOK_SECRET	—	If set, required as the Basic Auth password for POST /api/webhook/conversation; username can be anything
NEO4J_URI	—	Bolt URI override (auto-set inside Docker)
NEO4J_USER	—	Neo4j username, default neo4j
N8N_WEBHOOK_URL	N8N mode	N8N streaming webhook URL
SEARXNG_URL	—	SearXNG instance for built-in web search
DATABASE_PATH	—	SQLite path, default ./data/hestia.sqlite
TRUST_PROXY	—	Set to 1 behind an HTTPS reverse proxy
FRONTEND_ORIGIN	—	Dev frontend origin, default http://localhost:5173

agent.config.json

The agent harness activates when this file is present. Copy the example and customize:

cp agent.config.example.json agent.config.json

{
  "provider": {
    "type": "openai",
    "model": "gpt-5.4-mini"
  },
  "harness": {
    "approvals": true,
    "approvalTimeoutMs": 60000,
    "compactionEnabled": true,
    "contextMaxMessages": 40,
    "systemPromptLocked": true
  },
  "mcpServers": {
    "home-assistant": {
      "url": "http://ha-mcp:8086/mcp"
    },
    "graphiti": {
      "url": "http://graphiti:8000/mcp"
    }
  }
}

The core memory + Home Assistant policy is built in (locked by default). Per-install persona and tone go in data/SOUL.md, which is editable from the Agent Harness settings panel. Developers can opt out of the lock with harness.systemPromptLocked: false or HESTIA_SYSTEM_PROMPT_LOCKED=false, in which case systemPrompt from this file is used instead.

Supported providers:

type	Model examples	Notes
anthropic	claude-opus-4-7, claude-sonnet-4-6	Requires ANTHROPIC_API_KEY
openai	gpt-5.4, gpt-5.4-mini, gpt-5.4-nano	Requires OPENAI_API_KEY; set OPENAI_BASE_URL for local models

MCP servers support both stdio configs (command/args) and remote HTTP configs (url). ${VAR} references in env and headers blocks expand from the environment at startup.

All settings changed through the Agent Harness panel (provider, model, system prompt, context window, etc.) are written back to agent.config.json automatically and persist across container restarts.

---

Home Assistant Assist Integration

Hestia can serve as the AI backend for Home Assistant's Assist voice pipeline via the webhook-conversation custom integration. Voice commands spoken to an HA voice satellite (e.g. a Voice PE device) are routed through Hestia's full agent harness — including MCP tools, long-term memory, and entity awareness.

Setup

1. Install the Webhook Conversation integration in Home Assistant (HACS → Custom repositories → eulemitkeule/webhook-conversation)

2. Add a new Conversation agent in the integration and configure it:

   Field	Value
   Webhook URL	https://your-hestia-domain/api/webhook/conversation
   Authentication	Basic HTTP authentication
   Username	anything (e.g. hestia) — not validated
   Password	value of WEBHOOK_SECRET in your .env
   Output field name	output

3. Generate a secure webhook secret:

   openssl rand -base64 32
   

   Set it as WEBHOOK_SECRET in .env and restart the container.

4. In Home Assistant → Settings → Voice assistants, select your new Hestia conversation agent.

How it works

• Each HA voice conversation uses its own conversation_id, which maps to a dedicated thread in the HestiaClaw sidebar — browse past voice sessions just like regular chat sessions
• HA's list of exposed entities (with entity IDs, names, states, and areas) is injected into the agent's context each turn — Hestia knows your devices immediately without extra MCP lookups
• Hestia's own system prompt and memory system are used; HA's generic system prompt is ignored
• Write-tool approvals are bypassed for webhook calls (no browser is available during a voice interaction)
• The sidebar polls for new voice sessions every 30 seconds and sorts them to the top as they receive new messages

---

Memory System

Hestia uses a three-layer memory architecture so she actually remembers things:

1. Pinned facts (data/MEMORY.md) — the highest-confidence durable facts, injected into every turn. The agent can update this file directly (with your approval), and a nightly consolidation job regenerates it from the full Graphiti graph.

2. Active recall — before each reply, Hestia runs a fast semantic search against your Graphiti graph using your message as the query. The top results are silently injected into her context so relevant memories surface without her having to explicitly look them up.

3. Graphiti knowledge graph — all conversations write episodes to Graphiti. Entities and relationships accumulate over time and are visible in the 3D graph view. The nightly consolidation job prunes duplicates and promotes the best facts to the pinned layer.

To trigger consolidation manually: POST /api/agent/consolidate

The pinned memory file lives at data/MEMORY.md inside the hestia_data Docker volume. To inspect or edit it directly:

docker exec hestiaclaw-hestia-1 cat /app/data/MEMORY.md

---

Skills

Skills are pre-defined agent behaviors stored as SKILL.md files in the skills/ directory.

skills/
  morning-brief/
    SKILL.md
  lights-off/
    SKILL.md

SKILL.md format:

---
name: morning-brief
description: Deliver a morning briefing covering overnight activity and what's on for the day.
user-invocable: true
argument-hint: (optional hint shown in the palette)
disable-model-invocation: false
---

Your instructions for the agent go here. This becomes part of the system prompt
when this skill is active.

Invoking skills:

• Type / in the chat input to open the command palette — navigate with arrow keys, select with Enter or Tab
• Type /morning-brief and send to invoke directly
• Skills with disable-model-invocation: false are injected into the system prompt so the model can invoke them automatically when the context matches

Built-in slash commands:

Command	Description
/new-chat	Start a new conversation
/approvals	Toggle tool approval prompts on/off. Also accepts /approvals on or /approvals off. State is reflected immediately in the Agent Harness panel.

---

Knowledge Graph

Click KNOWLEDGE GRAPH in the header to open the full-screen 3D force-directed view. Requires Neo4j running with NEO4J_URI, NEO4J_USER, and NEO4J_PASSWORD set in .env.

• Nodes colored by Louvain community, sized by degree centrality
• Click a node to see its relationships in the info panel
• RECOMPUTE re-runs the full GDS pipeline (community detection + degree centrality) server-side

As Hestia converses and queries Home Assistant, Graphiti extracts entities and builds relationships in the graph automatically — it grows on its own.

---

Agent Harness

API routes

All routes require authentication except /api/auth/login and /api/webhook/conversation.

Method	Route	Description
POST	/api/auth/login	Authenticate and create session
POST	/api/auth/logout	Destroy session
GET	/api/auth/session	Current auth state
POST	/api/auth/change-password	Change username or password
POST	/api/agent/chat	Run agent loop, stream NDJSON
GET	/api/agent/config	Provider, model, tools, skills, MCP status, recent runs
GET	/api/agent/models	Live model list for a provider (cached 5 min)
POST	/api/agent/settings	Hot-swap provider / model / system prompt / reasoning effort
GET	/api/agent/tools	All registered tools with metadata
GET	/api/agent/runs	Recent traced agent runs
GET	/api/agent/approvals	Pending tool approval requests
POST	/api/agent/approvals/:id	Approve or deny a pending tool call
GET	/api/agent/conversations	List all server-side conversations with metadata
GET	/api/agent/conversations/:id/messages	Message history for a conversation (with tool badges)
POST	/api/agent/conversations/fork	Copy agent history into a new conversation
POST	/api/agent/consolidate	Trigger memory consolidation manually
POST	/api/webhook/conversation	HA Assist webhook endpoint (no session auth — see WEBHOOK_SECRET)
GET	/api/graph	Query Neo4j knowledge graph (nodes + edges)
POST	/api/graph/recompute	Re-run GDS Louvain + degree analytics
GET	/api/voice/voices	List ElevenLabs voices
POST	/api/voice/token	Single-use ElevenLabs WebSocket token
POST	/api/voice/transcribe	Fallback audio → text (ElevenLabs STT)
POST	/api/voice/speak	Text → audio (ElevenLabs TTS)

Context management

Long conversations use bounded context: recent messages are kept verbatim, older messages are summarized and injected into the effective system prompt. Configurable via harness.contextMaxMessages in agent.config.json or the Agent Harness panel at runtime.

---

N8N Mode

A legacy N8N streaming webhook integration is still available as a selectable backend. Toggle between N8N and Agent mode from the sidebar. Set N8N_WEBHOOK_URL in .env and configure your N8N node with Response Mode: Streaming.

---

Built With

• Frontend: React 19 + Vite 8, plain CSS, Orbitron & Rajdhani fonts
• Backend: Express 5, SQLite (better-sqlite3) — auth, sessions, agent history, audit log
• LLM: @anthropic-ai/sdk + openai SDK (Responses API)
• MCP: @modelcontextprotocol/sdk stdio + HTTP client
• Memory: Graphiti + Neo4j + GDS (via Docker)
• Rendering: react-markdown + remark-gfm + react-syntax-highlighter (Prism)
• 3D graph: three.js + 3d-force-graph
• Voice: ElevenLabs realtime STT WebSocket + TTS proxy
• Auth: Argon2 password hashing, express-session, helmet
• Scheduling: node-cron (daily memory consolidation)

---

Acknowledgments

Hestia stands on the shoulders of some excellent open-source projects:

• Graphiti by Zep AI — the temporal knowledge graph engine that powers Hestia's long-term memory
• ha-mcp by homeassistant-ai — the MCP server that gives Hestia her Home Assistant tools
• webhook-conversation by eulemitkeule — the HA custom integration that bridges Assist voice pipeline to external AI backends
• Home Assistant — the open-source smart home platform this whole thing is built around
• Neo4j — the graph database backing the knowledge graph visualization
• 3d-force-graph + three.js — the 3D graph rendering stack
• ElevenLabs — realtime voice STT and TTS
• Model Context Protocol — the open standard that makes MCP servers plug-and-play with any compliant agent