Alphaavatar

---

AlphaAvatar Introduction

AlphaAvatar is a self-hostable Omni Personal Assistant framework designed to evolve into an intelligent personal butler — a continuous, personalized, and proactive assistant that can remember, understand, plan, and act on behalf of the user.

It is built around a plugin-based real-time Agent architecture, combining:

• 🧠 Memory for long-term user, assistant, and tool interaction history
• 🧬 Persona for user understanding, identity continuity, and personalization
• 💡 Reflection for self-improvement and long-term behavioral adaptation
• 📅 Planning for task decomposition, reminders, and future-oriented actions
• ⚙️ Behavior for response style, workflow policy, and proactive assistance
• 🧰 Tools through MCP, RAG, DeepResearch, and external integrations
• 😊 Virtual Character for real-time voice/avatar interaction

✨ Fully self-hostable and privacy-first — AlphaAvatar can run locally or on your own infrastructure, giving you control over your data, memory, tools, and behavior.

---

What AlphaAvatar Is Designed For

1️⃣ Personal Data & Life Metrics Management 📊 Track and analyze personal metrics such as health, fitness, sleep, and study progress 📈 Provide long-term insights and trend analysis 🎯 Suggest improvements based on historical patterns	2️⃣ Knowledge & Notes Management 📖 Organize personal notes, documents, and knowledge 🔍 Retrieve relevant information through RAG 🧠 Build a personal knowledge base over time
3️⃣ Task & Event Management 📅 Schedule tasks and reminders ⏰ Proactively notify based on context and priority 🔄 Break down long-term goals into actionable steps	4️⃣ Autonomous Planning & Execution 🧠 Plan multi-step workflows such as learning plans, projects, and research 🔧 Call tools automatically to complete tasks 📌 Maintain consistency across long time horizons
5️⃣ Personalized Companion & Context Awareness 🧬 Understand user preferences, habits, and personality 💬 Provide highly personalized responses 🤝 Maintain continuity across conversations and modalities	6️⃣ External World Interaction 🌐 Search, research, and summarize real-world information 🧰 Integrate with tools such as email, databases, APIs, and messaging apps 🔗 Act as a bridge between user intent and external systems

💡 AlphaAvatar is not just a chatbot. It is a foundation for building stateful, proactive, multimodal, and self-evolving personal AI assistants.

---

AlphaAvatar Plugins

🧠 Memory Self-improving memory module for Omni-Avatar. README↗	🧬 Persona Automatic extraction and real-time matching of multimodal user persona. README↗
💡 Reflection A self-improvement module that reflects on memory, behavior, and interaction history. README↗	📅 Planning Long-horizon planning module for tasks, reminders, goals, and multi-step workflows. README↗
🤖 Behavior Controls response style, workflow policy, tool-use behavior, and proactive assistance rules. README↗	😊 Virtual Character The real-time generated virtual character that visually represents the Avatar during interactions. README↗

---

Tools Plugins

🔍 DeepResearch Allow AlphaAvatar to access the network and perform single-step/multi-step inference through a separate Agent service to search for more accurate content. README↗	📖 RAG Allow AlphaAvatar to access Documents/Skills (user-uploaded/generated by the Reflection module/URL access) to obtain document-related information. README↗
🧰 MCP Allows AlphaAvatar to discover and call real-world external tools such as databases, email, calendars, APIs, and productivity apps. README↗	🌍 SANDBOX Provide AlphaAvatar with a sandbox environment to interact with the external world or with other agents, thereby enabling multi-agent interaction and exploration. README↗

---

Docs and guides

Latest News 🔥

• [2026/05] Released AlphaAvatar version 0.5.4:

  • Added LanceDB-backed MCP tool retrieval, enabling AlphaAvatar to semantically search relevant MCP tools based on the Agent query.
  • Refactored system prompt and runtime prompt composition to improve prefix-cache efficiency and support dynamic runtime context injection.
  • Improved the Persona plugin with deterministic runtime state tracking, including timezone, login time, session ID, room type, and login count.
  • Added temporary-user to real-user identity merging, including user workspace migration and identity-aware plugin storage.
  • Improved RAG runtime behavior with initialization waiting, current/temporary workspace querying, and LLM-friendly query output.

• [2026/04] Released AlphaAvatar version 0.5.3:

  • Added localized Markdown backup for the Memory plugin.
  • Added LanceDB as the default local VDB option when Qdrant credentials are not provided.

• [2026/03] Released AlphaAvatar version 0.5.0:

  • Added the MCP plugin, enabling retrieval and concurrent invocation of MCP tools.
  • Released AlphaAvatar version 0.5.1: Added WhatsApp channel support via Baileys.
  • Released AlphaAvatar version 0.5.2: Added the AlphaAvatar Voice plugin with Voice.ai TTS support.

• [2026/02] Released AlphaAvatar version 0.4.0:

  • Added RAG support through RAG-Anything.
  • Optimized the Memory and DeepResearch modules.
  • Released AlphaAvatar version 0.4.1: Fixed Persona plugin bugs and added a new MCP plugin.

• [2026/01] Released AlphaAvatar version 0.3.0:

  • Added DeepResearch support through the Tavily API.
  • Released AlphaAvatar version 0.3.1: Added tool-call memory extraction during user–assistant interactions.

2025 Release History

• [2025/12] Released AlphaAvatar version 0.2.0:

  • Added AIRI Live2D-based virtual character display.

• [2025/11] Released AlphaAvatar version 0.1.0:

  • Added automatic memory extraction.
  • Added automatic user persona extraction and matching.

Installation ⚙️

Install stable AlphaAvatar version from PyPI:

uv venv .my-env --python 3.11
source .my-env/bin/activate
pip install alpha-avatar-agents

Install latest AlphaAvatar version from GitHub:

git clone --recurse-submodules https://github.com/AlphaAvatar/AlphaAvatar.git
cd AlphaAvatar

uv venv .venv --python 3.11
source .venv/bin/activate
uv sync --all-packages

Quick Start ⚡️

Start your agent in dev mode to connect it to LiveKit and make it available from anywhere on the internet.

---

🧩 Step 1. Configure Environment Variables

cd AlphaAvatar

# Copy template
cp .env.template .env.dev

Edit .env.dev and set required environment variables.

📦 Step 2. Download Required Files

alphaavatar download-files

✅ Step 3. Run the Agent

ENV_FILE=.env.dev alphaavatar dev examples/agent_configs/pipeline_openai_airi.yaml
# or
ENV_FILE=.env.dev alphaavatar dev examples/agent_configs/pipeline_openai_tools.yaml

To see more supported modes, please refer to the LiveKit doc.

To see more examples, please refer to the Examples README

Usage 🚀

AlphaAvatar supports multiple Access Channels, allowing different types of users — from end users to developers — to interact with the system.

---

🧠 Runtime Architecture

AlphaAvatar is built around a real-time multimodal Agent runtime. It separates streaming interaction, model reasoning, runtime context, plugin state, tool execution, and channel adapters.

                         AlphaAvatar Runtime
                         ───────────────────

        ┌──────────────────────────────────────────────┐
        │                 LiveKit Room                 │
        │        audio / text / video / data streams   │
        └──────────────────────┬───────────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │                 AgentSession                 │
        │      real-time turn detection / streaming IO │
        └──────────────────────┬───────────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │                 AvatarEngine                 │
        │  Realtime Model Pipeline                     │
        │  STT / VAD / LLM / TTS / video-ready input   │
        └──────────────────────┬───────────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │          Prompt & Runtime Context Layer       │
        │  system prompt / runtime prompt / time / mode │
        │  memory / persona / plan / reflection / rules │
        └──────────────────────┬───────────────────────┘
                               │
        ┌──────────────────────┼───────────────────────┐
        │                      │                       │
        ▼                      ▼                       ▼
┌──────────────────┐  ┌──────────────────┐   ┌──────────────────┐
│  Memory Plugin   │  │  Persona Plugin  │   │   Tools Plugins  │
│ conversation     │  │ user profile     │   │ MCP / RAG        │
│ tool memory      │  │ speaker / face   │   │ DeepResearch     │
│ avatar memory    │  │ runtime state    │   │ external APIs    │
└──────────────────┘  └──────────────────┘   └──────────────────┘

┌──────────────────┐  ┌──────────────────┐   ┌──────────────────┐
│ Reflection Plugin│  │ Planning Plugin  │   │ Behavior Plugin │
│ self-analysis    │  │ tasks / plans    │   │ response policy │
│ adaptation       │  │ reminders        │   │ workflow rules  │
└──────────────────┘  └──────────────────┘   └──────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │              UserPath Workspace              │
        │  local data / cache / logs / memory / RAG    │
        │  persona state / artifacts / user files      │
        └──────────────────────┬───────────────────────┘
                               │
        ┌──────────────────────┴───────────────────────┐
        │                                              │
        ▼                                              ▼
┌──────────────────────┐                    ┌──────────────────────┐
│ Native Web / App     │                    │ Channel Adapters     │
│ voice / text / video │                    │ WhatsApp / future    │
│ WebRTC / UI          │                    │ messaging platforms  │
└──────────────────────┘                    └──────────────────────┘

---

🌐 Web Access

AlphaAvatar now provides a browser-based realtime demo interface built on LiveKit.

👉 Try the Web Demo: https://www.alphaavatar.ai/demo

The Web Demo supports:

• 🎙️ Real-time voice interaction
• 💬 Text chat with the Avatar
• 📷 Camera preview and video-ready interaction
• 🔊 Agent audio playback
• 😊 Virtual character / avatar stage
• 🧠 Full plugin support, including Memory, Persona, RAG, MCP, and DeepResearch
• 🌍 Browser timezone metadata, enabling AlphaAvatar to understand local login time

The Web Demo is the recommended way to try AlphaAvatar with a full realtime multimodal experience.

---

💬 Social & Messaging Platforms

Interact with AlphaAvatar directly inside messaging platforms.

Capabilities:

• 💬 Text-based conversation
• 🎤 Voice message interaction
• 🧰 Tool invocation via chat interface

---

WhatsApp

📦 Channel introduction: README

▶️ Start WhatsApp Channel

Make sure AlphaAvatar Agent is already running (see Quick Start above).

ENV_FILE=.env.dev sh examples/channels/start_whatsapp.sh

💡 The WhatsApp channel runs as an independent bridge process and connects to the Agent runtime.

WeChat

Slack

---

📲 Native Mobile App

A dedicated AlphaAvatar mobile application providing:

• 🎙️ Real-time voice communication
• 😊 Live2D / Virtual character visualization
• 🧠 Persistent memory & persona

---

🧪 Developer Playground

Developers can immediately access AlphaAvatar via the LiveKit Playground.

👉 https://agents-playground.livekit.io/

After starting your AlphaAvatar server:

1. Connect to your LiveKit instance
2. Configure the Agent name in the Playground (must match avatar_name, default: Assistant) to enable Explicit Dispatch.
3. Connect to the agent room
4. Start testing real-time interaction

Supported capabilities:

• 🎙️ Voice interaction
• 🧠 Memory extraction
• 🔍 RAG retrieval
• 🧰 MCP tool invocation
• 😊 Virtual character display

---

💡 AlphaAvatar is currently developer-first, with a Web Demo available for realtime interaction.

More user-facing web and mobile experiences are under active development.