Alphaavatar

---

AlphaAvatar is a self-hostable Omni Personal Assistant framework designed to evolve into an intelligent personal butler — a continuous, personalized, and proactive assistant that can remember, understand, plan, and act on behalf of the user.

It is built around a plugin-based real-time Agent architecture, combining:

• 🧠 Memory for long-term user, assistant, and tool interaction history
• 🧬 Persona for user understanding, identity continuity, and personalization
• 💡 Reflection for self-improvement and long-term behavioral adaptation
• 📅 Planning for task decomposition, reminders, and future-oriented actions
• ⚙️ Behavior for response style, workflow policy, and proactive assistance
• 🧰 Tools through MCP, RAG, DeepResearch, and external integrations
• 😊 Virtual Character for real-time voice/avatar interaction

✨ Fully self-hostable and privacy-first — AlphaAvatar can run locally or on your own infrastructure, giving you control over your data, memory, tools, and behavior.

---

• 📊 Track and analyze personal metrics such as health, fitness, sleep, and study progress
• 📈 Provide long-term insights and trend analysis
• 🎯 Suggest improvements based on historical patterns

• 📖 Organize personal notes, documents, and knowledge
• 🔍 Retrieve relevant information through RAG
• 🧠 Build a personal knowledge base over time

• 📅 Schedule tasks and reminders
• ⏰ Proactively notify based on context and priority
• 🔄 Break down long-term goals into actionable steps

• 🧠 Plan multi-step workflows such as learning plans, projects, and research
• 🔧 Call tools automatically to complete tasks
• 📌 Maintain consistency across long time horizons

• 🧬 Understand user preferences, habits, and personality
• 💬 Provide highly personalized responses
• 🤝 Maintain continuity across conversations and modalities

• 🌐 Search, research, and summarize real-world information
• 🧰 Integrate with tools such as email, databases, APIs, and messaging apps
• 🔗 Act as a bridge between user intent and external systems

💡 AlphaAvatar is not just a chatbot. It is a foundation for building stateful, proactive, multimodal, and self-evolving personal AI assistants.

---

---

---

• [2026/05] Released AlphaAvatar version 0.5.4:

  • Added LanceDB-backed MCP tool retrieval, enabling AlphaAvatar to semantically search relevant MCP tools based on the Agent query.
  • Refactored system prompt and runtime prompt composition to improve prefix-cache efficiency and support dynamic runtime context injection.
  • Improved the Persona plugin with deterministic runtime state tracking, including timezone, login time, session ID, room type, and login count.
  • Added temporary-user to real-user identity merging, including user workspace migration and identity-aware plugin storage.
  • Improved RAG runtime behavior with initialization waiting, current/temporary workspace querying, and LLM-friendly query output.

• [2026/04] Released AlphaAvatar version 0.5.3:

  • Added localized Markdown backup for the Memory plugin.
  • Added LanceDB as the default local VDB option when Qdrant credentials are not provided.

• [2026/03] Released AlphaAvatar version 0.5.0:

  • Added the MCP plugin, enabling retrieval and concurrent invocation of MCP tools.
  • Released AlphaAvatar version 0.5.1: Added WhatsApp channel support via Baileys.
  • Released AlphaAvatar version 0.5.2: Added the AlphaAvatar Voice plugin with Voice.ai TTS support.

• [2026/02] Released AlphaAvatar version 0.4.0:

  • Added RAG support through RAG-Anything.
  • Optimized the Memory and DeepResearch modules.
  • Released AlphaAvatar version 0.4.1: Fixed Persona plugin bugs and added a new MCP plugin.

• [2026/01] Released AlphaAvatar version 0.3.0:

  • Added DeepResearch support through the Tavily API.
  • Released AlphaAvatar version 0.3.1: Added tool-call memory extraction during user–assistant interactions.

• [2025/12] Released AlphaAvatar version 0.2.0:

  • Added AIRI Live2D-based virtual character display.

• [2025/11] Released AlphaAvatar version 0.1.0:

  • Added automatic memory extraction.
  • Added automatic user persona extraction and matching.

Install stable AlphaAvatar version from PyPI:

uv venv .my-env --python 3.11
source .my-env/bin/activate
pip install alpha-avatar-agents

Install latest AlphaAvatar version from GitHub:

git clone --recurse-submodules https://github.com/AlphaAvatar/AlphaAvatar.git
cd AlphaAvatar

uv venv .venv --python 3.11
source .venv/bin/activate
uv sync --all-packages

Start your agent in dev mode to connect it to LiveKit and make it available from anywhere on the internet.

---

🧩 Step 1. Configure Environment Variables

cd AlphaAvatar

# Copy template
cp .env.template .env.dev

Edit .env.dev and set required environment variables.

📦 Step 2. Download Required Files

alphaavatar download-files

✅ Step 3. Run the Agent

ENV_FILE=.env.dev alphaavatar dev examples/agent_configs/pipeline_openai_airi.yaml
# or
ENV_FILE=.env.dev alphaavatar dev examples/agent_configs/pipeline_openai_tools.yaml

To see more supported modes, please refer to the LiveKit doc.

To see more examples, please refer to the Examples README

AlphaAvatar supports multiple Access Channels, allowing different types of users — from end users to developers — to interact with the system.

---

🧠 Runtime Architecture

AlphaAvatar is built around a real-time multimodal Agent runtime. It separates streaming interaction, model reasoning, runtime context, plugin state, tool execution, and channel adapters.

                         AlphaAvatar Runtime
                         ───────────────────

        ┌──────────────────────────────────────────────┐
        │                 LiveKit Room                 │
        │        audio / text / video / data streams   │
        └──────────────────────┬───────────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │                 AgentSession                 │
        │      real-time turn detection / streaming IO │
        └──────────────────────┬───────────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │                 AvatarEngine                 │
        │  Realtime Model Pipeline                     │
        │  STT / VAD / LLM / TTS / video-ready input   │
        └──────────────────────┬───────────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │          Prompt & Runtime Context Layer       │
        │  system prompt / runtime prompt / time / mode │
        │  memory / persona / plan / reflection / rules │
        └──────────────────────┬───────────────────────┘
                               │
        ┌──────────────────────┼───────────────────────┐
        │                      │                       │
        ▼                      ▼                       ▼
┌──────────────────┐  ┌──────────────────┐   ┌──────────────────┐
│  Memory Plugin   │  │  Persona Plugin  │   │   Tools Plugins  │
│ conversation     │  │ user profile     │   │ MCP / RAG        │
│ tool memory      │  │ speaker / face   │   │ DeepResearch     │
│ avatar memory    │  │ runtime state    │   │ external APIs    │
└──────────────────┘  └──────────────────┘   └──────────────────┘

┌──────────────────┐  ┌──────────────────┐   ┌──────────────────┐
│ Reflection Plugin│  │ Planning Plugin  │   │ Behavior Plugin │
│ self-analysis    │  │ tasks / plans    │   │ response policy │
│ adaptation       │  │ reminders        │   │ workflow rules  │
└──────────────────┘  └──────────────────┘   └──────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │              UserPath Workspace              │
        │  local data / cache / logs / memory / RAG    │
        │  persona state / artifacts / user files      │
        └──────────────────────┬───────────────────────┘
                               │
        ┌──────────────────────┴───────────────────────┐
        │                                              │
        ▼                                              ▼
┌──────────────────────┐                    ┌──────────────────────┐
│ Native Web / App     │                    │ Channel Adapters     │
│ voice / text / video │                    │ WhatsApp / future    │
│ WebRTC / UI          │                    │ messaging platforms  │
└──────────────────────┘                    └──────────────────────┘

---

AlphaAvatar now provides a browser-based realtime demo interface built on LiveKit.

👉 Try the Web Demo: https://www.alphaavatar.ai/demo

The Web Demo supports:

• 🎙️ Real-time voice interaction
• 💬 Text chat with the Avatar
• 📷 Camera preview and video-ready interaction
• 🔊 Agent audio playback
• 😊 Virtual character / avatar stage
• 🧠 Full plugin support, including Memory, Persona, RAG, MCP, and DeepResearch
• 🌍 Browser timezone metadata, enabling AlphaAvatar to understand local login time

The Web Demo is the recommended way to try AlphaAvatar with a full realtime multimodal experience.

---

Interact with AlphaAvatar directly inside messaging platforms.

Capabilities:

• 💬 Text-based conversation
• 🎤 Voice message interaction
• 🧰 Tool invocation via chat interface

---

📦 Channel introduction: README

▶️ Start WhatsApp Channel

Make sure AlphaAvatar Agent is already running (see Quick Start above).

ENV_FILE=.env.dev sh examples/channels/start_whatsapp.sh

💡 The WhatsApp channel runs as an independent bridge process and connects to the Agent runtime.

---

A dedicated AlphaAvatar mobile application providing:

• 🎙️ Real-time voice communication
• 😊 Live2D / Virtual character visualization
• 🧠 Persistent memory & persona

---

Developers can immediately access AlphaAvatar via the LiveKit Playground.

👉 https://agents-playground.livekit.io/

After starting your AlphaAvatar server:

1. Connect to your LiveKit instance
2. Configure the Agent name in the Playground (must match avatar_name, default: Assistant) to enable Explicit Dispatch.
3. Connect to the agent room
4. Start testing real-time interaction

Supported capabilities:

• 🎙️ Voice interaction
• 🧠 Memory extraction
• 🔍 RAG retrieval
• 🧰 MCP tool invocation
• 😊 Virtual character display

---

💡 AlphaAvatar is currently developer-first, with a Web Demo available for realtime interaction.

More user-facing web and mobile experiences are under active development.