🤖 FastAPI Langgraph Chatbot with Vector Store, Memory, MCP tools and Voice mode

A scalable AI chatbot platform built with FastAPI and LangGraph, featuring multi-agent orchestration, multi-tenant vector storage, cross-chat memory, and voice call capabilities through LiveKit integration.

Demo

https://github.com/user-attachments/assets/25adf403-6c66-4894-9f0e-27367787273c

🌟 Key Features

🔒 Secure Authentication & Multi-Tenancy

• OAuth2 Password Flow: Industry-standard authentication with JWT tokens for secure API access
• User Registration System: Complete registration flow with username, password, and tenant_id
• Multi-Tenant Architecture: Data isolation between different tenants for enhanced security and privacy
• Role-Based Access Control: Granular permissions for different user types

🧠 Advanced AI Orchestration with LangGraph

• Multi-Agent System: Specialized agents working together to solve complex tasks
  • Supervisor Agent: Coordinates workflow and delegates tasks to specialized agents
  • Research Agent: Retrieves information from the web and knowledge bases
  • Scraper Agent: Extracts and summarizes content from web pages
• Complex Workflows: Handle multistep reasoning and task decomposition
• State Management: Maintain conversation context across multiple turns

📊 Powerful Vector Storage with Qdrant

• Multi-Tenant Vector Store: Efficiently store and retrieve conversation history with tenant isolation
• Semantic Search: Find relevant past conversations using semantic similarity
• Payload Filtering: Efficient filtering by tenant_id for data security and performance
• Metadata Storage: Store and retrieve additional context alongside vector embeddings

💾 Persistent Memory with Mem0

• Cross-Chat Memory: Remember important information across different conversations
• Long-Term Context: Maintain context over extended interactions
• Selective Memory: Intelligently decide what information to remember
• Memory Retrieval: Retrieve relevant memories based on conversation context

🔍 MCP Tools Integration

• Firecrawl: Advanced web scraping and content extraction
  • Extract structured data from websites
  • Summarize long-form content
  • Process tables and lists
• Tavily: Intelligent web search capabilities
  • Semantic search across the web
  • Real-time information retrieval
  • Source attribution and citation

🎤 Voice Assistant via LiveKit

• Real-Time Voice Communication: Natural voice interaction using LiveKit's WebRTC platform
• High-Quality Speech Recognition: Accurate transcription with Deepgram's advanced STT
• Natural Text-to-Speech: Lifelike voice responses with Cartesia TTS
• Voice Activity Detection: Intelligent turn-taking with Silero VAD
• Multilingual Support: Voice interaction in multiple languages

🔎 LangSmith Integration

• Comprehensive Tracing: Capture detailed traces of all LangChain and LangGraph executions
• Visual Debugging: Visualize the multi-agent workflow and message passing
• Performance Monitoring: Track latency, token usage, and costs across different components
• Feedback Collection: Gather and analyze user feedback on model responses
• Experiment Tracking: Compare different prompts, models, and agent configurations

🎨 Modern React UI

• Clean Material-UI Design: Intuitive interface with modern design principles
• Responsive Layout: Seamless experience across desktop and mobile devices
• Real-Time Chat Interface: Dynamic message bubbles with user/assistant avatars
• Markdown Support: Rich text formatting in messages
• Voice Mode Integration: Seamless switching between text and voice interaction

Backend Architecture

• FastAPI Framework: High-performance asynchronous API framework with automatic OpenAPI documentation
• SQLAlchemy ORM: Async database operations with SQLite (configurable for PostgreSQL/MySQL)
• Pydantic Models: Type-safe data validation and serialization
• JWT Authentication: Secure token-based authentication with OAuth2 password flow

👨‍💻 Usage Guide

Getting Started

1. Register an Account:

   • Navigate to the registration page at /register
   • Create an account with a unique username, secure password, and your assigned tenant_id
   • Each tenant_id creates an isolated environment for your data

2. Login to the Platform:

   • Use your credentials to log in at the main page
   • Your JWT token will be stored securely for later API calls

Using the Chat Interface

1. Creating Conversations:

   • Click the "+" button in the sidebar to start a new conversation
   • Give your conversation a meaningful title for easy reference
   • Each conversation is stored with your tenant_id for data isolation

2. Interacting with the AI:

   • Type your message in the input field and press Enter or click the send button
   • The AI response will stream in real-time with token-by-token updates
   • Messages are displayed with clear user/assistant avatars for easy distinction
   • The system supports markdown formatting in both user and assistant messages

3. Managing Conversations:

   • Access previous conversations from the sidebar navigation
   • Search through your conversation history

Using the Voice Assistant

1. Launching Voice Mode:

   • Click the microphone icon in the chat interface to activate voice mode
   • The system will request microphone permissions if not already granted
   • Wait for the "Voice Assistant Ready" message before speaking

2. Voice Interaction:

   • Speak naturally when you see the "Listening..." indicator
   • The assistant will process your speech and respond with voice
   • Visual feedback shows the transcription of both your speech and the assistant's response
   • The voice assistant supports natural turn-taking in conversation

3. Advanced Voice Features:

   • The system automatically detects when you've finished speaking
   • Voice activity detection filters out background noise
   • Multilingual support allows interaction in different languages
   • Voice quality settings can be adjusted for different environments

4. Ending a Voice Session:

   • Click the disconnect button to end the voice session
   • Your conversation history is preserved for future reference

Using External Tools

The AI assistant can leverage powerful external tools to enhance your experience:

1. Web Search: Ask questions that require up-to-date information from the web
2. Web Scraping: Request summaries or information from specific websites

💻 Installation

Prerequisites

• Python 3.13+
• Node.js 16+
• npm 8+
• Qdrant (can be run via Docker)

Backend Setup

1. Clone the repository

git clone <repository-url>
cd ruban_fastapi-langgraph-chatbot

2. Create and activate a virtual environment

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install Python dependencies

pip install -r requirements.txt

4. Set up environment variables

Create a .env file in the root directory with the following variables:

# FastAPI settings
SECRET_KEY=your_secret_key_here
ACCESS_TOKEN_EXPIRE_MINUTES=30

# Database settings
SQLALCHEMY_DATABASE_URI=sqlite:///./app.db

# OpenAI settings
OPENAI_API_KEY=your_openai_api_key_here

# Qdrant settings
QDRANT_HOST=localhost
QDRANT_PORT=6333

# LiveKit settings
LIVEKIT_URL=your_livekit_url_here
LIVEKIT_API_KEY=your_livekit_api_key_here
LIVEKIT_API_SECRET=your_livekit_api_secret_here

# LangSmith (optional, for tracing)
LANGCHAIN_TRACING_V2=true
LANGSMITH_API_KEY=your_langsmith_api_key_here
LANGSMITH_PROJECT=your_project_name

Frontend Setup

1. Navigate to the frontend directory

cd frontend

2. Install Node.js dependencies

npm install

🚀 Running the Application

Starting the Backend Services

1. Start the MCP servers (in separate terminals)

# Terminal 1: Start the search server
python -m app.mcp_server.search_server

# Terminal 2: Start the web scraping server
python -m app.mcp_server.web_scrapping_server

2. Start the main FastAPI server

python app.py

3. Start the LiveKit voice assistant

python app/agent/livekit_agent.py dev

Starting the Frontend

cd frontend
npm start

Accessing the Application

• Backend API documentation: http://localhost:8000/docs
• Frontend interface: http://localhost:3000

If you don't have an account, register first and then log in to access the chat interface.