F.R.I.D.A.Y. — Tony Stark Demo

"Fully Responsive Intelligent Digital Assistant for You"

A Tony Stark-inspired AI assistant split into two cooperating pieces:

Demo: Instagram reel

How it works

Microphone ──► STT (Sarvam Saaras v3)
                    │
                    ▼
             LLM (Gemini 2.5 Flash)  ◄──────► MCP Server (FastMCP / SSE)
                    │                              ├─ get_world_news
                    ▼                              ├─ open_world_monitor
             TTS (OpenAI nova)                     ├─ search_web
                    │                              └─ …more tools
                    ▼
             Speaker / LiveKit room

The voice agent connects to the MCP server via SSE at http://127.0.0.1:8000/sse (auto-resolved to the Windows host IP when running inside WSL).

Project structure

friday-tony-stark-demo/
├── server.py           # uv run friday  → starts the MCP server (SSE on :8000)
├── agent_friday.py     # uv run friday_voice → starts the LiveKit voice agent
├── pyproject.toml
├── .env.example        # copy → .env and fill in your keys
│
└── friday/             # MCP server package
    ├── config.py       # env-var loading & app-wide settings
    ├── tools/          # MCP tools (callable by the LLM)
    │   ├── web.py      # search_web, fetch_url, get_world_news, open_world_monitor
    │   ├── system.py   # get_current_time, get_system_info
    │   └── utils.py    # format_json, word_count
    ├── prompts/        # MCP prompt templates (summarize, explain_code, …)
    └── resources/      # MCP resources exposed to clients (friday://info)

Quick start

1. Prerequisites

• Python ≥ 3.11
• uv — pip install uv or curl -Lsf https://astral.sh/uv/install.sh | sh
• A LiveKit Cloud project (free tier works)

2. Clone & install

git clone https://github.com/SAGAR-TAMANG/friday-tony-stark-demo.git
cd friday-tony-stark-demo
uv sync          # creates .venv and installs all dependencies

3. Set up environment

cp .env.example .env
# Open .env and fill in your API keys (see the section below)

4. Run — two terminals

Terminal 1 — MCP server (must start first)

uv run friday

Starts the FastMCP server on http://127.0.0.1:8000/sse. The voice agent connects here to fetch its tools.

Terminal 2 — Voice agent

uv run friday_voice

Starts the LiveKit voice agent in dev mode — it joins a LiveKit room and begins listening. Open the LiveKit Agents Playground and connect to your room to talk to FRIDAY.

uv run friday vs uv run friday_voice

Both processes must run simultaneously. The voice agent calls the MCP server in real time whenever it needs a tool (e.g. fetching news).

Environment variables

Copy .env.example → .env and fill in the values below.

Switching providers

Open agent_friday.py and change the provider constants at the top:

STT_PROVIDER = "sarvam"   # "sarvam" | "whisper"
LLM_PROVIDER = "gemini"   # "gemini" | "openai"
TTS_PROVIDER = "openai"   # "openai" | "sarvam"

Adding a new tool

1. Create or open a file in friday/tools/
2. Define a register(mcp) function and decorate tools with @mcp.tool()
3. Import and call register(mcp) inside friday/tools/__init__.py

The MCP server will pick it up on next start.

Tech stack

• FastMCP — MCP server framework
• LiveKit Agents — real-time voice pipeline
• Sarvam Saaras v3 — STT (Indian-English optimised)
• Google Gemini 2.5 Flash — LLM
• OpenAI TTS (nova voice) — TTS
• uv — fast Python package manager

License

MIT