c4 GenAI Suite

An AI chatbot application with Model Context Provider (MCP) integration, powered by ai-sdk and compatibility for all major Large Language Models (LLMs) and Embedding Models.

Administrators can create assistants with different capabilities by adding extensions, such as RAG (Retrieval-Augmented Generation) services or MCP servers. The application is built using a modern tech stack, including React, NestJS, and Python FastAPI for the REI-S service.

Users can interact with assistants through a user-friendly interface. Depending on the assistant's configuration, users may be able to ask questions, upload their own files, or use other features. The assistants interact with various LLM providers to provide responses based on the configured extensions. Contextual information provided by the configured extensions allows the assistants to answer domain-specific questions and provide relevant information.

The application is designed to be modular and extensible, allowing users to create assistants with different capabilities by adding extensions.

Features

Large Language Models (LLM) and Multimodal Models

The c4 GenAI Suite supports already many models directly. And if your preferred model is not supported already, it should be easy to write an extension to support it.

• OpenAI compatible models
• Azure OpenAI models
• Bedrock models
• Google GenAI models
• Ollama compatible models

Retrieval Augmented Generation (RAG)

The c4 GenAI Suite includes REI-S, a server to prepare files for consumption by the LLM.

• REI-S, a custom integrated RAG server
  • Vector stores
    • pgvector
    • Azure AI Search
  • Embedding models
    • OpenAI compatible embeddings
    • Azure OpenAI embeddings
    • Ollama compatible embeddings
  • File formats:
    • pdf, docx, pptx, xlsx, ...
    • audio file voice transcription (via Whisper)

Extensions

The c4 GenAI Suite is designed for extensibility. Writing extensions is easy, as is using an already existing MCP server.

• Model Context Protocol (MCP) servers
• Custom systemprompt
• Brave Search and Grounding with Bing Search
• Calculator

Getting Started

Using Docker-Compose

• Run docker compose up in the project root.
• Open the application in a browser. The default login credentials are user admin@example.com and password secret.

Using Helm & Kubernetes

For deployment in Kubernetes environments, please refer to the README of our Helm Chart.

Setting up Assistants and Extensions

The c4 GenAI Suite revolves around assistants. Each assistant consists of a set of extensions, which determine the LLM model and which tools it can use.

• In the admin area (click the username on the bottom left), go to the assistants section.
• Add an assistant with the green + button next to the section title. Choose a name and a description.
• Select the created assistant and click the green + Add Extension.
• Select the model and fill in the credentials.
• Use the Test Button to check that it works and save.

Now you can return to the chat page (click on c4 GenAI Suite in the top left) and start a new conversation with your new assistant.

[!TIP] Our docker-compose includes a local Ollama, which runs on the CPU. You can use this for quick testing. But it will be slow and you probably want to use another model. If you want to use it, just create the following model extension in your Assistant.

• Extension: Ollama
• Endpoint: http://ollama:11434
• Model: llama3.2

Model Context Protocol (MCP) [optional]

Use any MCP server offering an sse interface with the MCP Tools Extension (or use our mcp-tool-as-server as a proxy in front of an stdio MCP server). Each MCP server can be configured in detail as an extension.

Retrieval Augmented Generation (RAG) / File Search [optional]

Use our RAG server REI-S to search user provided files. Just configure a Search Files extension for the assistant. This process is described in detail in the services/reis subdirectory.

Contributing & Development

• See CONTRIBUTING.md for guidelines on how to contribute.
• For developer onboarding, check DEVELOPERS.md.

Main Building Blocks

The application consists of a Frontend , a Backend and a REI-S service.

┌──────────┐
│   User   │
└─────┬────┘
      │ access
      ▼
┌──────────┐
│ Frontend │
└─────┬────┘
      │ access
      ▼
┌──────────┐     ┌─────────────────┐
│ Backend  │────►│      LLM        │
└─────┬────┘     └─────────────────┘
      │ access
      ▼
┌──────────┐     ┌─────────────────┐
│  REI-S   │────►│ Embedding Model │
│          │     └─────────────────┘
│          │
│          │     ┌─────────────────┐
│          │────►│  Vector Store   │
└──────────┘     └─────────────────┘

Frontend

The frontend is built with React and TypeScript, providing a user-friendly interface for interacting with the backend and REI-S service. It includes features for managing assistants, extensions, and chat functionalities.

Sources: /frontend

Backend

The backend is developed using NestJS and TypeScript, serving as the main API layer for the application. It handles requests from the frontend and interacts with llm providers to facilitate chat functionalities. The backend also manages assistants and their extensions, allowing users to configure and use various AI models for their chats.

Additionally, the backend manages user authentication, and communicates with the REI-S service for file indexing and retrieval.

For data persistence, the backend uses a PostgreSQL database.

Sources: /backend

REI-S

The REI-S (Retrieval Extraction Ingestion Server) is a Python-based server that provides basic RAG (Retrieval-Augmented Generation) capabilities. It allows for file content extraction, indexing and querying, enabling the application to handle large datasets efficiently. The REI-S service is designed to work seamlessly with the backend, providing necessary data for chat functionalities and file searches.

The REI-S supports Azure AI Search and pgvector for vector storage, allowing for flexible and scalable data retrieval options. The service can be configured using environment variables to specify the type of vector store and connection details.

Sources: /services/reis

Eval Service

The Eval service provides LLM evaluation capabilities for measuring and comparing assistant quality. It supports QA catalog management, automated evaluation execution, and a reporting dashboard.

The service is disabled by default. To enable it, set the following in backend/.env:

EVAL_SERVICE_ENABLED=true
EVAL_SERVICE_URL=http://localhost:3202

For full setup instructions, see the Eval service README.

Sources: /services/eval