Awesome Open Source AI

A curated list of battle-tested, production-proven open-source AI models, libraries, infrastructure, and developer tools. Only elite-tier projects make this list. Updated May 8, 2026. CI verified - auto-fixed.

by Boring Dystopia Development

---

🌱 Emerging • Explore the List • Submission Guidelines • License

📋 Contents

• 🧬 1. Core Frameworks & Libraries
• 🧠 2. Open Foundation Models
• ⚡ 3. Inference Engines & Serving
• 🤖 4. Agentic AI & Multi-Agent Systems
• 🔍 5. Retrieval-Augmented Generation (RAG) & Knowledge
• 🎨 6. Generative Media Tools
• 🛠️ 7. Training & Fine-tuning Ecosystem
• 📊 8. MLOps / LLMOps & Production
• 📈 9. Evaluation, Benchmarks & Datasets
• 🛡️ 10. AI Safety, Alignment & Interpretability
• 🧩 11. Specialized Domains
• 🖥️ 12. User Interfaces & Self-hosted Platforms
• 🧪 13. Developer Tools & Integrations
• 📚 14. Resources & Learning

---

🧬 1. Core Frameworks & Libraries

Core libraries and frameworks used to build, train, and run AI and machine learning systems.

Deep Learning Frameworks

• PyTorch - Dynamic computation graphs, Pythonic API, dominant in research and production. The current standard for most frontier AI work.
• TensorFlow - End-to-end platform with excellent production deployment, TPU support, and large-scale serving tools.
• JAX + Flax - High-performance numerical computing with composable transformations (JIT, vmap, grad). Rising favorite for research and scientific ML.
• dm-haiku - JAX-based neural network library from Google DeepMind. Elegant functional API with state management, widely used in DeepMind's research. Apache 2.0 licensed.
• Equinox - Elegant easy-to-use neural networks and scientific computing in JAX. Callable PyTrees with filtered transformations, seamless interoperability with the JAX ecosystem. Apache 2.0 licensed.
• Diffrax - Numerical differential equation solvers in JAX. Autodifferentiable and GPU-capable ODE/SDE/CDE solvers for scientific machine learning and neural differential equations. Apache 2.0 licensed.
• vit-pytorch - Comprehensive Vision Transformer (ViT) implementations in PyTorch. Reference implementations of all major vision transformer variants including ViT, DeiT, Swin, and more. MIT licensed.
• NumPyro - Probabilistic programming with NumPy powered by JAX for autograd and JIT compilation. Bayesian modeling and inference at scale.
• Keras - High-level, beginner-friendly API that now runs on multiple backends (TensorFlow, JAX, PyTorch). Perfect for rapid experimentation.
• tinygrad - Minimalist deep learning framework with tiny code footprint. The "you like pytorch? you like micrograd? you love tinygrad!" philosophy - simple yet powerful.
• PaddlePaddle - Industrial deep learning platform from Baidu serving 23+ million developers and 760,000+ companies. China's first independent R&D framework with advanced distributed training and deployment capabilities.
• PyTorch Geometric - Library for deep learning on irregular input data such as graphs, point clouds, and manifolds. Part of the PyTorch ecosystem.
• timm (PyTorch Image Models) - The largest collection of PyTorch image encoders and backbones. 900+ pretrained models including ResNet, EfficientNet, Vision Transformer, ConvNeXt, and more with training and inference scripts. Apache 2.0 licensed.
• Triton - Language and compiler for writing highly efficient custom deep-learning primitives. Powers kernel optimizations in PyTorch, JAX, and other frameworks. MIT licensed.
• GGML - Tensor library for machine learning. The foundational C/C++ library powering llama.cpp and many on-device inference engines. MIT licensed.
• MLX - Array framework for machine learning on Apple silicon. Efficient unified memory design with NumPy-like API, automatic differentiation, and multi-device support. MIT licensed.

High-Performance Compute Libraries

• oneDNN - oneAPI Deep Neural Network Library. Cross-platform performance library of basic building blocks for deep learning, optimized for Intel CPUs, GPUs, and Arm architectures. Apache 2.0 licensed.
• ONNX - Open standard for machine learning interoperability. Open Neural Network Exchange provides an open ecosystem that empowers AI developers to choose the right tools as their project evolves. Apache 2.0 licensed.
• IREE - Retargetable MLIR-based machine learning compiler and runtime toolkit. Lowers ML models to unified IR that scales from datacenter to mobile and edge deployments. Apache 2.0 licensed.

Rust ML Frameworks

• Burn - Next-generation deep learning framework in Rust. Backend-agnostic with CPU, GPU, WebAssembly support.
• Candle (Hugging Face) - Minimalist ML framework for Rust. PyTorch-like API with focus on performance and simplicity.
• linfa - Comprehensive Rust ML toolkit with classical algorithms. scikit-learn equivalent for Rust with clustering, regression, and preprocessing.

Julia ML Frameworks

• Flux.jl - 100% pure-Julia ML stack with lightweight abstractions on top of native GPU and AD support. Elegant, hackable, and fully integrated with Julia's scientific computing ecosystem.
• MLJ.jl - Comprehensive Julia machine learning framework providing a unified interface to 200+ models with meta-algorithms for selection, tuning, and evaluation. MIT licensed.
• ModelingToolkit.jl - High-performance symbolic-numeric modeling framework for scientific machine learning. Automatically generates fast functions for model components like Jacobians and Hessians with automatic sparsification and parallelization. MIT licensed.

NLP & Transformers

• spaCy (Explosion AI) - Industrial-strength natural language processing with 75+ languages, transformer pipelines, and production-grade NER, parsing, and text classification.
• Transformers (Hugging Face) - The de facto standard library for pretrained NLP models. 1M+ models, 250,000+ downloads/day. BERT, GPT, Llama, Qwen, and hundreds more.
• sentence-transformers - Classic library for sentence and image embeddings.
• tokenizers (Hugging Face) - Fast state-of-the-art tokenizers for training and inference.
• fairseq2 - FAIR Sequence Modeling Toolkit 2. Complete rewrite of fairseq with modern PyTorch APIs, native support for LLM training (70B+ models), vLLM integration, and first-party recipes for instruction finetuning and preference optimization. MIT licensed.

Data Processing & Manipulation

• Pandas - The gold standard for data analysis and manipulation in Python.
• Polars - Blazing-fast DataFrame library (Rust backend) - modern alternative to pandas for large-scale workloads.
• cuDF - GPU DataFrame library from RAPIDS. Accelerates pandas workflows on NVIDIA GPUs with zero code changes using cuDF.pandas accelerator mode.
• Modin - Parallel pandas DataFrames. Scale pandas workflows by changing a single line of code - distributes data and computation automatically.
• Dask - Parallel computing for big data - scales pandas/NumPy/scikit-learn to clusters.
• NumPy - Fundamental array computing library that powers almost every AI stack.
• SciPy - Scientific computing algorithms (optimization, linear algebra, statistics, signal processing).
• NetworkX - Creation, manipulation, and study of complex networks. The foundational graph analysis library for Python data science.
• cuGraph - GPU graph analytics library with NetworkX-compatible API. 10-100x faster than CPU for large-scale graph algorithms. Apache 2.0 licensed.
• Vaex - Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python. Visualize and explore billion-row datasets at millions of rows per second. MIT licensed.
• Datashader - High-performance large data visualization. Renders billions of points interactively without aggregation artifacts. BSD-3-Clause licensed.
• Zarr - Chunked, compressed, N-dimensional array storage. Scalable tensor data format optimized for cloud and parallel computing. MIT licensed.
• NVIDIA DALI - GPU-accelerated data loading and augmentation library with highly optimized building blocks for deep learning applications. Apache 2.0 licensed.
• Narwhals - Lightweight compatibility layer between DataFrame libraries. Write Polars-like code that works seamlessly across pandas, Polars, cuDF, Modin, and more. MIT licensed.
• Ibis - Portable Python dataframe library with 20+ backends. Write pandas-like code that runs locally with DuckDB or scales to production databases (BigQuery, Snowflake, PostgreSQL) by changing one line. Apache 2.0 licensed.
• skrub - Machine learning with dataframes for dirty categorical data. Preprocessing and feature engineering for heterogeneous data with seamless pandas/Polars integration. BSD-3-Clause licensed.
• Oxen - Lightning fast data version control for machine learning. Optimized for large datasets with efficient diffing, branching, and collaboration. Apache 2.0 licensed.
• Pandera - Statistical data testing and validation for dataframes. Pydantic-like API for pandas, Polars, and other dataframe libraries with type hints and lazy validation. MIT licensed.
• Snorkel - System for quickly generating training data with weak supervision. Programmatically label, build, and manage training data using labeling functions and probabilistic consensus models. Powers Snorkel Flow and used by Google, Apple, and Intel. Apache 2.0 licensed.
• DuckDB - High-performance analytical in-process SQL database system. Fast, reliable, portable, and easy to use with rich SQL dialect support. Perfect for data processing and analytics workloads. MIT licensed.
• FiftyOne - Visual AI development toolkit for visualizing, labeling, and evaluating visual datasets and models. Supercharges computer vision workflows with dataset exploration and model analysis. Apache 2.0 licensed.
• Label Studio - Multi-type data labeling and annotation tool with standardized output format. Configurable interface for images, text, audio, video, and time series with ML-assisted labeling. Apache 2.0 licensed.
• Delta Lake - Open-source storage framework enabling Lakehouse architecture with ACID transactions, scalable metadata handling, and unified batch/streaming processing. Apache 2.0 licensed.
• Apache Iceberg - High-performance open table format for huge analytic tables. Brings SQL table reliability to big data with time travel, hidden partitioning, and schema evolution. Works with Spark, Trino, Flink, Presto, Hive and Impala. Apache 2.0 licensed.
• Apache Hudi - Open data lakehouse platform for ingesting, indexing, storing, serving, transforming and managing data across cloud environments. Supports upserts, deletes and incremental processing on big data with built-in ingestion tools for Spark and Flink. Apache 2.0 licensed.
• lakeFS - Data version control for your data lake that transforms object storage into Git-like repositories. Enables atomic, versioned data lake operations with branching, committing, and merging for data pipelines. Apache 2.0 licensed.
• Apache Airflow - Platform to programmatically author, schedule, and monitor workflows. Industry-standard orchestration for data pipelines and ML workflows with 500+ integrations. Apache 2.0 licensed.
• Apache Spark - Unified analytics engine for large-scale data processing. In-memory cluster computing with high-level APIs in Python, Scala, Java, and R. Powers MLlib for distributed machine learning and Structured Streaming for real-time data. Apache 2.0 licensed.
• Apache Flink - Stream processing framework with powerful batch and streaming capabilities. High-throughput, low-latency runtime with exactly-once processing guarantees. Ideal for real-time AI inference pipelines and event-driven ML applications. Apache 2.0 licensed.
• Apache Beam - Unified programming model for batch and streaming data processing. Write pipelines once, run anywhere on Flink, Spark, or Google Cloud Dataflow. Portable, extensible, and enterprise-ready for AI data pipelines. Apache 2.0 licensed.
• Scrapy - Fast, high-level web crawling and scraping framework for Python. Extract structured data from websites at scale with built-in support for handling common challenges like pagination, cookies, and concurrent requests. BSD-3-Clause licensed.
• Temporal - Durable execution platform for reliable workflow orchestration. Build resilient data pipelines and ML workflows that survive failures and continue execution exactly where they left off. MIT licensed.
• Luigi - Python module for building complex pipelines of batch jobs. Handles dependency resolution, workflow management, visualization, and Hadoop integration. Built at Spotify and battle-tested in production. Apache 2.0 licensed.
• Mage.ai - Modern open-source data pipeline tool for integrating and transforming data. AI-native ETL/ELT platform with 100+ integrations, real-time monitoring, and collaborative features. Apache 2.0 licensed.
• Hamilton - Declarative dataflow framework for building testable, modular, self-documenting data pipelines. Encode lineage and metadata directly in Python functions. Originally from Stitch Fix, now Apache incubating. Apache 2.0 licensed.
• D-Tale - Visualizer for pandas data structures with a Flask back-end and React front-end. Interactive data exploration with charting, filtering, and code export. LGPL-2.1 licensed.
• Sweetviz - Beautiful, high-density visualizations for exploratory data analysis in two lines of code. Self-contained HTML reports for dataset comparison and target analysis. MIT licensed.
• TextAttack - Python framework for adversarial attacks, data augmentation, and model training in NLP. Augment datasets to increase model robustness and generate adversarial examples. MIT licensed.

Classical ML & Gradient Boosting

• scikit-learn - Industry-standard library for traditional machine learning (classification, regression, clustering, pipelines).
• XGBoost - Scalable, high-performance gradient boosting library. Still dominates Kaggle and tabular competitions.
• LightGBM - Microsoft's ultra-fast gradient boosting framework, optimized for speed and memory.
• CatBoost - Gradient boosting that handles categorical features natively with great out-of-the-box performance.
• sktime - Unified framework for machine learning with time series. Scikit-learn compatible API for forecasting, classification, clustering, and anomaly detection.
• StatsForecast - Lightning-fast statistical forecasting with ARIMA, ETS, CES, and Theta models. Optimized for high-performance time series workloads.
• MLForecast - Scalable machine learning for time series forecasting. Train any sklearn-compatible model on millions of time series with efficient feature engineering. Apache 2.0 licensed.
• cuML - GPU-accelerated machine learning algorithms with scikit-learn compatible API. 10-50x faster than CPU implementations for large datasets. Apache 2.0 licensed.
• SynapseML - Distributed machine learning on Apache Spark. Scalable, composable APIs for text analytics, vision, anomaly detection with seamless Python/Scala/R/.NET integration. MIT licensed.
• Darts - User-friendly forecasting and anomaly detection for time series. Unifies classical statistical models (ARIMA, ETS) with modern neural networks (N-BEATS, TFT, DeepAR) in a single scikit-learn compatible API. Apache 2.0 licensed.
• PyTorch Forecasting - Time series forecasting with PyTorch. Multiple neural architectures (N-BEATS, TFT, DeepAR) with in-built interpretation capabilities, built on PyTorch Lightning for distributed training. MIT licensed.

Data Engineering & Feature Stores

• DataHub - The #1 open-source metadata platform for data and AI. Data discovery, governance, and observability with 80+ connectors, column-level lineage, and AI assistant integration. Originally built at LinkedIn. Apache 2.0 licensed.
• OpenMetadata - Unified metadata platform for data discovery, observability, and governance. Column-level lineage, semantic search, and team collaboration with 70+ data service connectors. Apache 2.0 licensed.
• Amundsen - Data discovery and metadata engine from Lyft. PageRank-style search for data resources with usage-based ranking. LF AI & Data Foundation project. Apache 2.0 licensed.

Data Transformation & Analytics Engineering

• dbt-core - Transform data using software engineering best practices. The industry-standard framework for analytics engineering with 15M+ monthly downloads. Enables version control, testing, and documentation for SQL transformations. Apache 2.0 licensed.
• SQLMesh - Scalable and efficient data transformation framework with dbt compatibility. Features automatic data lineage, time travel, and virtual data environments for testing. Optimized for large-scale data warehouses. Apache 2.0 licensed.

Data Quality & Validation

• Deequ - Library built on top of Apache Spark for defining "unit tests for data". Measures data quality in large datasets with constraint verification, anomaly detection, and incremental validation. Used at Amazon for production data quality. Apache 2.0 licensed.
• Great Expectations - Always know what to expect from your data. Data validation, profiling, and documentation for data pipelines. Apache 2.0 licensed.
• ydata-profiling - One line of code for comprehensive data quality profiling and exploratory data analysis. Generates detailed reports for Pandas and Spark DataFrames including statistics, correlations, missing values, and data quality alerts. MIT licensed.
• Soda Core - Data contracts engine for the modern data stack. Define data quality checks in YAML and automatically validate schema and data across your pipelines. Supports 20+ data sources including Snowflake, BigQuery, and PostgreSQL. Apache 2.0 licensed.
• TFX (TensorFlow Extended) - End-to-end platform for deploying production ML pipelines. Data validation, transformation, model training, and serving with TensorFlow. Powers Google's production ML infrastructure. Apache 2.0 licensed.

Data Labeling & Annotation

• Label Studio - Multi-type data labeling and annotation platform for computer vision, NLP, and audio. Supports image classification, object detection, named entity recognition, and more with customizable interfaces. Apache 2.0 licensed.
• FiftyOne - Open-source dataset curation and model analysis tool for computer vision. Visualize, explore, and improve image and video datasets with tight integration to annotation tools. Apache 2.0 licensed.
• Doccano - Open-source text annotation tool for machine learning practitioners. Features text classification, sequence labeling, and sequence-to-sequence tasks for sentiment analysis, NER, and summarization. MIT licensed.
• Snorkel - System for quickly generating training data with weak supervision. Programmatically label data using labeling functions instead of manual annotation. Apache 2.0 licensed.
• OpenRefine - Free, open-source power tool for working with messy data. Clean, transform, and extend data with web services. Formerly Google Refine. BSD-3-Clause licensed.

AutoML & Hyperparameter Optimization

• Optuna - Modern, define-by-run hyperparameter optimization with pruning and visualizations. Extremely popular in 2026.
• AutoGluon - AWS AutoML toolkit for tabular, image, text, and multimodal data - state-of-the-art with almost zero code.
• FLAML - Microsoft's fast & lightweight AutoML focused on efficiency and low compute.
• Katib (Kubeflow) - Kubernetes-native AutoML for hyperparameter tuning, early stopping, and neural architecture search. Framework-agnostic with support for TensorFlow, PyTorch, XGBoost, and custom training operators. Apache 2.0 licensed.
• AutoKeras - Neural architecture search on top of Keras.

Interactive ML Apps & Notebooks

• Streamlit - The fastest way to build and share data apps. Transform Python scripts into beautiful web applications with minimal code. Widely used for ML model demos, data visualization, and internal tools.
• Gradio - Build and share delightful machine learning apps, all in Python. The de facto standard for creating interactive ML demos with automatic UI generation from function signatures. Powers thousands of Hugging Face Spaces.
• Marimo - A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

Model Training & Optimization Utilities

• Hugging Face Accelerate - Simple API to make training scripts run on any hardware (multi-GPU, TPU, mixed precision) with minimal code changes.
• DeepSpeed - Microsoft's deep learning optimization library for extreme-scale training (ZeRO, offloading, MoE).
• Transformers - Library of pretrained transformer models and utilities for text, vision, audio, and multimodal training and inference.
• FlashAttention - Fast exact attention kernels that reduce memory usage and accelerate transformer training and inference.
• xFormers - Optimized transformer building blocks and attention operators for PyTorch.
• PyTorch Lightning - High-level wrapper for PyTorch that removes boilerplate and adds best practices.
• fastai - Deep learning library providing practitioners with high-level components for state-of-the-art results. Built on PyTorch with a focus on usability and transfer learning. Apache 2.0 licensed.
• PyTorch Ignite - High-level library for training and evaluating neural networks in PyTorch with an engine, events & handlers system for maximum flexibility. BSD-3-Clause licensed.
• ONNX Runtime - High-performance inference and training for ONNX models across hardware.
• einops - Flexible, powerful tensor operations for readable and reliable code. Supports PyTorch, JAX, TensorFlow, NumPy, MLX.
• safetensors - Simple, safe way to store and distribute tensors. Fast, secure alternative to pickle for model serialization.
• torchmetrics - Machine learning metrics for distributed, scalable PyTorch applications. 80+ metrics with built-in distributed synchronization.
• torchao - PyTorch native quantization and sparsity for training and inference. Drop-in optimizations for production deployment.
• SHAP - Game theoretic approach to explain the output of any machine learning model. Industry standard for model interpretability.
• skorch - Scikit-learn compatible neural network library that wraps PyTorch. Seamlessly integrate PyTorch models with scikit-learn pipelines, grid search, and cross-validation.
• Composer - Supercharge your model training. MosaicML's PyTorch training library with built-in algorithms for efficient training (FSDP, gradient compression, progressive resizing) and seamless distributed training on large-scale clusters. Apache 2.0 licensed.
• NVIDIA Apex - PyTorch extension for mixed precision training and distributed training optimizations. Powers many production deep learning workloads with tools for automatic mixed precision (AMP), distributed data parallel, and fused optimizers. BSD-3-Clause licensed.

---

🧠 2. Open Foundation Models

Pretrained language, multimodal, speech, and video models with publicly available weights.

Large Language Models (Base + Chat)

• RWKV-7 "Goose" (BlinkDL) - Novel RNN architecture with transformer-level LLM performance. 100% attention-free, linear-time, constant-space (no kv-cache), infinite ctx_len. Linux Foundation AI project with runtime already deployed in Windows & Office.

• Qwen3 (Alibaba) - Flagship dense and MoE models with hybrid thinking modes (32B/235B). Apache 2.0 licensed with 128K context and superior agentic capabilities.

• Qwen3.6 (Alibaba) - Latest flagship series released April 2026 with 1M context window, agentic coding performance competitive with Claude 4.5 Opus, and enhanced multimodal capabilities.

• MiMo-V2-Flash (Xiaomi) - 309B MoE model (15B active) with hybrid attention and Multi-Token Prediction for efficient high-speed reasoning. Apache 2.0 licensed.

• Nemotron (NVIDIA) - Open and efficient models for agentic AI with training recipes, deployment guides, and use-case examples. Apache 2.0 licensed.

• Gemma 4 (Google) - Released April 2026 in four sizes (E2B, E4B, 26B MoE, 31B Dense). First major update in a year with Apache 2.0 license, complex logic, and agentic workflows.

• Kimi K2 (Moonshot AI) - State-of-the-art 1T parameter MoE model with 32B activated parameters and 128K context. Trained with Muon optimizer for exceptional reasoning and coding performance.

• Kimi K2.5 (Moonshot AI) - Frontier open-weight MoE model with 256K context, strong coding and reasoning performance, and native multimodal + tool-use support for agentic workflows.

• Phi-4 (Microsoft) - Small but highly capable models optimized for reasoning, edge devices, and on-device inference. Includes Phi-4-reasoning variants with thinking capabilities.

• GLM-5 (Zhipu AI) - Strong open model line with solid coding, reasoning, and agentic-task performance.

• OLMo 2 (Allen AI) - Fully open-source LLMs (1B–32B) with complete transparency: models, data, training code, and logs. Designed by scientists, for scientists.

• Llama 4 (Meta) - First native multimodal MoE open-source models (Scout: 10M context, Maverick: 400B+ params). Released April 2025 with enterprise-grade capabilities.

• GPT-OSS (OpenAI) - OpenAI's first open-weight models since GPT-2 (120B and 20B MoE). Apache 2.0 licensed with state-of-the-art performance for their size class. Released August 2025.

• Mamba (State Space Models) - Novel State Space Model architecture with linear-time inference and transformer-level performance. 100% attention-free with constant memory usage, enabling efficient long-sequence modeling. Pretrained models from 130M to 2.8B parameters trained on 300B-600B tokens. Apache 2.0 licensed.

• Pythia (EleutherAI) - Suite of interpretability-focused LLMs (70M to 12B parameters) with fully open training data, intermediate checkpoints, and analysis tools. Designed for studying learning dynamics and interpretability with public domain training data. Apache 2.0 licensed.

• T5 (Google) - Text-to-Text Transfer Transformer that unified NLP tasks under a single encoder-decoder architecture. The foundation for Flan-T5 and many downstream applications. One of the first OSI-validated fully open-source language models with training data and code. Apache 2.0 licensed.

• GPT-NeoX-20B (EleutherAI) - 20B parameter autoregressive language model trained on the Pile dataset. One of the largest dense open-source models with publicly available weights at release. Complete training codebase with distributed training support. Apache 2.0 licensed.

Coding & Reasoning Models

• DeepSeek-Coder-V2 / R1-Coder - Best-in-class open coding model (236B MoE). Outperforms closed models on many code benchmarks.
• Qwen3-Coder-Next (Alibaba) - Leading open coding model. Strong Pareto frontier for cost-effective agent deployment.

Multimodal Models (Vision + Language)

• MMaDA (Gen-Verse) - Open-sourced multimodal large diffusion language model with unified architecture for text, image generation and multimodal reasoning. MIT licensed, NeurIPS 2025.
• Qwen3-VL (Alibaba) - Latest flagship VLM with native 256K context (expandable to 1M), visual agent capabilities, 3D grounding, and superior multimodal reasoning. Major leap over Qwen2.5-VL.
• GLM-4.5V / GLM-4.1V-Thinking (Zhipu AI) - Strong multimodal reasoning with scalable reinforcement learning. Compares favorably with Gemini-2.5-Flash on benchmarks.
• MiniCPM-o 2.6 - Gemini 2.5 Flash level MLLM for vision, speech, and full-duplex multimodal live streaming on your phone. Apache 2.0 licensed.
• Gemma 4 (Google) - Multimodal model supporting vision-language input, optimized for efficiency, complex logic, and on-device use.
• Magma (Microsoft) - Foundation model for multimodal AI agents that perceives the world and takes goal-driven actions across digital and physical environments. CVPR 2025.
• OpenCLIP - Open source implementation of CLIP with trained models and training code. Includes state-of-the-art trained ViT-G/14 models and comprehensive zero-shot evaluation suite.
• Show-o - Unified multimodal model for both multimodal understanding and text-to-image generation with transformative autoregressive modeling. Apache 2.0 licensed.
• Moondream (m87-labs) - Tiny vision language model (0.5B and 2B parameters) that runs anywhere. Powerful image understanding with remarkably small footprint for edge devices and real-time applications. Apache 2.0 licensed.
• VILA (NVIDIA) - Family of state-of-the-art vision language models for diverse multimodal AI tasks across edge, data center, and cloud. Features NVILA 8B/15B with efficient training and deployment. Apache 2.0 licensed.
• OmniGen (VectorSpaceLab) - Unified image generation model that handles text-to-image, subject-driven generation, identity-preserving generation, and image editing from multi-modal prompts without additional plugins. MIT licensed.
• Skywork-R1V (Skywork AI) - Advanced multimodal reasoning model specializing in vision-language tasks with chain-of-thought capabilities. State-of-the-art open multimodal reasoning with 76.0 on MMMU benchmark. MIT licensed.
• Depth Anything V2 - Foundation model for monocular depth estimation trained on 595K synthetic and 62M+ real images. Provides robust, fine-grained depth estimation for any image. Apache 2.0 licensed.
• OmniSVG - First family of end-to-end multimodal SVG generators leveraging pre-trained Vision-Language Models. Capable of generating complex SVGs from simple icons to intricate anime characters. NeurIPS 2025. Apache 2.0 licensed.

Speech & Audio Models (TTS, STT, Music)

• NVIDIA NeMo Speech - Scalable generative AI framework for Speech AI including ASR, TTS, and speech LLMs. Includes state-of-the-art Canary and Parakeet models with 25+ European language support. Apache 2.0 licensed.
• FunASR - Fundamental end-to-end speech recognition toolkit with SOTA pretrained models. Supports ASR, VAD, speaker verification, diarization, and multi-talker ASR. Industrial-grade with 31-language support and real-time transcription services. MIT licensed.
• Whisper (OpenAI → community forks) - The gold-standard open speech-to-text model. Massive community fine-tunes available.
• faster-whisper (SYSTRAN) - Reimplementation of Whisper using CTranslate2 for up to 4x faster inference with same accuracy. Supports batched processing and 8-bit quantization.
• OuteTTS / CosyVoice 2 - High-quality open TTS with natural prosody and multilingual support.
• Fish Speech / StyleTTS 2 - Zero-shot TTS with excellent voice cloning. Extremely popular in 2026.
• MusicGen / AudioCraft (Meta) - Open music and audio generation models.
• VibeVoice (Microsoft) - Open-source frontier voice AI with expressive, longform conversational speech synthesis. 7B parameter TTS with streaming support.
• Qwen3-TTS (Alibaba) - Open TTS series supporting stable, expressive, and streaming speech generation with free-form voice design and vivid voice cloning. Natural language instruction-driven control over timbre, emotion, and prosody. Apache 2.0 licensed.
• Chatterbox (Resemble AI) - State-of-the-art open TTS family with 350M parameter Turbo variant. Single-step generation with native paralinguistic tags for realistic dialogue.
• Dia (Nari Labs) - 1.6B parameter TTS generating ultra-realistic dialogue in one pass with nonverbal communications (laughter, coughing). Emotion and tone control via audio conditioning.
• Voxtral TTS (Mistral) - 4B parameter state-of-the-art TTS with zero-shot voice cloning, 9-language support, and ~90ms time-to-first-audio for voice agents.
• Ultravox (Fixie AI) - Fast multimodal LLM for real-time voice. Production-grade speech-to-text with streaming audio input and low-latency response for conversational AI applications. MIT licensed.
• WhisperSpeech - Open source text-to-speech system built by inverting Whisper. High-quality voice cloning with zero-shot capabilities. MIT licensed.
• VoxCPM - Tokenizer-free diffusion autoregressive TTS with 2B parameters. Supports 30+ languages with automatic detection, creative voice design from text descriptions, and high-fidelity voice cloning. Apache 2.0 licensed.
• F5-TTS - Flow matching-based TTS with fluent and faithful speech synthesis. Zero-shot voice cloning with high naturalness and prosody accuracy. MIT licensed.
• CosyVoice - Multi-lingual large voice generation model with full-stack inference, training and deployment capabilities. Supports cross-lingual voice cloning and emotional expression control. Apache 2.0 licensed.
• ChatTTS - Generative speech model optimized for daily dialogue. Natural, expressive conversational speech synthesis with fine-grained prosody control. AGPL-3.0 licensed.
• SpeechBrain - PyTorch-based speech toolkit for ASR, TTS, speaker recognition, and speech enhancement. Modular, extensible framework with state-of-the-art recipes. Apache 2.0 licensed.

Video & Animation Models

• Open-Sora (HPC-AI Tech) - Democratizing efficient video production for all. Complete open-source video generation system with 11B model achieving commercial-level quality. Apache 2.0 licensed.
• Mochi 1 (Genmo) - 10B open video model with impressive motion and consistency.

Image Generation Models

• Stable Diffusion XL - Next-generation image generation model with significantly improved quality, 1024px native resolution, and better prompt adherence. Foundation for SDXL-based video models. CreativeML Open RAIL++-M licensed.
• OmniGen (VectorSpaceLab) - Unified image generation model handling text-to-image, subject-driven generation, identity-preserving generation, and image editing from multi-modal prompts in a single framework. MIT licensed.

Additional Vision-Language Models

• MiniCPM-V (OpenBMB) - GPT-4V level multimodal LLM for single image, multi-image and high-FPS video understanding on edge devices. 8B parameters with superior OCR and reasoning capabilities. Apache 2.0 licensed.

---

⚡ 3. Inference Engines & Serving

Inference runtimes, serving systems, and optimization tools for running models locally or in production.

Local / On-device Inference

• llama.cpp - Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.
• Ollama - Dead-simple local LLM runner with a one-line install, model registry, and OpenAI-compatible API.
• MLX (Apple) - High-performance array framework + LLM inference optimized for Apple Silicon.
• MLC-LLM - Deployment engine that compiles and runs LLMs across browsers, mobile devices, and local hardware.
• WebLLM - High-performance in-browser LLM inference engine. Runs models directly in the browser with WebGPU acceleration.
• llama-cpp-python - Official Python bindings for llama.cpp.
• KoboldCpp - User-friendly llama.cpp fork focused on role-playing and creative writing.
• RamaLama - Container-centric tool for simplifying local AI model serving. Automatically detects GPUs, pulls optimized container images, and runs models securely in rootless containers with enterprise-grade isolation.
• LiteRT-LM - Google's production-ready inference framework for deploying LLMs on edge devices. Cross-platform support for Android, iOS, Web, Desktop, and IoT with GPU/NPU acceleration. Powers on-device GenAI in Chrome and Chromebook Plus. Apache 2.0 licensed.
• exo - Run frontier AI locally by connecting all your devices into an AI cluster. Features automatic device discovery, RDMA over Thunderbolt for 99% latency reduction, topology-aware auto parallel, and tensor parallelism. Uses MLX backend for distributed inference across Apple Silicon devices. Apache 2.0 licensed.

High-performance Serving & API Servers

• llm-d - Kubernetes-native distributed LLM inference framework. Donated to CNCF by RedHat, Google, and IBM. Intelligent scheduling, KV-cache optimization, and state-of-the-art performance across accelerators.
• LMDeploy - Toolkit for compressing, deploying, and serving LLMs from OpenMMLab. 4-bit inference with 2.4x higher performance than FP16, distributed multi-model serving across machines.
• vLLM - State-of-the-art serving engine with PagedAttention and continuous batching. Currently the fastest production-grade LLM server.
• LMCache - Supercharge LLM inference with the fastest KV Cache layer. 3-10x delay savings and GPU cycle reduction for multi-round QA and RAG. Integrates seamlessly with vLLM for distributed, high-throughput deployments. Apache 2.0 licensed.
• vLLM Production Stack - Kubernetes-native production stack for vLLM inference. Automated deployment, autoscaling, and monitoring for enterprise-grade LLM serving. Built by the vLLM team for seamless integration.
• nano-vLLM - Minimalist vLLM implementation in ~1,200 lines of Python. Educational yet performant with prefix caching, tensor parallelism, and CUDA graph acceleration. Comparable inference speeds to full vLLM. MIT licensed.
• SGLang - Next-gen serving framework with RadixAttention. Powers xAI's production workloads at 100K+ GPUs scale.
• TensorRT-LLM - NVIDIA's official high-performance inference backend.
• Aphrodite Engine - vLLM fork optimized for role-play and creative writing. Supports extensive quantization methods (AQLM, AWQ, GPTQ, GGUF, FP8) and modern samplers. Active development with multi-LoRA and speculative decoding support.
• AIBrix - Cost-efficient and pluggable infrastructure components for GenAI inference. Kubernetes-native control plane for vLLM with distributed KV cache, heterogeneous GPU serving, and intelligent routing. Apache 2.0 licensed.
• Triton Inference Server - NVIDIA's production-grade open-source inference serving software. Supports multiple frameworks (TensorRT, PyTorch, ONNX) with optimized cloud and edge deployment.
• mistral.rs - Fast, flexible Rust-native LLM inference engine built on Candle. Supports text, vision, audio, image generation, and embeddings with hardware-aware auto-tuning.
• KTransformers - Flexible framework for heterogeneous CPU-GPU LLM inference and fine-tuning. Enables running large MoE models by offloading experts to CPU with BF16/FP8 precision support.
• llamafile - Mozilla's single-file distributable LLM solution. Bundle model weights, inference engine, and runtime into one portable executable that runs on six OSes without installation.
• Xinference - Unified, production-ready inference API for LLMs, speech, and multimodal models. Drop-in GPT replacement with single-line code changes. Supports thousands of models with auto-batching and distributed inference.
• RTP-LLM (Alibaba) - Alibaba's high-performance LLM inference acceleration engine. Powers production LLM services across Taobao, Tmall, and Alibaba's international AI platform. Supports PagedAttention, FlashAttention, FlashDecoding, INT8/INT4 quantization, and heterogeneous hardware (GPU/ARM CPU/Intel). Apache 2.0 licensed.
• LitServe (Lightning AI) - Minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling. 2x faster than FastAPI with built-in batching, streaming, and multi-GPU autoscaling. Apache 2.0 licensed.
• LightLLM - Pure Python-based LLM inference and serving framework with lightweight design, easy extensibility, and high-speed performance. Integrates optimizations from FasterTransformer, TGI, vLLM, and SGLang.
• TabbyAPI - FastAPI-based API server for ExLlamaV2/V3 backends. OpenAI-compatible API with support for model loading/unloading, embeddings, speculative decoding, multi-LoRA, and streaming.
• GPUStack - GPU cluster manager that orchestrates inference engines like vLLM and SGLang. Automated engine selection, parameter optimization, and distributed multi-GPU deployment for high-performance AI workloads.
• One-API - LLM API management and key redistribution system. Unifies multiple providers (OpenAI, Anthropic, Azure, etc.) under a single OpenAI-compatible API with built-in rate limiting, quota management, and cost tracking. MIT licensed.
• OpenLLM (BentoML) - Production-grade platform for running any open-source LLMs as OpenAI-compatible API endpoints. Supports 50+ models with built-in streaming, batching, and auto-acceleration. Apache 2.0 licensed.
• Higress (Alibaba) - AI-native API gateway born from Alibaba's internal infrastructure with 2+ years of production validation. Provides unified LLM API and MCP (Model Context Protocol) management with enterprise-grade 99.99% availability. Apache 2.0 licensed.
• NVIDIA Dynamo - Datacenter-scale distributed inference serving framework from NVIDIA. Orchestration layer above vLLM/SGLang/TensorRT-LLM with disaggregated serving, KV-aware routing, and automatic scaling. Built in Rust with Python extensibility. Apache 2.0 licensed.
• Microsoft BitNet - Official inference framework for 1-bit LLMs (BitNet b1.58). Enables running large models on CPU with minimal memory footprint. Features custom kernels for ternary weight quantization and efficient matmul operations. MIT licensed.

Additional Inference Engines

• CTranslate2 - Fast inference engine for Transformer models supporting OpenNMT and Hugging Face models. Optimized for CPU and GPU with batching, quantization (INT8/FP16), and dynamic memory management. Powers faster-whisper and other production deployments. MIT licensed.
• llama-swap - Intelligent model swapping proxy for llama.cpp. Enables seamless hot-swapping between different GGUF models without restarting the server, with automatic model loading/unloading and OpenAI-compatible API. MIT licensed.
• optillm - Optimizing inference proxy for LLMs with load balancing, failover, and request routing across multiple providers and models. Improves reliability and performance for production deployments. Apache 2.0 licensed.
• mllm - Fast and lightweight multimodal LLM inference engine for mobile and edge devices. Optimized for running vision-language models on resource-constrained hardware with efficient memory management. MIT licensed.
• shimmy - Python-free Rust inference server with OpenAI API compatibility. Supports GGUF and SafeTensors formats with hot model swap, auto-discovery, and single binary deployment for zero-dependency inference. Apache 2.0 licensed.
• PowerInfer - High-speed LLM inference for local deployment on consumer GPUs. Achieves up to 11x speedup over llama.cpp on RTX 4090 by exploiting power-law neuron activation patterns. MIT licensed.
• distributed-llama - Distributed LLM inference connecting home devices into a powerful cluster. More devices means faster inference via tensor parallelism over Ethernet. Supports Linux, macOS, Windows, ARM, and x86_64 AVX2 CPUs. MIT licensed.
• ik_llama.cpp - High-performance llama.cpp fork with better CPU and hybrid GPU/CPU performance, SOTA quantization types, first-class Bitnet support, and improved DeepSeek performance via MLA, FlashMLA, and fused MoE operations. MIT licensed.
• xLLM - High-performance inference engine optimized for Chinese AI accelerators (Cambricon MLU, Hygon DCU, Huawei Ascend). Features service-engine decoupled architecture with elastic scheduling, PD disaggregation, and global KV cache management. Powers JD.com's core retail businesses. Apache 2.0 licensed.
• Mooncake - Production-grade serving platform for Kimi (Moonshot AI). Features distributed KV cache pool with intelligent offloading, prefill/decode disaggregation, and cross-instance KV reuse. Integrated with vLLM, SGLang, and TensorRT-LLM. Apache 2.0 licensed.

Quantization, Distillation & Optimization

• GGUF (part of llama.cpp) - Modern quantized format that powers most local inference.
• bitsandbytes - 8-bit and 4-bit optimizers + quantization.
• ExLlamaV2 - Highly optimized CUDA kernels for 4-bit/8-bit inference.
• Optimum - Hardware-specific acceleration and quantization.

---

🤖 4. Agentic AI & Multi-Agent Systems

Frameworks and platforms for building agent-based systems and multi-agent workflows.

Single-Agent Frameworks

• AutoGPT - The original autonomous AI agent framework that sparked the agent revolution. Vision of accessible AI for everyone with modular agent architecture, benchmark testing, and forge-based agent building. 183k+ stars.

• BabyAGI - Pioneering task-driven autonomous agent that inspired the AI agent movement. Simple, elegant implementation of an AI agent that creates, prioritizes, and executes tasks autonomously. 22k+ stars.

• LangGraph - Stateful, controllable agent orchestration.

• CrewAI - Role-based agent framework.

• AutoGen (AG2) - Flexible multi-agent conversation framework.

• DSPy - Framework for programming language model pipelines with modules, optimizers, and evaluation loops.

• Semantic Kernel - SDK for building and orchestrating AI agents and workflows across multiple programming languages.

• smolagents - Lightweight agent framework centered on tool use and code-executing workflows.

• LangChain - Foundational library for agents, chains, and memory.

• Neuron AI - PHP Agentic Framework for building production-ready AI driven applications. Connect components (LLMs, vector DBs, memory) to agents that can interact with your data. MIT licensed.

• II-Agent (Intelligent Internet) - New open-source framework to build and deploy intelligent agents with support for Claude, Gemini, and OpenAI models. Apache 2.0 licensed.

• Hermes Agent (NousResearch) - The agent that grows with you. Autonomous server-side agent with persistent memory that learns and improves over time.

• Strands Agents - Model-driven approach to building AI agents in just a few lines of code. Multi-agent systems, autonomous agents, and streaming support with built-in MCP. Apache 2.0 licensed.

• Agno - Build, run, and manage agentic software at scale. High-performance framework for multi-agent systems with memory, knowledge, and tools.

• Upsonic - Agent framework for fintech and banking with built-in MCP support, guardrails, and tool server architecture.

• VoltAgent - TypeScript-first AI agent engineering platform with memory, RAG, workflows, MCP integration, and voice support.

• PocketFlow - 100-line minimalist LLM framework for building agent workflows. Lightweight, extensible architecture for tool use and autonomous task execution.

• Agent Development Kit (Google) - Code-first Python toolkit for building sophisticated AI agents with multi-agent orchestration, built-in evaluation, and flexible deployment. Model-agnostic with tight Google ecosystem integration. Apache 2.0 licensed.

• PydanticAI - Type-safe AI agent framework from the creators of Pydantic. Model-agnostic with 20+ providers, built-in observability via Logfire, MCP/A2A protocol support, and YAML/JSON agent definitions. MIT licensed.

• Qwen-Agent - Agent framework built on Qwen models featuring function calling, MCP support, code interpreter, RAG, and Chrome extension. Powers Qwen Chat with advanced tool use and planning capabilities. Apache 2.0 licensed.

• Griptape - Modular Python framework for AI agents and workflows with chain-of-thought reasoning, tools, and memory. Enforces structures like sequential pipelines and DAG-based workflows for predictable AI systems. Apache 2.0 licensed.

• Langroid - Harness LLMs with multi-agent programming. Mature tool calling system based on Pydantic, supports hundreds of LLM providers including OpenAI and local servers. Built for robust agent behavior in real-world use cases. MIT licensed.

• Marvin - Python framework for structured outputs and agentic AI workflows. Simplifies LLM interactions with type-safe interfaces, automatic schema generation, and built-in observability. From the creators of Prefect. Apache 2.0 licensed.

• Burr - Apache incubating framework for building stateful AI applications (chatbots, agents, simulations). Monitor, trace, persist, and execute on your own infrastructure with built-in UI and pluggable memory. Apache 2.0 licensed.

• KaibanJS - JavaScript-native framework for building and managing multi-agent systems with a Kanban-inspired approach. Visual task board for AI agents with real-time collaboration features. MIT licensed.

Multi-Agent Orchestration

• MetaGPT - The Multi-Agent Framework: First AI Software Company. Assigns different roles to GPTs to form a collaborative software entity. Takes one-line requirements and outputs comprehensive software development artifacts including user stories, competitive analysis, requirements, data structures, APIs, and documents. ICLR 2024 oral presentation (top 1.2%). MIT licensed.
• ChatDev - Multi-agent software development framework where AI agents collaborate as programmers, designers, and testers to build software. Apache 2.0 licensed.
• CAMEL - First and best multi-agent framework for building scalable agent systems. Apache 2.0 licensed with extensive tooling for agent communication and task automation.
• Swarms - Bleeding-edge enterprise multi-agent orchestration.
• Mastra - TypeScript-first agent framework with built-in RAG, workflows, tool integrations, observability and observational memory.
• Deer-Flow (ByteDance) - Open-source long-horizon SuperAgent harness that researches, codes, and creates. Handles tasks from minutes to hours with sandboxes, memories, tools, skills, subagents, and message gateway.
• OpenAI Agents SDK - Production-ready lightweight framework for multi-agent workflows. The evolution of Swarm with enhanced orchestration capabilities and enterprise-grade features.
• Symphony - Turns project work into isolated, autonomous implementation runs. Monitors work boards, spawns agents to handle tasks, and provides proof of work including CI status, PR reviews, and walkthrough videos. Engineering preview for managing work instead of supervising coding agents. Apache 2.0 licensed.
• Paperclip - AI agent company and orchestration framework with 55K+ stars. MIT licensed.
• AgentScope - Alibaba's production-ready multi-agent framework with 23K+ stars. Features built-in MCP and A2A support, message hub for flexible orchestration, and AgentScope Runtime for production deployment.
• mcp-agent - Build effective agents using Model Context Protocol and simple workflow patterns. Handles connection mechanics, LLM integration, and persistent state for production MCP-based agents. MIT licensed.
• Microsoft Agent Framework - Microsoft's official framework combining AutoGen's agent abstractions with Semantic Kernel's enterprise features. Supports Python and .NET with graph-based workflows.
• Agency Swarm - Reliable multi-agent orchestration framework built on top of the OpenAI Assistants API with organizational structure modeling.
• elizaOS - Autonomous multi-agent framework for building and deploying AI-powered applications. Features Discord/Telegram/Farcaster connectors, RAG support, and a modern web dashboard.
• OpenManus - Open-source framework for building general AI agents. Modular agent architecture with planning, tool use, and autonomous task execution. 56k+ stars. MIT licensed.
• OpenAgents - AI Agent Networks for Open Collaboration. Platform for building collaborative multi-agent systems with shared knowledge and distributed task execution. Apache 2.0 licensed.
• Hive (Aden) - Production-grade multi-agent orchestration framework with 10K+ stars. Apache 2.0 licensed.
• Agent Squad (AWS Labs) - Flexible multi-agent orchestration framework with intelligent intent classification and context management. Supports Python and TypeScript with pre-built agents for Bedrock, Lex, and custom integrations. Apache 2.0 licensed.
• DeepResearchAgent - Hierarchical multi-agent system for deep research tasks with automated task decomposition and execution across complex domains.
• Composio Agent Orchestrator - Agentic orchestrator for parallel coding agents. Plans tasks, spawns agents, and autonomously handles CI fixes, merge conflicts, and code reviews. MIT licensed.
• Open Multi-Agent - TypeScript-native multi-agent orchestration with multi-model teams and parallel execution. Automatically converts goals to task DAGs. MIT licensed.
• BeeAI Framework (IBM) - Production-ready multi-agent framework in Python and TypeScript. Features workflow orchestration, ACP/MCP protocol support, and deep watsonx integration. Part of Linux Foundation AI & Data program.
• AI Town - Deployable starter kit for building virtual towns where AI characters live, chat and socialize. Inspired by Stanford's Generative Agents research with persistent agent memory and social interactions. MIT licensed.
• Conductor OSS - Event-driven agentic orchestration platform providing durable and resilient execution engine for applications and AI agents. Battle-tested at Netflix, Tesla, LinkedIn, and J.P. Morgan with 30K+ stars. Apache 2.0 licensed.
• A2A Protocol - Agent2Agent (A2A) open protocol enabling communication and interoperability between opaque agentic applications. Donated to Linux Foundation by Google with 50+ technology partners. Apache 2.0 licensed.
• Sim Studio - Open-source AI workspace for building, deploying, and orchestrating AI agents. Visual canvas with 1000+ integrations, multi-framework support (Agno, OpenAI, LangChain, Google ADK), and self-hosted or cloud deployment. Apache 2.0 licensed.
• 2FastLabs Agent Squad - Flexible, lightweight open-source framework for orchestrating multiple AI agents to handle complex conversations with parallel execution capabilities. Apache 2.0 licensed.

Agent Protocols & Standards

• Agent File - Open file format (.af) for serializing stateful AI agents with persistent memory and behavior. Share, checkpoint, and version control agents across compatible frameworks. Apache 2.0 licensed.

Autonomous Coding Agents

• OpenHands (ex-OpenDevin) - Full-featured open-source AI software engineer.
• Goose - Extensible on-machine AI agent for development tasks.
• OpenCode - Terminal-native autonomous coding agent.
• Aider - Command-line pair-programming agent.
• Pi (badlogic) - Terminal coding agent with hash-anchored edits, LSP integration, subagents, MCP support, and package ecosystem.
• Mistral-Vibe (Mistral) - Minimal CLI coding agent by Mistral. Lightweight, fast, and designed for local development workflows.
• Nanocoder (Nano-Collective) - Beautiful local-first coding agent running in your terminal. Built for privacy and control with support for multiple AI providers via OpenRouter.
• Gemini CLI (Google) - Open-source AI agent that brings Gemini's power directly into your terminal. Supports code generation, shell execution, and file editing with full Apache 2.0 licensing.
• Archon - Workflow engine for deterministic AI coding agents. Define development processes as YAML workflows (planning → implementation → validation → review → PR) with isolated git worktrees for parallel execution. MIT licensed.
• mini-SWE-agent - Lightweight coding agent for repository and issue-fixing workflows, designed for simple agentic software engineering experiments.
• Trae Agent - Software-engineering agent from ByteDance for autonomous coding tasks and repository-level development workflows.
• Kilo Code - Open-source agentic coding assistant with IDE workflows, tool use, and support for local or OpenAI-compatible models.
• Open SWE - Asynchronous coding agent from the LangChain ecosystem for background software engineering tasks.
• Letta Code - Memory-first coding harness designed for long-lived agents that learn from experience. Persistent agents with portable memory across models (Claude, GPT, Gemini, GLM, Kimi). CLI and desktop app for MacOS, Windows, and Linux. Apache 2.0 licensed.

Prompt Engineering & Structured Outputs

• Outlines - Structured outputs for LLMs. Guarantees valid JSON, regex-compliant text, and Pydantic model outputs during generation. Trusted by NVIDIA, Cohere, HuggingFace, and vLLM. Apache 2.0 licensed.

• Promptify - Task-based NLP engine with Pydantic structured outputs, built-in evaluation, and LiteLLM as the universal LLM backend. Think "scikit-learn for LLM-powered NLP". Apache 2.0 licensed.

• LangGPT - Pioneering framework for structured and meta-prompt design. Battle-tested by thousands of users worldwide with 10,000+ stars. The most popular prompt engineering paradigm for creating reusable, maintainable prompt templates. Apache 2.0 licensed.

• AutoPrompt - Intent-based prompt calibration framework that iteratively optimizes prompts through automated edge case generation and refinement. Reduces manual prompt engineering effort while addressing prompt sensitivity and ambiguity. Apache 2.0 licensed.

• Prompt Optimizer - AI prompt optimization tool with multi-round iterative improvements, dual-mode optimization for system and user prompts, and multi-model support. Available as web app, desktop app, Chrome extension, and Docker deployment. AGPL-3.0 licensed.

• Guidance - Efficient programming paradigm for steering language models. Control output structure with loops, conditionals, and regex constraints inline. Reduces latency and cost vs conventional prompting. MIT licensed.

• XGrammar - Fast, flexible and portable structured generation engine. Default backend for vLLM, SGLang, TensorRT-LLM, and MLC-LLM with flexible grammar support and zero-overhead mask generation. Apache 2.0 licensed.

• LM Format Enforcer - Enforce output format (JSON Schema, Regex, etc) of language models by filtering allowed tokens at each generation step. Compatible with HuggingFace, llama-cpp-python, and vLLM. MIT licensed.

• AdalFlow - Library to build and auto-optimize LLM applications with LLM-AutoDiff for fine-tuning-free optimization. End-to-end workflow optimization with tracing and human-in-the-loop capabilities. MIT licensed.

• PromptTools - Open-source tools for prompt testing and experimentation with support for LLMs and vector databases. Test prompt variants across multiple providers (OpenAI, LLaMA) and vector stores (Chroma, Weaviate, LanceDB). Apache 2.0 licensed.

Domain-Specific Agents

• Composio - Tool integration layer for AI agents with 1000+ toolkits, authentication management, and sandboxed workbench. Powers tool use across major frameworks.
• Langflow - Visual low-code platform for agentic workflows.
• Dify - Production-ready agentic workflow platform.
• OWL (camel-ai/owl) - Advanced multi-agent collaboration system.
• gpt-researcher - Autonomous agent that conducts deep online research on any topic. Generates comprehensive reports with citations by orchestrating web searches, content scraping, and synthesis. Apache 2.0 licensed.
• AI-Scientist-v2 (SakanaAI) - Workshop-level automated scientific discovery via agentic tree search. Generates novel research ideas, runs experiments, and writes papers.
• PraisonAI - 24/7 AI employee team for automating complex challenges. Low-code multi-agent framework with handoffs, guardrails, memory, RAG, and 100+ LLM providers.
• Agent-S (Simular AI) - Open agentic framework that uses computers like a human. SOTA on OSWorld benchmark (72.6%) for GUI automation and computer control.
• MobileAgent (Alibaba/X-PLUG) - Powerful GUI agent family for autonomous mobile device control. Multimodal agent framework designed to operate smartphone apps through visual UI perception and reasoning. MIT licensed.
• UI-TARS Desktop (ByteDance) - Open-source multimodal AI agent stack with native GUI agent capabilities. Desktop application bringing GUI agent and vision power to your computer, browser, and terminal. Apache 2.0 licensed.
• Browser Use - Makes websites accessible for AI agents. Enables autonomous web automation, data extraction, and task completion with natural language instructions. MIT licensed.
• Steel Browser - Open-source browser API for AI agents and apps. Batteries-included browser sandbox for web automation without infrastructure worries. Apache 2.0 licensed.
• TradingAgents - Multi-agent framework for financial trading. Simulates professional trading firm operations with 6+ specialized agent roles, backtesting, risk management, and portfolio optimization. Built with LangGraph, supports multiple LLM providers.
• Parlant - Conversational control layer for customer-facing AI agents. Enterprise-grade context engineering framework optimized for consistent, compliant, and on-brand B2C and sensitive B2B interactions. Apache 2.0 licensed.
• n8n - Self-hostable workflow automation platform with AI agent nodes, tool integrations, and production automation workflows.
• Activepieces - Open-source automation platform with AI agents, MCP integrations, and self-hosted workflow orchestration.
• Julep - Stateful agent workflow platform with memory, tools, branching, and long-running task execution.
• uAgents (Fetch.ai) - Fast and lightweight framework for creating decentralized agents with ease. Agents automatically join the network by registering on the Almanac smart contract. Supports agent-to-agent communication out of the box. Apache 2.0 licensed.
• Tracecat - Self-hostable security automation platform for building agentic workflows across alerts, cases, and operations.
• ToolJet - Self-hostable internal app builder with AI app and agent workflows for operations teams.

Agent Memory & State

• Letta (ex-MemGPT) - Platform for building stateful agents with advanced memory that learn and self-improve over time.
• Mem0 - Universal memory layer for AI agents. Persistent, multi-session memory across models and environments.
• Hindsight - State-of-the-art long-term memory for AI agents by Vectorize. Fully self-hosted, MIT-licensed, with integrations for LangChain, CrewAI, LlamaIndex, Vercel AI SDK, and more.

---

🔍 5. Retrieval-Augmented Generation (RAG) & Knowledge

Retrieval systems, vector databases, embedding models, and related tooling for RAG pipelines.

Vector Databases & Search Engines

• Chroma - Most popular open-source embedding database.
• Qdrant - High-performance vector search engine in Rust.
• Weaviate - GraphQL-native vector search engine.
• Milvus - Scalable cloud-native vector database.
• Faiss - Similarity search and clustering library for dense vectors with CPU and GPU implementations.
• LanceDB - Serverless vector DB optimized for multimodal data.
• Vespa - AI + Data platform with hybrid search (vector + keyword) and real-time indexing at scale. Battle-tested serving billions of queries daily.
• pgvector - PostgreSQL extension for vector similarity search.
• pgvectorscale - PostgreSQL extension for scalable vector search with DiskANN algorithm. Complements pgvector with significantly faster search and higher recall at large scale. PostgreSQL licensed.
• VectorChord - Scalable, fast, and disk-friendly vector search in Postgres. Successor to pgvecto.rs with production-grade performance and efficient storage. AGPL-3.0 licensed.
• Quickwit - Cloud-native search engine for observability. Open-source alternative to Datadog, Elasticsearch, Loki, and Tempo with native vector search support.
• Tantivy - Full-text search engine library inspired by Apache Lucene and written in Rust. Powers Quickwit and other production search systems.
• Manticore Search - Easy to use open source fast database for search. Good alternative to Elasticsearch with SQL-like interface and vector search capabilities.
• OpenSearch - Open-source distributed and RESTful search and analytics suite with native vector search. Enterprise-grade fork of Elasticsearch with k-NN plugin for semantic search at scale.
• Marqo - Multimodal vector search for text, image, and structured data. End-to-end indexing and search with built-in embedding models. Apache 2.0 licensed.
• Vald - Highly scalable distributed vector search engine. Cloud-native architecture with automatic indexing, horizontal scaling, and multiple ANN algorithm support. Apache 2.0 licensed.
• hnswlib - Header-only C++ library for fast approximate nearest neighbors with Python bindings. Supports CRUD operations and concurrent read/write - unique among ANN libraries. Powers many production vector databases. Apache 2.0 licensed.
• sqlite-vec - A vector search SQLite extension that runs anywhere. Extremely small, "fast enough" vector search written in pure C with no dependencies. Perfect for embedded and edge deployments. MIT/Apache-2.0 dual licensed.
• zvec - Lightweight, lightning-fast, in-process vector database from Alibaba. Built on Proxima (Alibaba's battle-tested vector search engine) for production-grade, low-latency similarity search. Apache 2.0 licensed.
• Meilisearch - Lightning-fast search engine API with AI-powered hybrid search. Features typo-tolerant full-text search combined with HNSW-based vector search for semantic retrieval. MIT licensed.
• Typesense - Open source alternative to Algolia + Pinecone. Fast, typo-tolerant, in-memory fuzzy search engine with native vector search capabilities. GPL-3.0 licensed.
• Elasticsearch - Distributed search and analytics engine with native k-NN vector search, hybrid search, and dense vector indexing. Industry-standard for full-text search now with powerful semantic search capabilities. AGPL-3.0/Elastic-2.0 dual licensed.
• Apache Solr - Mature Lucene-based search platform with dense vector search, filtering, faceting, and hybrid retrieval patterns for production search-heavy RAG systems.
• RediSearch - Full-text, secondary indexing, and vector similarity search for Redis deployments. Useful when retrieval needs low-latency Redis-native search.
• ParadeDB - Postgres-native search and analytics engine for full-text, faceted, and hybrid retrieval without moving data out of PostgreSQL.
• Orama - Lightweight search engine with full-text, vector, and hybrid search for browser, server, and edge applications.
• HelixDB - Graph-vector database for retrieval systems that need relationship traversal alongside semantic search.
• USearch - Fast single-file similarity search & clustering engine for vectors. Smaller and faster than FAISS with 20+ language bindings (C++, Python, JavaScript, Rust, Java, Go, etc.) and support for custom metrics. Apache 2.0 licensed.
• Voyager (Spotify) - Spotify's next-gen approximate nearest-neighbor search library for Python and Java. Up to 10x faster than Annoy with 4x less memory, designed for production use at billion-vector scale. Apache 2.0 licensed.
• Deep Lake - AI Data Runtime for Agents with serverless PostgreSQL and multimodal datalake. Store and search vectors, images, text, videos, and more with LangChain/LlamaIndex integrations. Used by Intel, Bayer, Yale, and Oxford. Apache 2.0 licensed.
• DiskANN (Microsoft) - Graph-structured indices for scalable, fast, fresh and filtered approximate nearest neighbor search. Handles billion-vector datasets on a single node with SSD-based indexing. MIT licensed.
• SPTAG (Microsoft) - Distributed approximate nearest neighbor search library with high-quality vector index build and online serving toolkits. Powers Bing's vector search at trillion-vector scale. MIT licensed.
• nanoflann - C++11 header-only library for fast nearest neighbor search with KD-trees. Zero dependencies, single-file integration, and 2-3x faster than FLANN with modern C++. BSD licensed.
• NMSLIB - Non-Metric Space Library for efficient similarity search in generic non-metric spaces. Comprehensive toolkit for evaluating k-NN methods with support for exotic distance functions. Apache 2.0 licensed.
• Vearch - Cloud-native distributed vector database for AI-native applications. Efficient similarity search of embedding vectors with horizontal scaling and real-time indexing. Apache 2.0 licensed.
• JVector (DataStax) - The most advanced embedded vector search engine for Java. DiskANN-based algorithm for billion-scale vector search with efficient memory mapping. Apache 2.0 licensed.
• VectorDBBench (Zilliz) - Industry-standard benchmark suite for vector databases. Test and compare performance of Milvus, Zilliz Cloud, and other vector DBs with your own datasets. MIT licensed.

Embedding Models

• BGE (FlagEmbedding) - BAAI's best-in-class embedding family.
• E5 (Microsoft) - High-performance text embeddings for retrieval.
• FastEmbed (Qdrant) - Lightweight, fast Python library for embedding generation with ONNX Runtime. Supports text, sparse (SPLADE), and late-interaction (ColBERT) embeddings without GPU dependencies. Apache 2.0 licensed.
• EmbedAnything - Minimalist, highly performant multimodal embedding pipeline built in Rust. Memory-safe, modular, and production-ready for text, image, and audio embeddings with seamless vector DB integration. Apache 2.0 licensed.
• Text Embeddings Inference (Hugging Face) - Blazing fast inference solution for text embedding models. High-performance extraction with token-based dynamic batching, Flash Attention, and support for FlagEmbedding, E5, GTE, and more. OpenAI-compatible API with Docker deployment. Apache 2.0 licensed.

Embedding Benchmarks

• MTEB - Massive Text Embedding Benchmark covering 1000+ languages and diverse tasks. The industry standard for evaluating and comparing embedding models.

RAG Frameworks & Advanced Retrieval Tools

• EmbedChain - Universal memory layer for AI agents. Simple API to create RAG applications over any dataset with support for multiple vector stores, embedding models, and LLM providers. Apache 2.0 licensed.
• LlamaIndex - Full-featured RAG pipeline with advanced indexing.
• R2R (SciPhi) - Production-ready agentic RAG engine with RESTful API. Features hybrid search, knowledge graphs, multimodal RAG, and deep research capabilities. MIT licensed.
• Haystack - End-to-end NLP and RAG framework.
• RAGFlow - Deep-document-understanding RAG engine.
• GraphRAG (Microsoft) - Knowledge-graph-based RAG.
• Docling - Document processing toolkit for turning PDFs and other files into structured data for GenAI workflows.
• Unstructured - Best-in-class document preprocessing.
• MinerU - High-accuracy document parsing for LLM and RAG workflows. Converts PDFs, Word, PPTs, and images into structured Markdown/JSON with VLM+OCR dual engine.
• Marker - Fast, accurate PDF-to-markdown converter with table extraction, equation handling, and optional LLM enhancement for RAG pipelines.
• ColPali / ColQwen - Vision-language models for document retrieval.
• LightRAG - Graph-based RAG with dual-level retrieval system. Simple and fast with comprehensive knowledge discovery (EMNLP 2025).
• RAG-Anything - All-in-One Multimodal RAG system for seamless processing of text, images, tables, and equations. Built on LightRAG.
• RAGLite (Superlinear) - Python toolkit for RAG with DuckDB or PostgreSQL. Lightweight, efficient retrieval-augmented generation without heavy dependencies. MPL 2.0 licensed.
• GPT-RAG (Azure) - Enterprise RAG pattern for Azure OpenAI at scale. Secure, production-ready architecture using Azure Cognitive Search and Azure OpenAI LLMs for ChatGPT-style Q&A experiences. MIT licensed.
• LangChain4j - Java library for integrating LLMs into Java applications. Implements RAG, tool calling (including MCP support), and agents with seamless integration into enterprise Java frameworks like Spring Boot. Apache 2.0 licensed.
• Kernel Memory (Microsoft) - Memory solution for users, teams, and applications. RAG pipelines with document ingestion, vector indexing, and natural language querying with citations. Supports multiple LLM providers and vector stores. MIT licensed.
• txtai - All-in-one AI framework for semantic search, LLM orchestration and language model workflows. Embeddings database with customizable pipelines.
• Infinity (Embeddings Server) - High-throughput, low-latency serving engine for text-embeddings, reranking, CLIP, and ColPali. OpenAI-compatible API.
• FlashRAG - Efficient toolkit for RAG research with 40+ retrieval and reranking models, 20+ benchmark datasets, and optimized evaluation pipelines (WWW 2025 Resource). MIT licensed.
• DocsGPT - Private AI platform for building intelligent agents and assistants with enterprise search. Features Agent Builder, deep research tools, multi-format document analysis, and multi-model support. MIT licensed.
• llmware - Unified framework for building enterprise RAG pipelines with small, specialized models. Optimized for AI PC and local deployment with 300+ models in catalog. Apache 2.0 licensed.
• AutoFlow - Graph RAG-based conversational knowledge base tool built on TiDB Vector and LlamaIndex. Features Perplexity-style search with built-in website crawler. Apache 2.0 licensed.
• rerankers (Answer.AI) - Lightweight unified API for all common reranking and cross-encoder models. Supports RankGPT, ColBERT, FlashRank, and API-based rerankers with a dependency-free core. Apache 2.0 licensed.
• KAG (OpenSPG) - Knowledge Augmented Generation framework for logical reasoning and factual Q&A in professional domains. Builds on OpenSPG knowledge graph engine to overcome traditional RAG vector similarity limitations. Supports multi-hop reasoning with schema-constrained knowledge construction. Apache 2.0 licensed.
• Chonkie - Lightweight document chunking library for fast, efficient RAG pipelines. Memory-safe with multiple chunking strategies (semantic, token, recursive) and direct vector DB integration. MIT licensed.
• PageIndex (VectifyAI) - Vectorless, reasoning-based RAG framework using document index structure. Achieves high accuracy without vector databases through intelligent context engineering and reasoning-based retrieval. MIT licensed.
• Kotaemon (Cinnamon) - Open-source RAG-based tool for chatting with your documents. Hybrid RAG pipeline with full-text and vector retriever, re-ranking, and multi-modal capabilities. Clean Gradio-based UI with support for local and API-based LLMs. Apache 2.0 licensed.
• Reader (Jina AI) - Convert any URL to LLM-friendly input with a simple prefix (r.jina.ai). Free service that extracts article content, removes clutter, and returns clean Markdown for RAG and agentic workflows. Apache 2.0 licensed.
• UltraRAG (OpenBMB) - First lightweight RAG framework based on Model Context Protocol (MCP) architecture. Low-code RAG pipeline builder with comprehensive evaluation system and DeepResearch capabilities. From Tsinghua THUNLP, NEUIR, OpenBMB, and AI9stars. Apache 2.0 licensed.
• Semantic Router - Superfast AI decision-making layer for LLMs and agents. Uses semantic vector space to route requests using semantic meaning rather than waiting for slow LLM generations. Cuts routing time from seconds to milliseconds. MIT licensed.
• Neurite - Fractal Graph-of-Thought mind-mapping for AI agents, web-links, notes, and code. Rhizomatic workspace blending chaos theory, graph theory, and fractal logic for creative thinking and RAG workflows. MIT licensed.
• Verba - The Golden RAGtriever - an end-to-end, streamlined, user-friendly RAG interface powered by Weaviate. Chat with your documents using hybrid search, multiple chunking strategies, and support for various LLM providers. BSD-3-Clause licensed.
• Pathway - Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG. Features 350+ connectors with always-in-sync data from SharePoint, Google Drive, S3, Kafka, PostgreSQL and more. BSL 1.1 license (becomes Apache 2.0 after 4 years).
• Infinity (AI Database) - AI-native database built for LLM applications with incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text. Powers RAGFlow's document engine. Apache 2.0 licensed.
• PrivateGPT - Private document Q&A project for local and offline RAG workflows where data stays inside the user's environment.
• FastGPT - Knowledge-base platform with RAG retrieval, document processing, visual AI workflows, and self-hosted deployment options.
• MaxKB - Self-hostable knowledge-base and agent platform for document ingestion, RAG pipelines, and enterprise assistant workflows.
• DB-GPT - Self-hosted AI data assistant for private knowledge, database-aware conversations, and data-heavy RAG workflows.
• localGPT - Local document-chat project for private, on-device Q&A over files without sending data to external APIs.
• SurfSense - Privacy-focused NotebookLM-style workspace for teams to search, organize, and query knowledge with self-hosted RAG.
• Morphik - Open-source multimodal RAG framework for building AI apps over private knowledge. Handles text, images, and documents with built-in embedding generation and vector search. MIT licensed.

Knowledge Graphs for RAG

• Graphiti - Build real-time temporal knowledge graphs for AI agents. Tracks how facts change over time with provenance to source data. Supports prescribed and learned ontology for evolving real-world data. Apache 2.0 licensed.

Web Data Ingestion

• Crawl4AI - LLM-friendly web crawler that turns websites into clean Markdown for RAG and agentic workflows.
• Lightpanda - Machine-first headless browser in Zig; rendering-free and ultra-lightweight for AI agent browsing.
• Paperless-AI - Automated document analyzer for Paperless-ngx with RAG-powered semantic search across your document archive.
• Firecrawl - Web Data API for AI - search, scrape, and interact with the web at scale. Clean markdown/JSON output with proxy rotation and JS-blocking handled automatically.

Document Conversion & Preprocessing

• MarkItDown (Microsoft) - Python tool for converting files and office documents to Markdown. Supports PDF, PowerPoint, Word, Excel, images, audio, HTML, and more with OCR and transcription capabilities. MIT licensed.
• OmniParse - Ingest and parse any unstructured data into structured, actionable data optimized for GenAI applications. Supports documents, tables, images, videos, audio, and web pages with local deployment on T4 GPU. GPL-3.0 licensed.
• DocETL (UC Berkeley) - Agentic LLM-powered data processing and ETL system for complex document processing. Query rewriting and evaluation for unstructured data analysis with 80% higher accuracy than baselines. MIT licensed.

LLM Application Frameworks

• aisuite - Simple, unified interface to multiple Generative AI providers. Use OpenAI, Anthropic, Google, and 10+ other providers with a standardized API similar to OpenAI's. Switch between models or providers with a single line of code. MIT licensed.
• Spring AI - Application framework for AI engineering in the Spring ecosystem. Unified API for LLMs, vector stores, and embedding models with seamless integration into Spring Boot applications. Supports RAG, tool calling, and structured outputs. Apache 2.0 licensed.
• Rig - Rust library for building scalable, modular LLM-powered applications. Type-safe agent framework with unified LLM interface, built-in vector store integrations, and ergonomic abstractions for production AI systems. MIT licensed.
• Ax - TypeScript framework for building reliable AI applications. "Official" DSPy-inspired framework for TypeScript with type-safe LLM interactions, chain-of-thought reasoning, and structured output validation. Apache 2.0 licensed.
• Genkit - Open-source framework for building full-stack AI-powered applications in JavaScript, Go, and Python. Built and used in production by Google's Firebase. Unified interface for integrating AI models from multiple providers with built-in RAG, tool calling, structured outputs, and developer tools. Apache 2.0 licensed.
• ContextGem - Effortless LLM extraction framework for documents. Powerful abstractions for building extraction workflows with automated dynamic prompts, data modeling, validation, and precise reference mapping. Apache 2.0 licensed.
• Eino - The ultimate LLM/AI application development framework in Go. Drawing from LangChain and Google ADK, designed to follow Go conventions with composable components for chains, agents, and workflows. Apache 2.0 licensed.
• ruby_llm - One beautiful Ruby API for OpenAI, Anthropic, Gemini, Bedrock, Azure, OpenRouter, DeepSeek, Ollama, and 15+ providers. Agents, Chat, Vision, Audio, PDF, Images, Embeddings, Tools, Streaming and Rails integration. MIT licensed.
• LangChain.rb - Build LLM-powered applications in Ruby. Idiomatic Ruby library for building AI applications with support for multiple LLM providers, vector stores, and RAG pipelines. MIT licensed.

---

🎨 6. Generative Media Tools

Open-source models and applications for image, video, audio, and 3D generation and editing.

Image Generation & Editing

• ComfyUI - Node-based visual workflow editor for Stable Diffusion, FLUX, etc.
• Stable Diffusion WebUI Forge - Neo - Actively maintained Forge-based Stable Diffusion web UI with the familiar extension-driven workflow.
• Fooocus - Midjourney-style UI with beautiful out-of-the-box results.
• Diffusers - PyTorch library for diffusion pipelines spanning image, video, and audio generation.
• InvokeAI - Full-featured creative studio.
• PowerPaint (OpenMMLab) - Versatile image inpainting model supporting text-guided inpainting, object removal, and outpainting (ECCV 2024).
• SD.Next - All-in-one WebUI for AI generative image and video creation with multi-platform support, SDNQ quantization, and balanced CPU/GPU memory offload.
• Qwen-Image (Alibaba) - 20B MMDiT image foundation model with state-of-the-art complex text rendering and precise image editing. Strong performance in Chinese text generation. Apache 2.0 licensed.
• Upscayl - Free and open-source AI image upscaler for Linux, macOS, and Windows. Uses Real-ESRGAN and Vulkan architecture to enhance images by reconstructing high-resolution details. Cross-platform desktop app with batch processing. AGPL-3.0 licensed.
• Z-Image (Tongyi) - Powerful and efficient image generation model family with 6B parameters. Includes Z-Image-Turbo for sub-second inference and Z-Image-Omni-Base for both generation and editing. Strong bilingual text rendering and instruction adherence. Apache 2.0 licensed.
• Krita AI Diffusion - Streamlined AI image generation plugin for Krita. Inpaint and outpaint with optional text prompt, no tweaking required. Integrates ComfyUI backend for professional digital painting workflows. GPL-3.0 licensed.

Face Swap & Deepfake

• Deep-Live-Cam - Real-time face swap and one-click video deepfake with only a single image. High-quality face swapping for live video streaming and content creation. AGPL-3.0 licensed.

Portrait Animation

• EchoMimic (Ant Group) - Lifelike audio-driven portrait animations through editable landmark conditioning. High-quality talking head generation with precise lip synchronization and natural head movements. AAAI 2025. Apache 2.0 licensed.

Video Generation

• Wan2.2 (Alibaba) - Leading open Mixture-of-Experts text-to-video model.
• HunyuanVideo (Tencent) - 13B-parameter systematic video generation framework. Leading quality among open models.
• SkyReels V2/V3 (Skywork) - First open-source infinite-length film generative model using AutoRegressive Diffusion-Forcing.
• Mochi 1 (Genmo) - 10B-parameter open video model.
• LTX-Video (Lightricks) - Fast native 4K video generation.
• Stable Video Diffusion (Stability AI) - Official image-to-video and text-to-video implementation within Stability AI's generative models repository.
• Open-Sora-Plan (PKU-YuanGroup) - Reproduction of Sora with full open-source pipeline for text-to-video generation. MIT licensed.
• Open-Sora (HPC-AI Tech) - Fully open-source video generation with 11B model achieving on-par performance with HunyuanVideo. Complete training pipeline for $200K. Apache 2.0 licensed.
• Helios (PKU-YuanGroup) - Efficient long-video generation framework with 24GB VRAM support for up to 10,000 frames (5+ minutes) and 1280×768 resolution. Apache 2.0 licensed.
• WhisperLive - Nearly-live implementation of OpenAI's Whisper for real-time speech-to-text transcription. Supports faster-whisper, tensorrt, and openvino backends with WebSocket streaming. MIT licensed.

Audio / Music / Voice Generation

• AudioCraft / MusicGen (Meta) - Controllable text-to-music and audio models.
• ACE-Step 1.5 - Local-first music generation model with broad hardware support across Mac, AMD, Intel, and CUDA devices.
• Fish Speech - Zero-shot TTS and voice cloning.
• CosyVoice 2 - Natural multilingual TTS with emotional control.
• OuteTTS - High-quality open TTS.
• Amphion - Comprehensive toolkit for Audio, Music, and Speech Generation (9.7K stars).
• Stable Audio Tools - Stability AI's open-source audio and music generative models. Latent diffusion model for generating audio conditioned on metadata and timing, providing faster inference times and creative control for sound effects and music production. MIT licensed.
• GPT-SoVITS - Few-shot voice cloning with just 1 minute of voice data. Combines GPT and SoVITS architectures for high-quality TTS with cross-lingual support and emotional expression. MIT licensed.
• Chatterbox - State-of-the-art open-source TTS with natural prosody and emotional control. Zero-shot voice cloning with high speaker similarity. MIT licensed.
• F5-TTS - Fully non-autoregressive text-to-speech using flow matching with diffusion transformer (DiT). Fast, high-quality synthesis with zero-shot voice cloning capabilities. MIT licensed.
• Real-Time Voice Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time. SV2TTS implementation with speaker encoder and vocoder for instant voice synthesis. MIT licensed.

3D & Creative Tools

• Trellis (Microsoft) - Structured 3D latents for high-quality generation.
• gsplat (3D Gaussian Splatting tools) - High-performance 3D Gaussian Splatting library.
• LichtFeld-Studio - Native application for training, editing, and exporting 3D Gaussian Splatting scenes with MCMC optimization and timelapse generation. GPL-3.0 licensed.
• OpenSplat - Production-grade, portable implementation of 3D Gaussian Splatting with CPU/GPU support for Windows, Mac, and Linux. Creates 3D scenes from camera poses and sparse points. AGPL-3.0 licensed.

---

🛠️ 7. Training & Fine-tuning Ecosystem

Tools for model training, fine-tuning, synthetic data generation, and distributed training.

Full Training Frameworks

• Oumi - Fully open-source platform for the complete foundation model lifecycle - from data preparation and training to evaluation and deployment. Supports 100+ models with 200+ recipes for fine-tuning gpt-oss, Qwen3, DeepSeek-R1, and more. Apache 2.0 licensed.
• LLaMA-Factory - One-stop unified framework for SFT, DPO, ORPO, KTO with web UI.
• Axolotl - YAML-driven full pipeline for SFT, DPO, GRPO.
• ms-swift - Unified training framework for 600+ LLMs and 300+ MLLMs with CPT/SFT/DPO/GRPO (AAAI 2025).
• Unsloth - 2× faster, 70% less memory fine-tuning.
• LitGPT - Clean from-scratch implementations of 20+ LLMs.
• LLM Foundry - Databricks' training framework for composable LLM training with StreamingDataset and Composer.
• torchtune - PyTorch-native library for post-training, fine-tuning, and experimentation with LLMs.
• kohya_ss - Gradio-based GUI and CLI for training Stable Diffusion models (LoRA, Dreambooth, fine-tuning, SDXL). Provides accessible interface to Kohya's powerful training scripts.
• TRL (Transformers Reinforcement Learning) - Official library for RLHF, SFT, DPO, ORPO.
• verl - Volcano Engine Reinforcement Learning for LLMs with PPO, GRPO, REINFORCE++, DAPO (EuroSys 2025).
• NeMo-RL - Scalable toolkit for efficient model reinforcement with DTensor and Megatron backends.
• OpenRLHF - Easy-to-use, scalable RLHF framework based on Ray. Supports PPO, GRPO, REINFORCE++, DAPO with vLLM integration and async training. Apache 2.0 licensed.
• LMFlow - Extensible toolkit for finetuning and inference of large foundation models. Features RAFT alignment algorithm and comprehensive model support. Apache 2.0 licensed.
• XTuner - A next-generation training engine built for ultra-large MoE models with efficient QLoRA and full-parameter fine-tuning. Apache 2.0 licensed.
• Ludwig - Low-code framework for building custom LLMs and deep neural networks. Declarative YAML configuration for training state-of-the-art models with PEFT/LoRA, 4-bit quantization, distributed training via HuggingFace Accelerate, and native Kubernetes support. Linux Foundation AI project. Apache 2.0 licensed.
• nanoGPT (Andrej Karpathy) - The simplest, fastest repository for training/finetuning medium-sized GPTs. Clean, minimal, and hackable codebase for understanding transformer training from scratch. MIT licensed.
• TorchTitan (PyTorch) - PyTorch native platform for training generative AI models at scale. Showcases 4D parallelism (FSDP, tensor, pipeline, context) for LLM pretraining with 65%+ speedups over optimized baselines. BSD-3-Clause licensed.
• VeOmni (ByteDance) - Versatile framework for both single- and multi-modal pre-training and post-training. Model-centric distributed recipe zoo supporting text, vision, audio, and video models with unified training interface. Apache 2.0 licensed.
• H2O LLM Studio - No-code GUI framework for fine-tuning LLMs. Streamlined interface for SFT, reward modeling, and model deployment. Apache 2.0 licensed.
• TinyZero - Minimal reproduction of DeepSeek R1-Zero for countdown and multiplication tasks. Clean, accessible implementation for understanding RL-based reasoning training. Apache 2.0 licensed.
• PRIME-RL - Agentic RL Training at Scale from Prime Intellect. Framework for large-scale reinforcement learning capable of scaling to 1000+ GPUs with fully asynchronous RL, FSDP2 training, and vLLM inference. Apache 2.0 licensed.
• slime - LLM post-training framework for RL Scaling from THUDM. Supports SFT and RL training with multi-turn compilation feedback, powering projects like TritonForge for automated GPU kernel generation. Apache 2.0 licensed.
• rLLM - Democratizing Reinforcement Learning for LLMs. Framework for training AI agents with RL featuring near-zero code changes, CLI-first workflow, and 50+ built-in benchmarks. Supports GRPO, REINFORCE, RLOO with verl and tinker backends. Apache 2.0 licensed.
• EasyR1 - Efficient, scalable, multi-modality RL training framework based on veRL. Extends veRL to support vision-language models with GRPO algorithm for efficient RL training. Apache 2.0 licensed.
• simpleRL-reason - Simple reinforcement learning recipe to improve models' reasoning abilities. Rule-based reward with GSM8K/Math datasets, extending from OpenRLHF. MIT licensed.
• align-anything - Training all-modality models with feedback. Supports RLHF, DPO, and alignment fine-tuning for text, image, audio, and video models with seamless Slurm cluster integration. Apache 2.0 licensed.
• LeRobot - Making AI for robotics more accessible with end-to-end learning. State-of-the-art approaches for imitation learning and reinforcement learning with pretrained models, datasets, and simulated environments. Apache 2.0 licensed.
• AI-Toolkit - Ultimate training toolkit for finetuning diffusion models. Easy-to-use all-in-one training suite supporting FLUX.1, FLUX.2, Stable Diffusion, and video models with both GUI and CLI interfaces. Consumer-grade hardware friendly with comprehensive LoRA and full fine-tuning support. MIT licensed.
• OneTrainer - One-stop solution for all your Diffusion training needs. Supports FLUX, Stable Diffusion 1.5/2.x/3.x/SDXL, Würstchen, PixArt, Hunyuan Video and more. Features full fine-tuning, LoRA, embeddings, masked training, automatic backups, and TensorBoard integration. GPL-3.0 licensed.
• FluxGym - Dead simple FLUX LoRA training UI with LOW VRAM support (12GB/16GB/20GB). WebUI forked from AI-Toolkit with backend powered by Kohya Scripts. Combines simplicity of Gradio interface with flexibility of Kohya's powerful training scripts. GPL-3.0 licensed.
• MiniMind - Train a 64M-parameter LLM from scratch in just 2 hours for $3. Complete from-scratch implementation covering MoE, data cleaning, pretraining, SFT, LoRA, RLHF (DPO/PPO/GRPO), tool use, and model distillation. All core algorithms implemented in pure PyTorch without high-level abstractions. Educational framework for understanding LLM internals. Apache 2.0 licensed.
• FastChat - Open platform for training, serving, and evaluating large language model chatbots. Powers Chatbot Arena (lmarena.ai) serving 10M+ requests for 70+ LLMs. Includes training code for Vicuna, MT-Bench evaluation, and distributed multi-model serving with OpenAI-compatible APIs. Apache 2.0 licensed.
• PaddleNLP - Easy-to-use and powerful LLM library built on Baidu's PaddlePaddle framework. Supports 100+ models with efficient training, compression, and high-performance inference on diverse hardware. Features RsLoRA+ algorithm, DeepSeek V3/R1 support with FP8/INT8 quantization, and unified checkpointing. Apache 2.0 licensed.

LoRA / PEFT Tools

• PEFT (Parameter-Efficient Fine-Tuning) - Official library with LoRA, QLoRA, DoRA, etc.
• Liger Kernel - Ultra-fast custom kernels for training speedup.
• MergeKit - Advanced model merging tools.

Synthetic Data Generation

• distilabel - End-to-end pipeline for synthetic instruction data.
• Data-Juicer - High-performance data processing for LLM training.
• Argilla - Open-source data labeling + synthetic data platform.
• SDV (Synthetic Data Vault) - High-fidelity tabular and relational synthetic data.
• DataTrove (Hugging Face) - Platform-agnostic data processing pipelines for LLM training at scale. Handles filtering, deduplication, and tokenization on local machines or SLURM clusters.
• Bespoke Curator - Synthetic data curation for post-training and structured data extraction. Makes it easy to build pipelines around LLMs with batching and progress tracking. Apache 2.0 licensed.
• SDG (Harbin Institute) - Specialized framework for generating high-quality structured tabular synthetic data with CTGAN models supporting billion-level data processing. Apache 2.0 licensed.

Distributed Training

• DeepSpeed - Extreme-scale training optimizations.
• Colossal-AI - Unified system for 100B+ models.
• Megatron-LM - Distributed training framework and reference codebase for large transformer models at scale.
• Composer - MosaicML's PyTorch library for scalable, efficient neural network training with algorithmic speedups.
• Ray Train - Scalable distributed training.
• Nanotron (Hugging Face) - Minimalistic 3D-parallelism LLM pretraining with tensor, pipeline, and data parallelism. Designed for simplicity and speed.
• veScale (ByteDance) - Hyperscale PyTorch distributed training with flexible FSDP implementation for LLMs and RL training at scale.
• GPT-NeoX (EleutherAI) - Production-grade distributed training framework for large autoregressive transformers, powering models like GPT-J and GPT-NeoX-20B.
• RLinf - Scalable open-source RL infrastructure for post-training foundation models via reinforcement learning. Features M2Flow paradigm for embodied AI and agentic workflows with real-world robotics integrations. Apache 2.0 licensed.
• dstack - Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal. MPL-2.0 licensed.
• Streaming (MosaicML) - High-performance data streaming library for efficient neural network training. Streams training data from cloud storage (S3, GCS, Azure) with local caching and deterministic shuffling. Apache 2.0 licensed.

Model Quantization & Optimization

• LLM Compressor (vLLM) - Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM. Supports GPTQ, AWQ, SmoothQuant, AutoRound, and FP8/INT8 quantization with seamless Hugging Face integration.
• NVIDIA Model Optimizer - Unified library of SOTA model optimization techniques including quantization, pruning, distillation, and speculative decoding. Compresses deep learning models for deployment with TensorRT-LLM, TensorRT, and vLLM to optimize inference speed across NVIDIA hardware.

---

📊 8. MLOps / LLMOps & Production

Tooling for tracking, deploying, monitoring, and operating AI systems in production.

Experiment Tracking & Versioning

• MLflow - End-to-end open platform for the ML/LLM lifecycle.
• DVC (Data Version Control) - Git-like versioning for data and models.
• ClearML - Open-source platform for experiment tracking, orchestration, data management, and model serving.
• Weights & Biases Weave - Open-source tracing and experiment tracking.
• Aim - Self-hosted ML experiment tracker designed to handle 10,000s of training runs with performant UI and SDK for programmatic access. Apache 2.0 licensed.
• Feast - Open source feature store for ML. Manages offline/online feature storage with point-in-time correctness to prevent data leakage. Apache 2.0 licensed.
• OpenLineage - Open standard for lineage metadata collection designed to instrument jobs as they run. Defines a generic model of run, job, and dataset entities for consistent data lineage tracking. Apache 2.0 licensed.
• Marquez - LF AI & Data Foundation Graduated project for metadata collection, aggregation, and visualization. Maintains provenance of how datasets are consumed and produced with global visibility into job runtime and dataset lifecycle management. Integrates with OpenLineage. Apache 2.0 licensed.

Model Hubs & Registries

• Civitai - Open-source AI model hub and community platform for sharing and discovering generative AI models, with focus on image generation models. Features model versioning, reviews, and integrated inference. Apache 2.0 licensed.
• Hugging Face Hub - Official Python client for the Hugging Face Hub. Download, upload, and manage 1M+ open-source ML models and datasets programmatically. The de facto standard for model sharing and distribution. Apache 2.0 licensed.
• ModelScope - Model-as-a-Service platform bringing together 700+ state-of-the-art ML models from the AI community. Covers NLP, CV, Audio, Multi-modality, and AI for Science with streamlined model inference, fine-tuning and evaluation. Apache 2.0 licensed.
• OpenVINO Open Model Zoo - Pre-trained deep learning models and demos optimized for Intel hardware. 200+ public pre-trained models for vision, speech, and NLP with benchmarking tools and accuracy metrics. Apache 2.0 licensed.
• ONNX Model Zoo - Collection of pre-trained, state-of-the-art models in the ONNX format. 80+ models spanning vision, NLP, and audio with validation data and reference implementations. Apache 2.0 licensed.
• Transformers.js - State-of-the-art Machine Learning for the web. Run Hugging Face Transformers directly in your browser with no server needed. Supports 1000+ models including BERT, GPT-2, T5, and more via ONNX Runtime Web. Apache 2.0 licensed.
• DJL (Deep Java Library) - Engine-agnostic deep learning framework for Java with built-in model zoo. Load and run PyTorch, TensorFlow, MXNet, and ONNX models with a unified API. Includes 80+ pre-trained models for CV and NLP. Apache 2.0 licensed.
• PaddleSeg - Easy-to-use image segmentation library with awesome pre-trained model zoo. Supports semantic segmentation, interactive segmentation, panoptic segmentation, image matting, and 3D segmentation with 200+ pre-trained models. Apache 2.0 licensed.
• TorchVision Models - PyTorch's official computer vision library with 50+ pre-trained model architectures including ResNet, EfficientNet, Vision Transformers (ViT), ConvNeXt, and more. The de facto standard model zoo for PyTorch computer vision. BSD-3-Clause licensed.
• TensorFlow Model Garden - Official TensorFlow repository of state-of-the-art (SOTA) models and modeling solutions. Contains reference implementations for BERT, ResNet, Transformer, and many more with pre-trained weights and training scripts. Apache 2.0 licensed.
• MLflow - Open-source AI engineering platform with comprehensive Model Registry for managing the full lifecycle of ML models. 20K+ GitHub stars, 60M+ monthly downloads. Features experiment tracking, model versioning, deployment management, and collaborative model governance. Apache 2.0 licensed.
• PINTO Model Zoo - Repository for storing models inter-converted between various frameworks. Supports TensorFlow, PyTorch, ONNX, OpenVINO, TFJS, TFTRT, TensorFlowLite (Float32/16/INT8), EdgeTPU, and CoreML. 4,100+ stars with extensive model conversion tools for edge deployment. MIT licensed.
• Cerebras Model Zoo - Collection of deep learning models and utilities optimized for Cerebras hardware. Includes reference implementations for Llama, Mixtral, DINOv2, and Llava with configuration files, data preprocessing tools, and checkpoint converters. 1,150+ stars. Apache 2.0 licensed.
• PaddleClas - Comprehensive image recognition and classification toolkit with rich model zoo. 5,800+ stars featuring 24 series of classification networks, 122 pretrained models, and end-to-end image recognition systems including PP-ShiTuV2. Apache 2.0 licensed.

Model Packaging & Deployment

• Cog (Replicate) - Containerize and deploy ML models with production-grade inference servers. Packages models into standardized containers with automatic API generation, GPU support, and one-command deployment. Powers thousands of production AI models on Replicate. Apache 2.0 licensed.

Deployment & Orchestration

• BentoML - Unified framework to build, ship, and scale AI apps.
• Ray Serve - Scalable model serving library.
• ZenML - Pipeline and orchestration framework for taking ML and LLM systems from development to production.
• Kubeflow - Kubernetes-native ML/LLM platform.
• KServe - Kubernetes-based model serving.
• Seldon Core - MLOps and LLMOps framework for deploying, managing and scaling AI systems in Kubernetes. Standardized deployment across model types with autoscaling, multi-model serving, and A/B experiments.
• Metaflow - Netflix's ML platform for building and managing real-world AI systems. Powers thousands of projects at Netflix, Amazon, and DoorDash. Apache 2.0 licensed.
• Flyte - Kubernetes-native workflow orchestration platform for AI/ML pipelines. Dynamic, resilient orchestration with strong type safety and reproducibility. Used by Lyft, Spotify, and Gojek. Apache 2.0 licensed.
• Prefect - Workflow orchestration framework for building resilient data and ML pipelines. Python-native with modern observability and 200+ integrations. Apache 2.0 licensed.
• Dagster - Cloud-native orchestration platform for developing and maintaining data assets including ML models. Declarative programming model with integrated lineage and observability. Apache 2.0 licensed.
• Kubeflow Pipelines - Machine Learning Pipelines for Kubeflow. Platform for building and deploying portable, scalable ML workflows using Kubernetes and Argo. Apache 2.0 licensed.
• Argo Workflows - CNCF graduated container-native workflow engine for orchestrating parallel jobs on Kubernetes. Powers Kubeflow Pipelines and widely used for ML/data processing at scale. Apache 2.0 licensed.
• MLRun - Open-source AI orchestration platform for quickly building and managing continuous ML and generative AI applications across their lifecycle. Automates data preparation, model tuning, and deployment. Apache 2.0 licensed.
• Kestra - Event-driven orchestration and scheduling platform for mission-critical workflows. Infrastructure-as-Code approach with declarative YAML, Git version control integration, and hundreds of plugins for data pipelines and ML workflows. Apache 2.0 licensed.
• KitOps - CNCF open source DevOps tool for packaging, versioning, and securely sharing AI/ML models, datasets, code, and configuration. Packages everything into OCI artifacts stored in existing container registries. Apache 2.0 licensed.
• Polyaxon - MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle. Reproducible and scalable machine learning workflows on Kubernetes with experiment tracking, model management, and pipeline orchestration. Apache 2.0 licensed.
• Netflix Maestro - Netflix's next-generation workflow orchestrator for data and ML pipelines at massive scale. Highly scalable and flexible scheduler designed to handle millions of workflows across thousands of nodes. Apache 2.0 licensed.
• HAMi - Heterogeneous GPU Sharing on Kubernetes. CNCF sandbox project providing GPU virtualization, slicing, and scheduling for efficient AI workload management across heterogeneous accelerators (GPUs, NPUs, MLUs). Apache 2.0 licensed.
• NVIDIA KAI Scheduler - Kubernetes-native GPU scheduler for AI workloads at large scale. Originally developed by Run:ai, now open-sourced by NVIDIA. Optimizes GPU resource allocation with dynamic allocation and efficient queue management. Apache 2.0 licensed.
• NVIDIA DeepOps - Infrastructure automation tools for building GPU clusters with Kubernetes and Slurm. Deploys multi-node GPU clusters with monitoring, logging, and storage for AI/HPC workloads. BSD-3-Clause licensed.
• SkyPilot - Run, manage, and scale AI workloads on any AI infrastructure. Unified interface to access and manage compute across Kubernetes, Slurm, and 20+ cloud providers. Used by Shopify and research institutions for training and inference. Apache 2.0 licensed.
• Volcano - Cloud-native batch scheduling system for compute-intensive workloads. CNCF incubating project with gang scheduling, job dependency management, and topology-aware scheduling for AI/ML and deep learning. Apache 2.0 licensed.
• Apache YuniKorn - Kubernetes resource scheduler for batch, data, and ML workloads. Provides hierarchical resource queues, multi-tenancy fairness, and gang scheduling for big data and machine learning applications. Apache 2.0 licensed.
• Kueue - Kubernetes-native job queueing system for batch, HPC, AI/ML, and similar applications. Cloud-native job queueing with resource flavor fungibility, fair sharing, cohorts, and preemption policies. Integrates with Kubeflow, Ray, and JobSet. Apache 2.0 licensed.

Feature Engineering & Data Preparation

• Featuretools - Open-source Python library for automated feature engineering. Transforms transactional and relational datasets into feature matrices for machine learning using Deep Feature Synthesis with reusable primitives. BSD-3-Clause licensed.
• Kedro - Toolbox for production-ready data science. Uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular. Apache 2.0 licensed.
• Feature-engine - Python library with multiple transformers to engineer and select features for machine learning models. Scikit-learn compatible with fit() and transform() methods for encoding, imputation, variable transformation, and feature selection. BSD-3-Clause licensed.
• NVTabular - GPU-accelerated feature engineering and preprocessing library for tabular data. Manipulates terabyte-scale datasets to train deep learning recommender systems. Component of NVIDIA Merlin framework. Apache 2.0 licensed.
• OpenMLDB - Open-source machine learning database providing a feature platform for consistent features between training and inference. Real-time relational data feature computation system for online ML applications. Apache 2.0 licensed.

Monitoring, Evaluation & Observability

• Langfuse - #1 open-source LLM observability platform.
• Phoenix (Arize) - AI observability & evaluation platform.
• Evidently - ML & LLM monitoring framework.
• Deepchecks - Holistic validation and testing suite for ML models and data. Continuous validation from research to production with 50+ built-in checks for data integrity, distribution drift, and model performance.
• Opik (Comet) - Production-ready LLM evaluation platform.
• LiteLLM - AI Gateway to call 100+ LLM APIs in OpenAI format with unified cost tracking, guardrails, load balancing, and logging.
• OpenLIT - OpenTelemetry-native LLM observability platform with GPU monitoring, evaluations, prompt management, and guardrails.
• OpenLLMetry (Traceloop) - Open-source observability for GenAI/LLM applications based on OpenTelemetry with 25+ integration backends.
• Agenta - Open-source LLMOps platform combining prompt playground, prompt management, LLM evaluation, and observability.
• Latitude - Open-source agent engineering platform with prompt management, evaluations, and optimization. Features prompt playground, LLM-as-judge evals, and GEPA prompt optimizer for production LLM features. LGPL-3.0 licensed.
• Helicone - Open-source LLM observability with request logging, caching, rate limiting, and cost analytics.
• Giskard - Open-source evaluation and testing library for LLM agents. Red teaming, vulnerability scanning, RAG evaluation, and safety testing with modular architecture. Apache 2.0 licensed.
• Portkey Gateway - Blazing fast AI Gateway to route 200+ LLMs with unified API. Integrated guardrails, load balancing, fallbacks, and cost tracking. MIT licensed.
• TensorZero - Open-source LLMOps platform unifying LLM gateway, observability, evaluation, and experimentation. Production-grade with sub-1ms latency, used by Fortune 10 companies.
• Envoy AI Gateway - Manages unified access to generative AI services built on Envoy Gateway. Kubernetes-native AI gateway for routing, load balancing, and managing LLM traffic with enterprise-grade reliability. Apache 2.0 licensed.
• Pezzo - Cloud-native LLMOps platform with prompt management, versioning, and observability. Features collaborative prompt editing, A/B testing, and cost analytics. Apache 2.0 licensed.
• Microsoft PromptFlow - Comprehensive suite for LLM-based AI app development from prototyping to production. Includes prompt engineering, evaluation, and deployment tools with VS Code integration. MIT licensed.
• ChainForge - Visual programming environment for battle-testing prompts and evaluating LLM outputs. Features node-based prompt chains, multi-model comparison, and hypothesis testing. MIT licensed.

Guardrails & Safety Tools

• LLM Guard - Comprehensive security toolkit for LLM interactions with input/output scanners for prompt injection, PII anonymization, toxic content, secrets detection, and adversarial attack prevention. MIT licensed.
• PurpleLlama (Meta) - Comprehensive set of tools to assess and improve LLM security. Includes Llama Guard safety classifiers, CyberSec Eval benchmarks, and Prompt Guard for prompt injection detection. BSD-3-Clause licensed.
• Garak (NVIDIA) - The LLM vulnerability scanner. Probes models for hallucinations, data leakage, prompt injection, misinformation, toxicity, and jailbreaks. Extensive plugin-based architecture with 100+ vulnerability probes. Apache 2.0 licensed.
• Promptfoo - Open-source LLM evaluation and red teaming framework. Test prompts, agents, and RAGs with automated security vulnerability scanning, side-by-side model comparison, and CI/CD integration. Now part of OpenAI. MIT licensed.
• DeepTeam (Confident AI) - Red teaming framework for LLM systems with 50+ vulnerabilities, 20+ adversarial attacks, and production-ready guardrails. Includes OWASP, NIST, and MITRE ATLAS framework mappings. Apache 2.0 licensed.

---

📈 9. Evaluation, Benchmarks & Datasets

Benchmarks, evaluation frameworks, datasets, and supporting tools for model assessment.

Benchmark Suites

• LiveBench - Contamination-free LLM benchmark with objective ground-truth scoring. ICLR 2025 spotlight paper featuring frequently-updated questions from recent sources. Tests math, coding, reasoning, language, instruction following, and data analysis.
• lm-evaluation-harness (EleutherAI) - De-facto standard for generative model evaluation.
• HELM (Stanford) - Holistic Evaluation of Language Models.
• SWE-bench - Evaluates LLMs on real-world GitHub issues from 15+ Python repositories.
• GAIA - Real-world multi-step agentic benchmark.
• OpenCompass - Evaluation platform for benchmarking language and multimodal models across large benchmark suites.
• MLPerf Inference - Industry-standard ML inference benchmarks with reference implementations for AI accelerators.
• MLPerf Training - Industry-standard ML training benchmarks from MLCommons. Reference implementations for training AI models at scale across image classification, object detection, NLP, and recommendation tasks. Apache 2.0 licensed.
• VLMEvalKit - Open-source evaluation toolkit for large multi-modality models (LMMs). Supports 220+ LMMs and 80+ benchmarks including MMMU, MathVista, and ChartQA. Powers the OpenVLM Leaderboard. Apache 2.0 licensed.
• MTEB - Massive Text Embedding Benchmark. The standard evaluation suite for text embeddings with 1000+ evaluation tasks and an interactive leaderboard. Powers the Hugging Face Embedding Leaderboard. Apache 2.0 licensed.
• Vectara Hallucination Leaderboard - Leaderboard comparing LLM performance at producing hallucinations when summarizing short documents. Systematic evaluation of factual consistency across major models. Apache 2.0 licensed.
• SWE-rebench (Nebius) - Continuously updated benchmark with 21,000+ real-world SWE tasks for evaluating agentic LLMs. Decontaminated, mined from GitHub.
• AgentBench (THUDM) - Comprehensive benchmark to evaluate LLMs as agents across 8 diverse environments including household, web shopping, OS interaction, and database tasks. ICLR 2024. Apache 2.0 licensed.
• MLE-bench (OpenAI) - Benchmark for measuring how well AI agents perform at machine learning engineering. Evaluates agents on 75 Kaggle competitions covering diverse ML tasks. MIT licensed.
• WebArena - Realistic web environment for building and evaluating autonomous agents. Self-hostable benchmark with 812 diverse web tasks across shopping, CMS, Reddit, GitLab, and more. ICLR 2024. MIT licensed.
• PinchBench - Benchmarking system for evaluating LLM models as OpenClaw coding agents. Built with Rust by the kilo.ai team. MIT licensed.

Evaluation Frameworks

• DeepEval - The "Pytest for LLMs".
• Inspect AI - Framework for large language model evaluations from the UK AI Security Institute.
• RAGAs - End-to-end RAG evaluation framework.
• Lighteval - Evaluation toolkit for LLMs across multiple backends with reusable tasks, metrics, and result tracking.
• Hugging Face Evaluate - Standardized evaluation metrics.
• OpenAI Evals - Framework for evaluating LLMs and LLM systems with an open-source registry of 100+ community-contributed benchmarks. MIT licensed.
• LMMs-Eval - Unified multimodal evaluation toolkit for text, image, video, and audio tasks with 100+ supported benchmarks.
• FlashRAG - Efficient Python toolkit for RAG research with 32+ pre-processed benchmark datasets and 12 state-of-the-art RAG algorithms. Includes optimized evaluation pipelines and standard metrics. WWW 2025 Resource. MIT licensed.
• BrowserGym - Gym environment for web task automation and agent evaluation. Includes MiniWoB, WebArena, WorkArena, and more. Apache 2.0 licensed.
• TruLens - Evaluation and tracking for LLM experiments and AI agents. Provides feedback functions for measuring quality, relevance, and groundedness with LangChain and LlamaIndex integrations. MIT licensed.
• OpenEvals - Open-source evaluation library for LLM and agent applications. Built by LangChain with pre-built evaluators for common use cases including RAG, agents, and structured output validation. MIT licensed.
• AutoRAG - RAG AutoML tool for automatically finding optimal RAG pipelines. Evaluates and optimizes retrieval-augmented generation with AutoML-style automation for your own data and use-case. Apache 2.0 licensed.
• E2B Code Interpreter - Python & JS/TS SDK for running AI-generated code in secure isolated sandboxes. Essential infrastructure for evaluating code-generating LLMs with safe execution environments. Apache 2.0 licensed.
• SimpleEvals (OpenAI) - Lightweight library for evaluating language models with transparent accuracy numbers. Reference implementations for MMLU, GPQA, MATH, HumanEval, MGSM, DROP, and SimpleQA benchmarks. MIT licensed.
• EvalScope (ModelScope) ![GitHub stars](https://img.shields.io/github/stars/model