- Conduid
- Marketplace
- #llm-inference
MCP servers tagged llm-inference
43 live MCP servers tagged "llm-inference", ranked by trust score. Tags come from package metadata and repository topics, so this list covers servers across every category that work with llm-inference.
Lemonade
Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk
Qurio
Qurio brings multi-provider models, custom agents, reusable skills, MCP servers, HTTP tools, retrieval, long-term memory, Deep Research, and Scrapbook into one operating surface.
Rapid MLX
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents.…
RuVector
RuVector is a High Performance, Real-Time, Self-Learning, Vector GNN, Memory DB built in Rust.
Optillm
Optimizing inference proxy for LLMs
Spiceai
A portable accelerated SQL query, search, and LLM-inference engine, written in Rust, for data-grounded AI apps and agents.
Neuron AI
The Agentic Framework of the PHP ecosystem to build production-ready AI driven applications. Connect components (LLMs, Tools, vector DBs, memory) to agents that interact with your…
Mindbridge MCP
MindBridge is an AI orchestration MCP server that lets any app talk to any LLM — OpenAI, Anthropic, DeepSeek, Ollama, and more — through a single unified API. Route queries, compa…
Sample Genai On Eks Starter Kit
A comprehensive toolkit for deploying production-ready Generative AI infrastructure on Amazon EKS. Includes pre-configured components for: 🚀 AI Gateway (LiteLLM) 🤖 LLM Serving (v…
Atomic Chat
Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer.
Deeppowers
DEEPPOWERS is a Fully Homomorphic Encryption (FHE) framework built for MCP (Model Context Protocol), aiming to provide end-to-end privacy protection and high-efficiency computatio…
Captain Claw
Self-hosted framework for orchestrating fleets of specialist AI agents — ensemble reasoning and a full agentic coding pipeline, model-agnostic and local-friendly.
AutoMegaKernel
An agent harness that compiles a model into one provably-correct, self-retargeting CUDA megakernel and self-tunes it past cuBLAS at batch-1 LLM decode, paper: https://arxiv.org/ab…
Tokio Prompt Orchestrator
Multi-core, Tokio-native orchestration for LLM pipelines.
Rai
CPU-only LLM inference engine in pure Rust — 4-bit quantized models, hand-written AVX2 kernels, speculative decoding, and a local HTTP/MCP server. No GPU, no Python runtime.
Polymer Agent
LLM aided multi tool kit for initial polymer discovery
AIMA
AI-Inference-Managed-by-AI: Go binary for managing AI inference on edge devices
Fak
Agentic Runtime
ToolsFilter
Fetch only relevant tools for the current conversation and save cost while increasing the precision of your LLM Response
Plux Agent Gui Mvp
💡AI finder/explorer. One click @files via a visual filetree and save insights in a notepad. build with Tauri
Rwkv Router
Cut up to ~89% of cloud LLM tokens. Self-evolving local-first smart router (R0-R3, ~100% routing accuracy after evolution) - Stanford & Together AI baseline: 60-80% cost/energy sa…
Fituna
Stop guessing your llama.cpp config. Give it a target tok/s and a quality budget, and it measures quantization levels and GPU offload on your own machine to find the lightest setu…
Basalt
Turn your Mac into a local LLM server: MLX on Metal, OpenAI + Anthropic API + MCP, pure Swift. Point Claude Code at it.
Gpu Perf Tune
Agent Skills and an MCP server for GPU performance profiling, benchmarking, optimization, and reporting, with an inference focus.
Spraay Compute
💧 OpenClaw / Claude Code skill for AI agents: rent GPU compute, run AI model inference (LLM, image, video, speech, embeddings), and buy prepaid compute futures credits — paid in U…
Datagovin MCP Server
Core server for exposing India’s open government data (data.gov.in) APIs via the Model Context Protocol, powering agentic AI data exploration.
Customer Support Ticket Automation Using AI Agents And MCP
This Project uses large language models to automate customer support. It classifies tickets, analyzes content, generate and send responses automatically to the given customer emai…
Floe Labs Docs
Cost & margin ledger for voice AI
Nekton Server
A production-grade Model Context Protocol (MCP) server that augments local LLM inference (via vLLM or Ollama) with real-time external data (e.g., weather, finance, news) to make r…
Standardcompute MCP
MCP server for Standard Compute — endpoint health, model catalog, live plan pricing, and a completion probe for any MCP client (Claude Code, Claude Desktop, opencode)
Aihappey AI
Stateless unified multi-provider AI runtime with OpenAI & Anthropic-compatible APIs, Vercel UI streaming, managed agents and Skills across 130k+ models.
Compare MCP
Compare code reviews across models in Claude Code with ranked todos and subagent dispatch via MCP.
Inference Margins
Interactive model of frontier LLM serving costs and unit margins — margins.ashitaorbis.com
Agent Orchestration Core
Self-hostable, multi-tenant orchestration engine for AI agents. Every run is pinned to the exact versioned definitions and policies that governed it.
Oai Chat
Multi-modal Chatbot based on OpenAI
Hirn
Monorepo for Hirn. Hirn is a composable, private, local-first AI agent ecosystem that, by default, runs 100% private natively on your own hardware
HumanWrite X MCP Server Adaptive LLM Based Content Humanization System
a MCP server for professional-grade neuro-stylistic content humanization engine designed to neutralize commercial AI detectors. By integrating advanced stylometric entropy, syntac…
Caix
caix — native Apple Core AI inference server for Apple silicon (beta): OpenAI/Anthropic API, dashboard, streaming chat with tools/skills/MCP, MTP speculative decoding.
Computesage MCP
Public metadata for the ComputeSage StackBench remote MCP server.
Lymerorium Local LLM Debate
Self-hosted heterogeneous LLM debate engine - Gemma on Jetson edge + Qwen + Llama argue in parallel, confidence-weighted voting. +40pp MMLU vs single-model baseline.
AccessIQ
MCP-based authorization for protected data access using RBAC and ABAC.
Understanding LLMs
Understanding LLMs for Software Engineers: a systems-first, bilingual guide to LLM principles, mechanisms, and boundaries.
Chipleap
Design the chip your model needs: open-source architecture exploration from an AI workload to verified hardware, an X-HEEP RISC-V SoC or AXI4 IP, checked down to a routed layout o…