- Conduid
- Marketplace
- #inference
MCP servers tagged inference
58 live MCP servers tagged "inference", ranked by trust score. Tags come from package metadata and repository topics, so this list covers servers across every category that work with inference.
Vllm Mlx
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code supp…
client
Use Hugging Face with JavaScript
Fastapi Bitnet
Running Microsoft's BitNet inference framework via FastAPI, Uvicorn and Docker.
Amazon Sorftime Research MCP Skill
Analyze Amazon listings and categories using Sorftime MCP to deliver detailed competitor insights and automated product selection.
Mlxstudio
MLX Studio - Home of JANG_Q - Image Gen/Edit + Chat/Code All in one - + OpenClaw (Anthropic API)
Flama
The production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in chat UI, and native…
Llama Terminal Completion
AI terminal assistant for any OpenAI-compatible API. Features interactive chat TUI, command generation, code explanation, and streaming responses. Works with Ollama, OpenAI, LM St…
Quickstart Streaming Agents
Build, deploy, and orchestrate event-driven agents natively on Apache Flink® and Apache Kafka®
Global Capacity Orchestrator On AWS
AI/ML and HPC at planetary scale. One API. Every Accelerator. Any Region. MIT-0 licensed
Asiai
Multi-engine LLM benchmark & monitoring CLI for Apple Silicon
Rasputin Memory
The ultimate memory backend for OpenClaw and Claude Code. Persistent conversation memory with LLM fact extraction, foundation-model reranking, and 77.7% LoCoMo accuracy. MCP nativ…
Aigate
A self-hosted AI platform — inference, tool use, browser automation, image generation, speech synthesis, transcription, object storage, agentic code execution, and more — behind a…
Guidance For Scalable Model Inference And Agentic AI On Amazon Eks
Comprehensive, scalable ML inference architecture using Amazon EKS, leveraging Graviton processors for cost-effective CPU-based inference and GPU instances for accelerated inferen…
Ghostlink
Distributed LLM inference fabric for heterogeneous local clusters. Features zero-copy SPSC ring buffers, automated tuning, and an OpenAI-compatible API.
LightLX
Run models too big for RAM on Apple Silicon — any Hugging Face model, dense or MoE — by streaming weights from disk
SwarmLLM
Decentralized peer-to-peer LLM inference network. Single Rust binary, BitTorrent-inspired incentives, OpenAI-compatible API.
Frugal
Run AI agents cheap, local, and verified — metering, cost routing, cache, eval, MCP cost server. Offline, zero-dep, Apache-2.0.
Capix
Capix MCP Server — deploy and manage private LLM instances, compute, websites, and verified workloads on the Capix network from any AI coding agent that supports MCP.
Inferna
An experimental nanobind wrapper around llama.cpp, whisper.cpp, and stable-diffusion.cpp
Scalattice CLI
Scalattice developer CLI: setup, API keys, credits, optional MCP. npm: scalattice-cli
Deeplit MCP Server
Claude Code and Codex plugin for deeplit: manage GPU inference instances, volumes, models and billing from your coding agent, authorized by a scoped API key.
bridgenode-ai/bridgenode-mcp
y @bridgenode/mcp` or remote `https://bridgenode.cc/mcp`.
Gpu Perf Tune
Agent Skills and an MCP server for GPU performance profiling, benchmarking, optimization, and reporting, with an inference focus.
Bridgenode SDK Ts
BridgeNode — x402 pay-per-request AI inference. TypeScript SDK.
TokLedger
The local inference ledger -> a zero-dependency OpenAI-compatible proxy that prices every request your home GPU serves: tokens, TTFT, tok/s, kWh, local $ and cloud-equivalent cost.
Junglegrid Examples
Example workloads, MCP prompts, CLI commands, and integration templates for running AI jobs on Jungle Grid.
Vecboost
Production embedding & decision inference service in Rust — Candle/ONNX dual engine, CUDA/Metal acceleration, REST/gRPC/MCP/CLI generated via sdforge, rerank, Matryoshka dimension…
Dspytools
self-evolving CLI for DSPy program management — featuring hot-swap inference, MCP agent interoperability, a Generative Feedback Loop (GFL) pipeline, skills system, llms.txt genera…
Sovereign Projects
Maximal sovereign workspace layer and monorepo
TARILIO
AI-powered Information Retrieval with integrated AI Assistant, MCP Client, Local LLM server
VrtxOmega/Ollama-Omega
Official Ollama MCP Server. Exposes ollama_chat, ollama_generate, ollama_pull_model, ollama_list_models and ollama_show_model tools for advanced AI interactions.
Diffusion LLM MCP
FastMCP fleet MCP server for diffusion LMs (dLLM). DiffusionGemma on Goliath RTX 4090 — batch inference, HLE-shaped reasoning, ~200–400 tok/s. Doc phase; llama-diffusion-cli sidec…
Inferbench
Benchmark local LLM inference speed (tokens/sec) on your own hardware — llama.cpp native + cloud APIs, 124-model catalog, optimal-quant picker, and an MCP serve mode.
AI Roadmap
AL/ML learning Gym including traditional ML, deep learning, fine-tuning, Gen AI, Agentic AI. Everything from experiments to production with evaluation and observability.
Axelera Voyager Local Assistant
Local AI assistant for the Axelera Voyager SDK and Metis AIPU — semantic RAG over the public Axelera sources + workflow skills, via MCP. No account, no cloud.
Molla
Local inference server in pure Mojo. Speaks OpenAI, Anthropic, and MCP. Models are OCI artifacts. Same source runs on CPU, NVIDIA, AMD, and Apple GPUs. Apache-2.0 all the way down.
Cistern
A semantic cache for LLM calls: collapses paraphrased and reformatted prompts into one cached answer. On realistic agent traffic a byte-exact cache leaks (46% hit rate); Cistern r…
Cancerbero
Local, offline-first GGUF and llama.cpp pre-load inspector
Chutes Agent Toolkit
One toolkit, every agent — drop decentralized, all-TEE, OpenAI-compatible Chutes.ai inference into Claude, Hermes, OpenClaw, Codex, or any OpenAI-compatible client.
Kaepsele
Full-stack AI Chatbot deployment for universities. Includes RAG, secure Code Interpreter, vLLM inference, and educational prompt libraries.
Bridgenode Skill
BridgeNode — x402 pay-per-request AI inference. Agent skill and buyer-side examples for AI agents.
Dontguess
Cached-inference exchange for AI agents. Retired experiment.
Bridgenode SDK Python
BridgeNode — x402 pay-per-request AI inference. Python SDK.
Agent Quickstart
🤖 Build and prototype custom coding agents with this self-hostable quickstart app, inspired by Claude Code for agile development.
Edgemesh
Cross-OS federated gateway unifying every OpenAI-compatible inference backend (Cognis fleet, Ollama, llama.cpp, vLLM) behind one /v1 endpoint
Io Net
io.net — independent third-party profile of a public API surface, by API Evangelist. io.net is a decentralized GPU network for AI compute, built on Solana, that aggregates idle co…
Karakan App Selector Wen
Manage concurrent LLM requests with priority queues, multi-model routing, fault tolerance, and semantic caching for efficient AI workflows.
Moemodels
Open tooling and verifiable evidence for planning, measuring, and reviewing mixture-of-experts deployments.
Dedaluslabs
Dedalus Labs — independent third-party profile of a public API surface, by API Evangelist. Dedalus Labs builds infrastructure for AI agents. It runs two production APIs: the Dedal…
Squirrel
Pure Rust AI coordination — MCP server, multi-provider inference routing, local-first LLM orchestration. Sovereign AI for distributed computing. AGPL-3.0