For Public Health
Framework for building evaluated, methodology-aware public health data agents using MCP servers
Ask AI about For Public Health
Powered by Claude · Grounded in docs
I know everything about For Public Health. Ask me about installation, configuration, usage, or troubleshooting.
0/500
Reviews
Documentation
MCP for Public Health
A framework for building evaluated, methodology-aware public health data agents using MCP (Model Context Protocol) servers. This project provides templates and patterns for creating agents that don't just retrieve data, but reason about its methodology, scope, and limitations before answering.
The Three Layers
1. MCP Server -- A tool server that enriches every data response with methodology context, geographic scope, cross-dataset comparisons, and supported/unsupported conclusions alongside the raw data. Templates in templates/mcp-server/.
2. Agent with Reasoning Protocol -- An agent that uses a structured 6-step reasoning protocol before answering any data question: cross-dataset reasoning, methodology awareness, scope adherence, causal inference boundaries, geographic resolution knowledge, and terminology fluency. Templates in templates/agent/.
3. Evaluation Pipeline -- A KFP (Kubeflow Pipelines) evaluation pipeline that scores agent responses on 7 capability dimensions using an LLM-as-judge, with gold standard comparisons. Templates in templates/eval-pipeline/.
Prerequisites
- fips-agents CLI for scaffolding agents
- An OpenShift cluster with OpenShift AI (or any Kubernetes cluster with KFP)
- A vLLM-served model (or any OpenAI-compatible endpoint)
- Python 3.11+
Quick Start
-
Scaffold your MCP server:
fips-agents create mcp-server your-domain-server. Add tools following the enrichment pattern intemplates/mcp-server/example-tool.py. -
Customize the agent prompt: Copy
templates/agent/prompts/system.md, replace{DOMAIN}with your domain, and update the tool descriptions and reasoning examples. -
Write seed questions and gold standards: Use
templates/eval-pipeline/questions/seed-template.yamlandtemplates/eval-pipeline/gold-standards/example-seed.yamlas starting points. -
Run the eval pipeline: Adapt
templates/eval-pipeline/pipeline.pywith your model endpoint and MCP server URL, compile withpython pipeline.py, and submit to your KFP instance.
Documentation
- Reasoning Protocol -- How to apply the 6-step protocol to your domain
- Evaluation Methodology -- The 7-dimension capability judge explained
- Architecture -- Component diagram and deployment overview
License
Apache 2.0. See LICENSE.
