# Ai Infra Control Plane

> AI Infrastructure OS control plane for governed private AI: policy, identity, audit, intent, MCP, FinOps, SLOs, Redis, Prometheus and OIDC on Kubernetes.

- **Type:** MCP server
- **Install:** `agentstack add mcp-justrunme-ai-infra-control-plane`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [justrunme](https://agentstack.voostack.com/s/justrunme)
- **Installs:** 0
- **Category:** [Databases](https://agentstack.voostack.com/c/databases)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [justrunme](https://github.com/justrunme)
- **Source:** https://github.com/justrunme/ai-infra-control-plane

## Install

```sh
agentstack add mcp-justrunme-ai-infra-control-plane
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# AI Infrastructure Control Plane

[](https://github.com/justrunme/ai-infra-control-plane/actions/workflows/ci.yml)
[](https://github.com/justrunme/ai-infra-control-plane/actions/workflows/release.yml)

> **AI Infrastructure OS** — open-source operating layer for private AI platforms: policy, cost, capacity, observability, and fleet operations on Kubernetes.

[](docs/videos/hero-overview.mp4)

This repository is the **Control Plane** of the AI Infrastructure OS. The reference **Execution Plane** is [AI Runtime Platform](https://github.com/justrunme/ai-runtime-platform).

Read the [product roadmap](docs/product-roadmap.md) and [portfolio overview](docs/portfolio-overview.md) for the full platform map.

### One-command platform demo

Bring up **Control Plane + Execution Plane + Ollama** and verify governance enforcement:

```sh
make platform-demo              # laptop demo
make platform-demo-verify

make platform-demo-production   # + Redis quota + Prometheus inputs
make platform-demo-production-verify

make platform-demo-enterprise   # + Keycloak OIDC (reference architecture)
make platform-demo-enterprise-verify
```

See [demo/platform/README.md](demo/platform/README.md) for URLs and manual curls.

This project is intentionally scoped as an AI infrastructure platform, not an agent framework.

The core workflow is:

```text
AI request
  -> tenant quota
  -> model risk registry
  -> cost decision
  -> risk score
  -> approval decision
  -> final verdict
```

Read the portfolio overview in `docs/portfolio-overview.md`, the [product roadmap](docs/product-roadmap.md), and the technical system design in `docs/platform-architecture.md`.

## Operator Dashboard

The control API serves a live operator dashboard at `/` with platform status,
topology health, model inventory, a **governance playground**, and **inventory
drift detection**, refreshed every few seconds.

### Governance Playground

> **Interactive demo** | Click the animated preview to watch the governance playground and drift detection walkthrough.

[](docs/videos/governance-playground.mp4)

Submit an AI platform request through the live governance pipeline without leaving the browser:

- `POST /governance/evaluate` — quota → registry → cost → risk → approval → final verdict
- Presets on `/` for low-risk dev, production external, and budget-exceeded scenarios

### Inventory Drift Detection

Compare configured model inventory (`MODEL_INVENTORY_PATH` / Helm ConfigMap) against live Ollama and vLLM probes:

- `GET /drift` — desired vs actual models per backend, missing/unexpected lists
- Prometheus gauges: `ai_control_inventory_in_sync`, `ai_control_inventory_drift`

## How the Projects Fit Together

This repository is the **Control Plane** of the [AI Infrastructure OS](docs/product-roadmap.md). The **Execution Plane** lives in [ai-runtime-platform](https://github.com/justrunme/ai-runtime-platform).

| Layer | Role in AI Infrastructure OS | Repository |
| --- | --- | --- |
| **Execution Plane** | Runs inference, routes traffic, enforces governance verdicts | [justrunme/ai-runtime-platform](https://github.com/justrunme/ai-runtime-platform) |
| **Control Plane** | Policy, cost, topology, drift, SLO, fleet operations | [justrunme/ai-infra-control-plane](https://github.com/justrunme/ai-infra-control-plane) |

The Execution Plane runs workloads. The Control Plane evaluates policy and the runtime enforces verdicts at the inference boundary via `CONTROL_PLANE_URL`.

## Product Walkthroughs

### Governance Decision Flow

[](docs/videos/governance-pipeline.mp4)

### AI Infrastructure Digital Twin

[](docs/videos/digital-twin.mp4)

### Forecast-driven Scaling

[](docs/videos/forecast-driven-scaling.mp4)

## Visual Overview

### Platform Overview

```mermaid
flowchart TB
    User["AI Consumer"]
    API["Control APIFastAPI"]

    subgraph ControlPlane["AI Infrastructure Control Plane"]
        Capacity["Capacity Planner"]
        Cost["Cost Governance"]
        Risk["Risk Scoring"]
        Approval["Approval Engine"]
        Twin["Digital Twin"]
    end

    subgraph AI["AI Workloads"]
        Ollama["Ollama"]
        VLLM["vLLM"]
        Models["Foundation Models"]
    end

    subgraph Observability["Observability"]
        OTel["OpenTelemetry"]
        Prom["Prometheus"]
        Graf["Grafana"]
        Loki["Loki"]
    end

    User --> API
    API --> Capacity
    API --> Cost
    API --> Risk
    API --> Approval
    API --> Twin
    Capacity --> Ollama
    Capacity --> VLLM
    Ollama --> Models
    VLLM --> Models
    Ollama --> OTel
    VLLM --> OTel
    OTel --> Prom
    OTel --> Loki
    Prom --> Graf
    Loki --> Graf
    Twin --> Graf
```

### Governance Flow

```mermaid
flowchart LR
    Request["AI Request"]
    Cost["Cost Analysis"]
    Risk["Risk Analysis"]
    Capacity["Capacity Check"]
    Decision{"Policy Engine"}
    Allow["ALLOW"]
    Warn["WARN"]
    Block["BLOCK"]

    Request --> Cost
    Request --> Risk
    Request --> Capacity
    Cost --> Decision
    Risk --> Decision
    Capacity --> Decision
    Decision --> Allow
    Decision --> Warn
    Decision --> Block
```

### Digital Twin

```mermaid
flowchart LR
    Real["Real AI Cluster"]
    Metrics["Telemetry"]
    Twin["Digital Twin Model"]
    Simulate["Scenario Simulation"]
    Decision["Capacity Decision"]

    Real --> Metrics
    Metrics --> Twin
    Twin --> Simulate
    Simulate --> Decision
```

### Forecast-driven Scaling

```mermaid
flowchart TB
    Metrics["Historical Metrics"]
    TimesFM["TimesFM Forecasting"]
    Forecast["Demand Forecast"]
    ScaleUp["Scale Up"]
    ScaleDown["Scale Down"]

    Metrics --> TimesFM
    TimesFM --> Forecast
    Forecast --> ScaleUp
    Forecast --> ScaleDown
```

### GitOps Delivery

```mermaid
flowchart LR
    Dev["Developer"]
    Git["Git Repository"]
    Actions["GitHub Actions"]
    Registry["Container Registry"]
    Argo["Argo CD"]
    Cluster["Kubernetes Cluster"]

    Dev --> Git
    Git --> Actions
    Actions --> Registry
    Registry --> Argo
    Argo --> Cluster
```

## Scope

- Expose a control API for private AI backend health, latency, capacity, cost, identity, audit, and governance signals.
- Operate model, tool, agent, and intent metadata through registries and policy packs.
- Evaluate governance decisions across prompt security, tenant quota, model registry, cost, risk, approval, sovereign AI, and response evaluation stages.
- Feed live governance inputs from Redis-backed tenant quota state and Prometheus telemetry.
- Run a full reference stack with Control Plane, Execution Plane, Ollama, Redis, Prometheus, and Keycloak OIDC.
- Package the API with Docker and Helm, including production defaults and External Secrets / JWKS wiring.
- Provide GitOps, security, SLO, FinOps, fleet topology, capacity, and GPU placement reference modules.
- Keep experimental modules clearly isolated from the production deployment path.

## Repository Layout

```text
apps/
  control-api/        FastAPI service with health and model status endpoints
infra/
  helm/               Kubernetes packaging
  terraform/          Cloud bootstrap modules
    k3s-bootstrap/    Example Hetzner VM bootstrap with cloud-init and k3s
observability/
  grafana/            Dashboards and metrics notes
  loki/               Loki and Promtail logging examples
  otel-genai/         OpenTelemetry GenAI telemetry reference implementation
forecasting/
  timesfm/            Experimental capacity forecasting module
experiments/
  inference-autoscaling/ Forecast-driven inference scaling recommendations
governance/
  agents/             Agent-to-model/tool/policy bindings
  cost/               AI cost governance policy engine
  intent/             Natural-language intent to governed execution plan
  policy-packs/       Environment and team-specific policy overlays
  prompt-security/    PII, secret, and prompt-injection checks
  quota/              Tenant quota policy checks
  registry/           Signed model registry and attestation metadata
  risk/               AI request risk scoring engine
  approval/           Human approval workflow reference implementation
  pipeline/           End-to-end AI governance decision pipeline
  tools/              MCP tool registry and action allowlists
security/
  trivy/              Container and IaC scan configuration
  opa/                Kubernetes policy gates for rendered manifests
docs/
  architecture.md     System design notes
  case-study.md       Portfolio case study and demo flow
  digital-twin.md     AI infrastructure topology model
  platform-architecture.md Technical platform architecture
```

## Local Development

```sh
cd apps/control-api
python3.12 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reload
```

Run tests:

```sh
make venv
make test
```

The project targets Python 3.12 for local development and CI.

Run the portfolio demo:

```sh
make demo
```

The demo prints the key control API endpoints and runs the end-to-end governance pipeline from `governance/pipeline/sample_requests.csv`.

## Control API

The control API exposes operator-facing signals for private AI infrastructure:

- `GET /` - live operator dashboard with governance playground and drift panel (HTML).
- `GET /health` - operator-facing service health.
- `GET /healthz` - Kubernetes-compatible health check.
- `GET /models` - configured model backends and status.
- `GET /metrics` - Prometheus-compatible text metrics.
- `GET /capacity` - aggregate model serving capacity.
- `GET /cost` - estimated hourly, daily, and monthly cost.
- `GET /summary` - compact status for dashboards and demos.
- `GET /governance/inputs/status` - Redis quota and Prometheus input status.
- `POST /intent/resolve` - intent-to-agent/model/tool/region execution planning.
- `POST /governance/evaluate-tool` - governed MCP tool call decision.
- `POST /governance/evaluate-response` - post-response quality, latency, and cost evaluation.

### Model Inventory

The model inventory is configuration-driven. By default the API loads
`app/model_inventory.json` shipped with the image, but you can point it at any
JSON file:

```sh
export MODEL_INVENTORY_PATH=/etc/ai-control-plane/model_inventory.json
```

The file is a JSON array of model entries; see
`apps/control-api/examples/model_inventory.sample.json` for a multi-backend
example. If the file is missing or malformed, the API falls back to a built-in
inventory so the control plane stays observable.

### Ollama Backend Probe

Set `OLLAMA_BASE_URL` to point the control API at an Ollama backend:

```sh
export OLLAMA_BASE_URL=http://localhost:11434
```

The API exposes:

- `GET /backends/ollama/health` - backend reachability and status.
- `GET /backends/ollama/models` - model names returned by Ollama `/api/tags`.
- `GET /backends/ollama/latency` - lightweight latency measurement for `/api/tags`.

### vLLM Backend Probe

Set `VLLM_BASE_URL` to point the control API at a vLLM OpenAI-compatible server:

```sh
export VLLM_BASE_URL=http://localhost:8000
```

The API exposes:

- `GET /backends/vllm/health` - backend reachability and status.
- `GET /backends/vllm/models` - model ids returned by vLLM `/v1/models`.
- `GET /backends/vllm/latency` - lightweight latency measurement for `/v1/models`.

### Prometheus Metrics

`GET /metrics` exposes Prometheus-compatible metrics for request traffic, backend health, model inventory, capacity, and estimated cost.

Core metrics:

- `ai_control_http_requests_total`
- `ai_control_http_request_latency_ms`
- `ai_control_backend_up`
- `ai_control_backend_latency_ms`
- `ai_control_model_available`
- `ai_control_capacity_available`
- `ai_control_estimated_hourly_cost_usd`

### AI Infrastructure Digital Twin

`GET /topology` exposes a live platform graph for private AI infrastructure components, dependencies, health, telemetry, and operational signals. The Ollama and vLLM nodes reflect live backend probe results (healthy/degraded plus measured latency). See `docs/digital-twin.md`.

`GET /drift` compares configured model inventory against live Ollama and vLLM probes and reports missing or unexpected models per backend.

`POST /governance/evaluate` runs a single AI request through workload identity, policy packs, prompt security, tenant quota, model registry, cost, risk, approval, sovereign AI, and live telemetry checks, then returns the final verdict for the operator dashboard playground and runtime enforcement path.

### AI Cost Governance

`governance/cost` evaluates model usage, team budgets, token spend, and forecasted monthly cost into `allow`, `warn`, or `block` decisions.

### AI Approval Workflow

`governance/approval` evaluates high-risk AI platform requests into `allow`, `approval_required`, or `block` decisions for human approval gates.

### AI Governance Layer

`governance/cost`, `governance/risk`, and `governance/approval` model cost control, risk scoring, and human approval gates for private AI infrastructure.

`governance/pipeline` connects those signals into an end-to-end decision flow: request telemetry, cost decision, risk score, approval decision, and final verdict.

Run the demo pipeline:

```sh
python3.12 governance/pipeline/run_pipeline.py \
  --requests governance/pipeline/sample_requests.csv
```

## Container Images

Every merge to `main` builds and pushes the control API image to GitHub Container Registry:

```text
ghcr.io/justrunme/ai-infra-control-plane:latest
ghcr.io/justrunme/ai-infra-control-plane:
```

Images are signed with cosign and accompanied by an SPDX SBOM artifact from the release workflow. Tag releases with `v*` (for example `v0.2.0`) to publish semver tags.

## Kubernetes Deployment

```sh
helm upgrade --install ai-control-plane infra/helm/ai-control-plane \
  --set image.repository=ghcr.io/justrunme/ai-infra-control-plane \
  --set image.tag=latest
```

The chart ships production defaults: non-root execution, read-only root filesystem, model inventory ConfigMap, HPA, PodDisruptionBudget, and optional ServiceMonitor, Ingress, and NetworkPolicy. See `infra/helm/ai-control-plane/README.md`.

## Portfolio Docs

- `docs/case-study.md` explains the problem, architecture, capabilities, governance pipeline, observability, forecasting, GitOps, security, and demo flow.
- `docs/platform-architecture.md` describes the system boundary, logical layers, control API, governance architecture, delivery path, and extension points.

## Remaining Backlog

See `docs/product-roadmap.md` for the current roadmap and maturity map. The next portfolio step is packaging evidence: record the enterprise demo GIF from `docs/portfolio/demo-gif-script.md` and add it to this README.

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [justrunme](https://github.com/justrunme)
- **Source:** [justrunme/ai-infra-control-plane](https://github.com/justrunme/ai-infra-control-plane)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-justrunme-ai-infra-control-plane
- Seller: https://agentstack.voostack.com/s/justrunme
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.