# Ops Pilot > Production-grade multi-agent AI DevOps system for SRE incident response, where a central orchestrator coordinates 12 specialist agents to classify incidents, traverse service dependency graphs, analyze repositories, Terraform, and telemetry, determine root cause, and stream real-time remediation plans to operators via SSE. - **Type:** MCP server - **Install:** `agentstack add mcp-khaledayman9-ops-pilot` - **Verified:** Pending review - **Seller:** [Khaledayman9](https://agentstack.voostack.com/s/khaledayman9) - **Installs:** 0 - **Category:** [Databases](https://agentstack.voostack.com/c/databases) - **Latest version:** 0.1.0 - **License:** Apache-2.0 - **Upstream author:** [Khaledayman9](https://github.com/Khaledayman9) - **Source:** https://github.com/Khaledayman9/ops-pilot ## Install ```sh agentstack add mcp-khaledayman9-ops-pilot ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # ⚡ Ops-Pilot — AI-Powered SRE Incident Response Platform Ops-Pilot is a production-grade, multi-agent AI system for SRE incident response. A central orchestrator coordinates twelve specialist AI agents that classify incidents, traverse a service dependency knowledge graph, scan repositories and Terraform state, analyse telemetry, identify root causes, and generate actionable remediation plans — all streamed in real time to the operator via Server-Sent Events. --- ## Table of Contents - [Layout](#layout) - [Architecture Overview](#architecture-overview) - [Tech Stack](#tech-stack) - [Agent Pipeline](#agent-pipeline) - [Quick Start](#quick-start) - [API Routes](#api-routes) - [Authentication](#authentication) - [Security Guardrails](#security-guardrails) - [Streaming (SSE)](#streaming-sse) - [Neo4j Knowledge Graph](#neo4j-knowledge-graph) - [LLM Provider Switching](#llm-provider-switching) - [Alembic Migrations](#alembic-migrations) - [Celery Periodic Tasks](#celery-periodic-tasks) - [MCP Integrations](#mcp-integrations) - [Docker Targets](#docker-targets) - [Environment Variables](#environment-variables) - [Testing](#testing) - [CI/CD](#cicd) --- ## Layout Home Page **Dark Mode (Default):** **Light Mode:** Settings Chat Interface **Chat window:** **Query:** **Explainability Panel:** **Cancel Operation:** Authentication & Profile **Login:** **Register:** **Profile:** Help & Contact **Help:** **Contact:** --- ## Architecture Overview ``` ┌──────────────────────────────────────────────────────────────────────┐ │ Next.js 15 │ TypeScript │ Tailwind CSS │ Framer Motion │ │ / /chat /login /register /help /settings /contact │ └──────────────────────────┬───────────────────────────────────────────┘ │ SSE + REST (JWT Bearer) ┌──────────────────────────▼───────────────────────────────────────────┐ │ FastAPI /api/v1/{auth, incident, chat, stream, health} │ │ JWT access + refresh tokens │ bcrypt │ Guardrails (Presidio) │ └────────────────┬─────────────────────────────────────────────────────┘ │ ┌────────────────▼─────────────────────────────────────────────────────┐ │ IncidentOrchestrator (LangGraph async generator) │ │ │ │ 1. Classifier — severity, service, urgency, type │ │ 2. Entity Extractor — services, deployments, owners, metrics │ │ 3. Document Processor — PDF, DOCX, PPTX, CSV, Markdown │ │ 4. Repo Scanner — GitHub commits, PRs, CI checks │ │ 5. Terraform Scanner — IaC drift, plans, workspace state │ │ 6. Graph Analyzer — Neo4j blast-radius + dependency map │ │ 7. Web Intelligence — DuckDuckGo: CVEs, advisories, outages │ │ 8. Ops Analyst — latency, error rate, saturation │ │ 9. Crew Intelligence — CrewAI Researcher→Analyst→Writer │ │ 10. Root Cause Finder — causal chain + deployment correlation │ │ 11. Remediator — rollback, runbooks, escalation paths │ │ 12. Conversationalist — operator-ready Markdown narrative │ └────────┬──────────────────────┬───────────────────────┬─────────────┘ │ │ │ ┌────────▼──────┐ ┌────────────▼──────┐ ┌────────────▼────────────┐ │ Neo4j 5 │ │ PostgreSQL 16 │ │ Redis 7 │ │ Service graph│ │ Users / Chats / │ │ Celery broker │ │ knowledge │ │ Messages / │ │ + result backend │ │ base │ │ Executions │ │ + periodic tasks │ └───────────────┘ └───────────────────┘ └─────────────────────────┘ ``` --- ## Tech Stack | Layer | Technology | | ---------- | ----------------------------------------------------------------------- | | Frontend | Next.js 15, TypeScript, Tailwind CSS, Framer Motion, anime.js, Jest | | Backend | Python 3.11, FastAPI, LangGraph, CrewAI, LangChain | | LLM | OpenAI (default) · Anthropic · Google — runtime-switchable, no rebuild | | Graph DB | Neo4j 5 — service dependency knowledge graph + blast-radius traversal | | Relational | PostgreSQL 16 + SQLAlchemy 2 async + Alembic migrations | | Queue | Redis 7 + Celery (periodic graph maintenance tasks) | | Auth | JWT (access + refresh tokens), bcrypt password hashing, python-jose | | Guardrails | Prompt injection detection, PII scrubbing (Presidio + regex fallback) | | MCP | GitHub MCP server, Terraform MCP server, custom Ops Inspector server | | IaC | Docker Compose — dev and prod overlays | | CI/CD | GitHub Actions — pytest, type-check, lint, migration check, MLflow eval | | Streaming | Server-Sent Events (SSE) via sse-starlette | --- ## Agent Pipeline Every incident query travels through the orchestrator's async generator, which yields `StreamEvent` objects that are forwarded directly to the frontend via SSE. Each agent is independently enabled/disabled at runtime by the operator. ### 1. Classifier Extracts service name, severity (P0–P3), incident type, affected components, trigger event, and confidence score from the raw query using structured LLM output (Pydantic model). ### 2. Entity Extractor Parses the query for structured entities: service names, deployments, metrics, error codes, time ranges, and Cypher-ready search keywords for downstream graph and web queries. ### 3. Document Processor (optional) Converts uploaded attachments (PDF, DOCX, PPTX, XLS/XLSX, HTML, Markdown, CSV, TXT) to Markdown and injects them into the pipeline context. All downstream agents receive the full document context. ### 4. Repo Scanner (optional) Uses the GitHub MCP server to fetch recent commits, open pull requests, failing CI checks, and branch activity for the service repository. Requires `GITHUB_TOKEN`. ### 5. Terraform Scanner (optional) Uses the Terraform MCP server to inspect workspace state, detect infrastructure drift, and summarise recent plan/apply runs that may correlate with the incident. Requires Terraform MCP configuration. ### 6. Graph Analyzer Executes nine Cypher queries against the Neo4j knowledge graph: direct dependencies, upstream callers, full blast-radius (3-hop transitive closure), recent deployments, historical incidents, associated runbooks, team ownership, configuration change events, and cross-entity incidents. ### 7. Web Intelligence (optional) Runs DuckDuckGo searches (Instant Answer API → HTML scrape fallback) for known provider issues, CVEs, post-mortems, and dependency advisories. Results are deduped by URL. ### 8. Ops Analyst (optional) Uses the custom Ops Inspector MCP server to parse stack traces, calculate error rates, format incident briefs, and check service health from observability tool outputs. ### 9. Crew Intelligence (optional) Runs a three-role CrewAI crew — Researcher → Analyst → Writer — that gathers, correlates, and synthesises external intelligence into a structured intelligence report injected into the analysis context. ### 10. Root Cause Finder Synthesises all pipeline context (graph, web, telemetry, repo, IaC) using an LLM to identify the primary root cause, build a causal chain with confidence scores, detect deployment correlation, and reconstruct the incident timeline. ### 11. Remediator Generates a full remediation plan: immediate kubectl/CLI actions, rollback steps, mitigation steps, escalation paths with Slack contacts, runbook references, and post-incident action items. ### 12. Conversationalist Synthesises all structured pipeline outputs into a human-readable Markdown narrative including a conversation summary for chat history compaction. --- ## Quick Start ### Prerequisites - Docker and Docker Compose - Node.js 20+ - Python 3.11+ with `uv` (install: `pip install uv`) ### 1. Clone and configure ``` git clone https://github.com/your-org/ops-pilot.git cd ops-pilot cp backend/.env.example backend/.env ``` Edit `backend/.env` and set at minimum: - `OPENAI_API_KEY` — your OpenAI key - `SECRET_KEY` — generate with: `openssl rand -hex 32` ### 2. Start infrastructure ``` cd backend docker compose up -d postgres neo4j redis # Allow ~20 s for Neo4j to fully initialise before running migrations ``` ### 3. Run migrations and seed the knowledge graph ``` uv sync uv run alembic upgrade head uv run python -m app.db.neo4j_seed ``` ### 4. Start the API server ``` uv run uvicorn app.main:app --reload --port 8000 # Verify: curl http://localhost:8000/health ``` ### 5. Start Celery (two separate terminals) ``` uv run celery -A app.tasks.celery_app worker --loglevel=info uv run celery -A app.tasks.celery_app beat --loglevel=info ``` ### 6. Start the frontend ``` cd ../frontend npm install cp .env.example .env.local # Set: NEXT_PUBLIC_API_URL=http://localhost:8000 npm run dev # Open http://localhost:3000 ``` --- ## API Routes ``` GET /health Liveness + dependency check POST /api/v1/auth/register Register new user POST /api/v1/auth/login Login, receive JWT tokens POST /api/v1/auth/refresh Refresh access token GET /api/v1/auth/me Get current user profile POST /api/v1/incident/analyze Full analysis (Bearer required) GET /api/v1/stream/incident SSE stream (optional Bearer) POST /api/v1/chat/ Create chat session (Bearer) GET /api/v1/chat/ List chat sessions (Bearer) GET /api/v1/chat/{id} Get chat by ID GET /api/v1/chat/{id}/messages Get messages for chat GET /api/v1/chat/{id}/executions Get agent execution log DELETE /api/v1/chat/{id} Delete chat session GET /api/v1/settings Get LLM settings (Bearer) PUT /api/v1/settings Update LLM settings (Bearer) ``` --- ## Authentication Ops-Pilot uses a dual-token JWT strategy: - **Access token** — short-lived (default 30 min), signed with `SECRET_KEY` using HS256. Sent as `Authorization: Bearer `. - **Refresh token** — long-lived (default 7 days). Used to obtain a new access token via `POST /api/v1/auth/refresh`. - Passwords are hashed with **bcrypt** before storage in PostgreSQL. - The `/api/v1/stream/incident` endpoint accepts an optional Bearer token. Unauthenticated users can still stream results, but their sessions are not persisted to a user account. - `get_current_user` dependency raises 401 for missing or invalid tokens. `get_optional_user` returns `None` for unauthenticated requests without raising. --- ## Security Guardrails Every user query (and document context) passes through `app/core/guardrails.py` before reaching any LLM or agent: 1. **Control character sanitisation** — strips null bytes and non-printable characters. 2. **Length enforcement** — caps input at 4,000 characters (`MAX_QUERY_LENGTH`). 3. **Prompt injection detection** — regex pattern matching against known attack phrases such as "ignore all previous instructions", "forget everything", "you are now", "pretend to be", and "act as". 4. **PII scrubbing** — uses Microsoft Presidio (if installed) to redact emails, phone numbers, credit card numbers, IP addresses, and names. Falls back to regex patterns when Presidio is unavailable. Guardrail violations yield an SSE `error_event` with code `GUARDRAIL_VIOLATION` and terminate the stream immediately. The violation is never forwarded to any LLM. --- ## Streaming (SSE) The `/api/v1/stream/incident` endpoint opens a Server-Sent Events connection and emits `StreamEvent` objects in real time as the orchestrator progresses through the pipeline. ### Event types | Event type | Description | | ----------- | -------------------------------------------------------------------- | | `session` | Emitted first — contains the `session_id` for this analysis turn | | `step` | Agent lifecycle update (start, complete, error, skipped) | | `graph` | Graph Analyzer result — blast-radius, dependencies, runbooks | | `reasoning` | Root Cause Finder result — causal chain, timeline, confidence | | `result` | Final combined output — natural response, structured data, citations | | `error` | Stream-level error (guardrail violation, unexpected exception) | | `done` | Stream closed — session_id echoed for confirmation | ### StreamEvent schema Each event's `data` field is a JSON object that always includes: - `description` — human-readable explanation of what this step does - `input` — the data this step received - `output` — the result produced (on complete events) - `completed_steps` — list of all pipeline steps finished so far - `error` — error message (on error events only) ### Frontend consumption The frontend calls `streamIncident()` in `app/lib/apis.ts`, which opens an `EventSource` and dispatches each parsed event to `recordExplainabilityEvent()`. The explainability panel in the chat UI renders each event as a clickable card showing the step name, status, and a hover preview of input/output. Clicking opens a modal with full detail. --- ## Neo4j Knowledge Graph The graph models the live service dependency topology of your infrastructure. The `GraphAnalyzerAgent` runs nine Cypher queries per incident turn: 1. **Direct dependencies** — services directly called by the affected service 2. **Upstream callers** — services that call into the affected service 3. **Blast radius** — 3-hop transitive closure of all affected nodes 4. **Deployments** — recent deployments across the blast-radius services 5. **Historical incidents** — past incidents on related services 6. **Runbooks** — associated runbook documents with URLs 7. **Ownership** — team ownership records with Slack channels 8. **Config changes** — recent configuration change events 9. **Cross-entity incidents** — incidents touching any extracted entity ### Seeding the knowledge graph ``` uv run python -m app.db.neo4j_seed ``` This creates Service, Deployment, Incident, Runbook, Team, and ConfigChange nodes with realistic relationships for a sample e-commerce microservices topology (checkout, payment, inventory, api-gateway, redis, postgres, etc.). ### Celery graph maintenance The `sync_web_intelligence_to_graph` Celery task (runs hourly) writes `WebKnowledge` nodes back into Neo4j from web search findings, enabling the graph to accumulate external intelligence over time. --- ## LLM Provider Switching Edit `backend/.env` — no rebuild required: ``` # OpenAI (default) LLM_PROVIDER=openai LLM_MODEL=gpt-4o OPENAI_API_KEY=sk-... # Anthropic Claude LLM_PROVIDER=anthropic LLM_MODEL=claude-3-5-sonnet-20241022 ANTHROPIC_API_KEY=sk-ant-... # Google Gemini LLM_PROVIDER=google LLM_MODEL=gemini-1.5-pro GOOGLE_API_KEY=AIza... ``` The LLM provider is resolved at startup in `app/core/llm.py`. All agents use `llm.with_structured_output(PydanticModel)` for type-safe structured outputs. --- ## Alembic Migrations ``` uv run alembic upgrade head apply all pending migrations uv run alembic revision --autogenerate -m "describe change" generate a new revision uv run alembic downgrade -1 roll back one step uv run alembic downgrade roll back to a specific hash uv run alembic current show current applied revision uv run alembic check assert no unapplied migrations uv run alembic history … ## Source & license This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [Khaledayman9](https://github.com/Khaledayman9) - **Source:** [Khaledayman9/ops-pilot](https://github.com/Khaledayman9/ops-pilot) - **License:** Apache-2.0 Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** yes - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** yes - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: flagged — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/mcp-khaledayman9-ops-pilot - Seller: https://agentstack.voostack.com/s/khaledayman9 - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.