Devops Platform

DevOps & Platform Engineering

Approach every platform task as the engineer who gets paged at 3am when it breaks. The pipeline isn't done when it works on the happy path — it's done when a failure is easy to detect, easy to explain, and easy to reverse. Every config decision here exists to serve one of those three properties.

---

Step 0: Ground the Platform Work

Before writing any pipeline, container, or deployment config, get clear on:

1. What is being deployed? Stateless service, stateful service, batch job, scheduled task — each has a different deployment shape.
2. What are the environments? Dev, staging, prod — what's the promotion path between them, and what gates it?
3. What's the rollback strategy? If this deploy fails, how long to last-known-good, and is that automatic or manual?
4. What does healthy look like? Define the health check before writing the deployment manifest, not after.
5. Who gets paged, and on what signal? Observability and alerting are part of the deployment, not an afterthought bolted on later.

If the user hasn't specified these, don't guess silently — a deployment plan built on the wrong assumption (e.g., stateless when it's actually stateful) causes real incidents. Ask, or state the assumption you're making and why.

---

Docker

# Stage 1: Dependencies
FROM node:20-alpine AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

# Stage 2: Build
FROM node:20-alpine AS builder
WORKDIR /app
COPY . .
COPY --from=deps /app/node_modules ./node_modules
RUN npm run build

# Stage 3: Runtime — minimal, non-root
FROM node:20-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 appgroup && \
    adduser --system --uid 1001 --ingroup appgroup appuser

COPY --from=builder --chown=appuser:appgroup /app/dist ./dist
COPY --from=deps --chown=appuser:appgroup /app/node_modules ./node_modules

USER appuser
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD wget -qO- http://localhost:3000/health || exit 1

CMD ["node", "dist/main.js"]

Why each rule matters:

• Pin base image versions — latest means the image you tested isn't the image you shipped; you lose the ability to say exactly what's running.
• Multi-stage builds — the production image should contain nothing an attacker could use if they got a shell: no compilers, no source, no dev dependencies.
• Non-root user — a container escape from a root process is a host compromise; from a non-root process it's much more contained.
• .dockerignore — exclude node_modules, .git, *.md, tests, local .env. Keeps build context small and secrets out of layers.
• HEALTHCHECK — without it, the orchestrator can only tell if the process is running, not if it's actually serving traffic correctly.

---

Docker Compose (Local Dev)

services:
  app:
    build: .
    ports:
      - "3000:3000"
    env_file: .env
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://localhost:3000/health"]
      interval: 30s
      timeout: 5s
      retries: 3

  db:
    image: postgres:16-alpine
    volumes:
      - pg_data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: appdb
      POSTGRES_USER: appuser
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U appuser -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 5

  cache:
    image: redis:7-alpine
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 3

volumes:
  pg_data:
  redis_data:

depends_on: condition: service_healthy matters more than it looks — without it, app can start before db is actually accepting connections, causing flaky failures that only show up under load or on a slow CI runner.

---

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: user-service
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0        # zero-downtime rollout
  template:
    metadata:
      labels:
        app: user-service
    spec:
      containers:
        - name: user-service
          image: registry/user-service:1.2.3   # pinned tag, never latest
          ports:
            - containerPort: 3000
          resources:
            requests:
              cpu: "100m"
              memory: "128Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"
          livenessProbe:
            httpGet:
              path: /health
              port: 3000
            initialDelaySeconds: 10
            periodSeconds: 30
          readinessProbe:
            httpGet:
              path: /ready
              port: 3000
            initialDelaySeconds: 5
            periodSeconds: 10
          envFrom:
            - configMapRef:
                name: user-service-config
            - secretRef:
                name: user-service-secrets
          securityContext:
            runAsNonRoot: true
            runAsUser: 1001
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: user-service-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: user-service
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: user-service-netpol
spec:
  podSelector:
    matchLabels:
      app: user-service
  policyTypes: [Ingress, Egress]
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: api-gateway
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: postgres

Mandatory for every workload, and why:

• resources.requests/limits — without limits, one misbehaving pod starves every other pod on the node ("noisy neighbor").
• livenessProbe + readinessProbe — liveness restarts a stuck pod; readiness stops traffic from reaching a pod that's up but not ready (e.g., still warming a cache). Conflating the two causes either premature restarts or traffic sent to dead pods.
• maxUnavailable: 0 — guarantees the rollout never drops below full capacity; combine with PodDisruptionBudget so voluntary disruptions (node drains, cluster upgrades) respect the same floor.
• NetworkPolicy — by default every pod in a cluster can talk to every other pod; this is almost never what you want in production. Default-deny and explicitly allow.
• securityContext — non-root, no privilege escalation, read-only root filesystem wherever the app allows it.
• namespace — never deploy to default; it makes RBAC and quota scoping meaningless.

Horizontal Pod Autoscaler:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: user-service-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: user-service
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

---

Deployment Strategies

Rolling update (above) is the right default, but not always the right choice. Match the strategy to what failure would cost:

| Strategy | How it works | Use when | |---|---|---| | Rolling update | Replace pods incrementally, old and new versions serve traffic simultaneously | Default choice; stateless services tolerant of brief version-skew | | Blue-green | Deploy full new environment, cut traffic over at once (e.g., via Service selector or load balancer swap) | Need instant, clean rollback and can afford 2x resource cost during cutover | | Canary | Route a small % of traffic to the new version, expand gradually while watching error rate/latency | High-risk changes, large user base, want automated rollback on regression |

If the user hasn't said which they want, rolling update is the safe default — but flag it if the change sounds risky (schema migration, payment path, auth changes) and canary or blue-green would catch a bad deploy before it hits everyone.

---

CI/CD Pipeline

Stage order, no skipping:

lint → type-check → unit-test → build → integration-test → security-scan → deploy-staging → [approval] → deploy-production

name: CI/CD Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint-and-type-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm run lint
      - run: npm run type-check

  unit-test:
    needs: lint-and-type-check
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run test:unit -- --coverage
      - uses: actions/upload-artifact@v4
        with:
          name: coverage-report
          path: coverage/

  build:
    needs: unit-test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build Docker image
        run: docker build -t $IMAGE_NAME:${{ github.sha }} .
      - name: Generate SBOM
        run: syft $IMAGE_NAME:${{ github.sha }} -o spdx-json > sbom.json
      - name: Push to registry
        run: docker push $IMAGE_NAME:${{ github.sha }}
      - name: Sign image
        run: cosign sign --yes $IMAGE_NAME:${{ github.sha }}

  security-scan:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Scan image with Trivy
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: $IMAGE_NAME:${{ github.sha }}
          severity: 'HIGH,CRITICAL'
          exit-code: '1'

  deploy-staging:
    needs: security-scan
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - name: Deploy to staging
        run: kubectl set image deployment/app app=$IMAGE_NAME:${{ github.sha }}

  deploy-production:
    needs: deploy-staging
    runs-on: ubuntu-latest
    environment: production   # requires manual approval, configured in repo settings
    steps:
      - name: Deploy to production
        run: kubectl set image deployment/app app=$IMAGE_NAME:${{ github.sha }}

Rules, with reasoning:

• Secrets live in the platform's secret store (GitHub Secrets, GitLab CI Variables, Azure Key Vault) — never in YAML. A secret committed once is compromised forever, even if you delete it later, because git history and CI logs persist it.
• Cache dependencies between runs (node_modules, pip cache, Docker layers) — this is the single biggest lever on pipeline speed, and slow pipelines get bypassed under pressure.
• Security scanning is mandatory, not optional — Trivy for containers, npm audit/pip audit for dependencies. SBOM generation (Syft) and image signing (cosign) matter for supply-chain integrity if you distribute images externally or operate under compliance requirements (SOC 2, FedRAMP).
• Production requires manual approval — the one gate that stops "it passed CI so it must be fine" from becoming an incident.
• Every run produces an artifact tagged with the commit SHA — traceability from "what's running in prod" back to "what code that actually is" is what makes incident response fast instead of archaeological.

GitOps alternative: instead of the pipeline pushing to the cluster directly (kubectl set image above), a GitOps controller (ArgoCD, Flux) watches a Git repo of manifests and reconciles the cluster to match. The pipeline's job becomes "update the manifest repo," not "talk to the cluster." This gives you: the cluster state is always exactly what's in Git (drift is auto-corrected), and rollback is git revert. Prefer this pattern when the user already has or wants a dedicated ops/manifest repo, or cares about audit trails on cluster changes.

---

Infrastructure as Code

Always define infrastructure in code — Terraform or Pulumi — never make manual console changes to production. The reasoning: a console change is invisible to everyone else, has no review step, and leaves no record for the next incident's "what changed recently" question.

terraform {
  backend "s3" {
    bucket         = "company-terraform-state"
    key            = "production/network/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-locks"   # state locking — prevents concurrent applies corrupting state
    encrypt        = true
  }
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "production-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["us-east-1a", "us-east-1b", "us-east-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]

  enable_nat_gateway = true
  single_nat_gateway = false   # one per AZ for production; true is fine for dev to save cost
}

• Remote state (S3+DynamoDB for Terraform, or equivalent) with locking — two engineers running apply at once without a lock corrupts state.
• Separate state per environment — a bug in a terraform destroy against staging should be structurally unable to touch prod's state file.
• Module-ize repeated infrastructure rather than copy-pasting between environments; drift between "what dev has" and "what prod has" is a recurring source of "works on staging, breaks in prod."

Key services by cloud:

| Concern | AWS | Azure | GCP | |---|---|---|---| | Container orchestration | EKS / ECS Fargate | AKS | GKE | | Serverless | Lambda | Azure Functions | Cloud Run | | Managed DB | RDS / Aurora | Azure SQL | Cloud SQL | | Cache | ElastiCache | Azure Cache for Redis | Memorystore | | Object storage | S3 | Blob Storage | Cloud Storage | | CDN | CloudFront | Azure CDN | Cloud CDN | | Secrets | Secrets Manager | Key Vault | Secret Manager | | Message queue | SQS / SNS | Service Bus | Pub/Sub | | IaC state backend | S3 + DynamoDB | Azure Storage + blob lease | GCS + Cloud Storage lock | | AI/ML | Bedrock | Azure OpenAI | Vertex AI |

---

DNS & Certificate Management

DNS and TLS are the two things that are invisible when they work and catastrophic when they don't. Both need to be versioned, monitored, and automated — not manually touched in a console at 2am.

DNS

Rules:

• All DNS records are defined in code (Terraform aws_route53_record, google_dns_record_set, Cloudflare Terraform provider). Manual console edits to production DNS records are the same class of mistake as manual production database edits — they happen once, they work, and then nobody can explain what broke six months later.
• TTL strategy: Use low TTLs (60–300s) before a planned migration or cutover. Use high TTLs (300–3600s) in steady state to reduce resolver load and improve cache hit rate. Change the TTL down 24 hours before you need it to propagate, not the day of.
• Never delete a record until you've verified nothing depends on it. DNS propagation means some clients are still using a record long after you think you've removed it. Audit → reduce TTL → verify no traffic → delete.

Common DNS operations:

# Verify DNS resolution from multiple resolvers
dig api.yourapp.com @8.8.8.8       # Google
dig api.yourapp.com @1.1.1.1       # Cloudflare
dig api.yourapp.com @208.67.222.222 # OpenDNS

# Check propagation globally
# Use: https://dnschecker.org — paste record, check worldwide

# Check what's currently resolving (including cached TTL remaining)
dig +ttl api.yourapp.com

# Trace the full DNS resolution chain
dig +trace api.yourapp.com

# Check all records for a domain
dig api.yourapp.com ANY

# Verify a CNAME points to the right target
dig CNA

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [CODE-SAURABH](https://github.com/CODE-SAURABH)
- **Source:** [CODE-SAURABH/OpenSkills](https://github.com/CODE-SAURABH/OpenSkills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.