Ci Cd Pipeline

CI/CD Pipeline Engineering

Philosophy

A CI/CD pipeline isn't a YAML file — it's the immune system of your codebase. Every merge to main should be a non-event. If deploying makes you nervous, your pipeline is broken.

Core principles:

• Fast feedback: developers should know if they broke something within 5 minutes
• Reproducible: same commit = same result, every time
• Progressive: unit → integration → e2e → staging → canary → production
• Reversible: any deployment can be rolled back in under 2 minutes

---

GitHub Actions: Complete Production Workflow

Reusable Workflow Architecture

Structure your workflows as composable units. Don't copy-paste between repos.

.github/
├── workflows/
│   ├── ci.yml                  # Main CI pipeline
│   ├── deploy-staging.yml      # Staging deployment
│   ├── deploy-production.yml   # Production deployment
│   └── release.yml             # Release management

The Reusable Workflow Pattern

Create org-level reusable workflows in a .github repository:

# org/.github/.github/workflows/node-ci.yml
name: Node.js CI (Reusable)

on:
  workflow_call:
    inputs:
      node-version:
        type: string
        default: '22'   # 22 = Active LTS in 2026; 20 went EOL 2026-04-30, 18 EOL 2025-04-30
      working-directory:
        type: string
        default: '.'
      run-e2e:
        type: boolean
        default: false
    secrets:
      NPM_TOKEN:
        required: false
      CODECOV_TOKEN:
        required: false

jobs:
  lint-and-typecheck:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
          cache: 'npm'
          cache-dependency-path: '${{ inputs.working-directory }}/package-lock.json'

      - name: Install dependencies
        working-directory: ${{ inputs.working-directory }}
        run: npm ci

      - name: Lint
        working-directory: ${{ inputs.working-directory }}
        run: npm run lint

      - name: Type check
        working-directory: ${{ inputs.working-directory }}
        run: npm run typecheck

  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
          cache: 'npm'
          cache-dependency-path: '${{ inputs.working-directory }}/package-lock.json'

      - run: npm ci
        working-directory: ${{ inputs.working-directory }}

      - name: Unit tests with coverage
        working-directory: ${{ inputs.working-directory }}
        run: npm run test:unit -- --coverage --reporter=junit --outputFile=junit.xml

      - name: Upload coverage
        if: inputs.working-directory == '.'
        uses: codecov/codecov-action@v4
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          flags: unit

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: unit-test-results
          path: ${{ inputs.working-directory }}/junit.xml

  integration-tests:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16-alpine
        env:
          POSTGRES_USER: test
          POSTGRES_PASSWORD: test
          POSTGRES_DB: testdb
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
      redis:
        image: redis:7-alpine
        ports:
          - 6379:6379
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
          cache: 'npm'
          cache-dependency-path: '${{ inputs.working-directory }}/package-lock.json'

      - run: npm ci
        working-directory: ${{ inputs.working-directory }}

      - name: Run migrations
        working-directory: ${{ inputs.working-directory }}
        env:
          DATABASE_URL: postgresql://test:test@localhost:5432/testdb
        run: npm run db:migrate

      - name: Integration tests
        working-directory: ${{ inputs.working-directory }}
        env:
          DATABASE_URL: postgresql://test:test@localhost:5432/testdb
          REDIS_URL: redis://localhost:6379
          NODE_ENV: test
        run: npm run test:integration

  e2e-tests:
    if: inputs.run-e2e
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
          cache: 'npm'

      - run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Build application
        run: npm run build

      - name: Run E2E tests
        run: npx playwright test
        env:
          CI: true

      - name: Upload Playwright report
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 7

Consume it from any repo. Make this local ci.yml itself reusable (on: workflow_call) so your deploy workflow can call it as a gate — without that trigger, uses: ./.github/workflows/ci.yml fails to resolve:

# your-repo/.github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  workflow_call:          # REQUIRED so deploy-production.yml can `uses:` this file
    inputs:
      run-e2e:
        type: boolean
        default: false
    secrets:
      NPM_TOKEN:
        required: false
      CODECOV_TOKEN:
        required: false

jobs:
  ci:
    uses: your-org/.github/.github/workflows/node-ci.yml@v1   # pin to a tag/SHA, not @main
    with:
      node-version: '22'
      # On workflow_call, inherit the caller's run-e2e; on push/PR, derive it.
      run-e2e: ${{ inputs.run-e2e || (github.event_name == 'push' && github.ref == 'refs/heads/main') }}
    secrets:
      NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
      CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

> Pin third-party and org reusable workflows to an immutable tag or full SHA (@v1, @), never @main — a moving ref is a supply-chain foothold. Dependabot's github-actions ecosystem will bump pinned SHAs for you.

Matrix Builds

Use matrices for cross-version testing, but be smart about it. Test only supported runtimes — as of mid-2026 that's the Active LTS (22) and Current (24); 18 (EOL 2025-04-30) and 20 (EOL 2026-04-30) are off the support matrix unless you have a contractual reason to keep them. Check the schedule at https://nodejs.org/en/about/previous-releases:

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false  # Don't cancel other jobs if one fails
      matrix:
        node-version: [22, 24]
        os: [ubuntu-latest]
        include:
          # Only test macOS on Active LTS (saves minutes; macOS minutes cost 10x)
          - node-version: 22
            os: macos-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci
      - run: npm test

Caching Strategies That Actually Work

Node.js — npm ci with built-in cache

- uses: actions/setup-node@v4
  with:
    node-version: '22'
    cache: 'npm'
# npm ci uses the cache automatically. Done.

Docker Layer Caching

GHCR push needs packages: write — without it the push 403s. Pin to current major action versions (as of mid-2026: build-push-action@v6, setup-buildx-action@v3, login-action@v3; verify at https://github.com/docker/build-push-action/releases). Tag by full github.sha and reuse that exact tag downstream, so deploy never references an image that was never pushed:

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write   # REQUIRED to push to ghcr.io with GITHUB_TOKEN
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to GHCR
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:${{ github.sha }}
            ghcr.io/${{ github.repository }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

Turborepo Remote Cache

- name: Build with Turborepo
  run: npx turbo run build --filter=...[origin/main]
  env:
    TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
    TURBO_TEAM: ${{ vars.TURBO_TEAM }}

---

Testing Pyramid: What to Run Where

        /  E2E  \          ← 5-10 critical user journeys. Main merges only.
       / ——————— \
      / Integration \      ← API contracts, DB queries. All PRs.
     / ————————————— \
    /   Unit Tests    \    ← Pure logic, fast. Every push.
   / ————————————————— \

Unit Tests (Every Push)

• Run in > "$GITHUB_OUTPUT"

# ---- Reusable in-cluster deploy job: auth -> kubeconfig -> kubectl. ---- # Realistically this lives in your org .github repo; shown inline for clarity. deploy-staging: needs: build uses: ./.github/workflows/_kube-deploy.yml with: environment: staging namespace: staging deployment: app image: ${{ needs.build.outputs.image }} base-url: https://staging.example.com secrets: inherit

approve-production: needs: deploy-staging runs-on: ubuntu-latest environment: production # Configure required reviewers under Settings → Environments steps:

• run: echo "Production deployment approved"

deploy-canary: needs: [build, approve-production] uses: ./.github/workflows/_kube-deploy.yml with: environment: production namespace: production deployment: app-canary image: ${{ needs.build.outputs.image }} analyze: true # gate on metrics before promoting secrets: inherit

deploy-production: needs: [build, deploy-canary] uses: ./.github/workflows/_kube-deploy.yml with: environment: production namespace: production deployment: app image: ${{ needs.build.outputs.image }} base-url: https://app.example.com secrets: inherit

```yaml
# .github/workflows/_kube-deploy.yml — the reusable deploy unit
name: kube-deploy
on:
  workflow_call:
    inputs:
      environment: { type: string, required: true }
      namespace:   { type: string, required: true }
      deployment:  { type: string, required: true }
      image:       { type: string, required: true }   # full image@sha256 digest
      base-url:    { type: string, default: '' }
      analyze:     { type: boolean, default: false }

permissions:
  contents: read
  id-token: write   # OIDC -> cloud, no stored kube creds

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}
    steps:
      - uses: actions/checkout@v4

      # 1) Cloud auth via OIDC (EKS example; swap for GKE/AKS as needed).
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ vars.AWS_DEPLOY_ROLE_ARN }}
          aws-region: ${{ vars.AWS_REGION }}

      # 2) kubectl binary + 3) cluster kubeconfig.
      # Pin kubectl within +/-1 minor of your cluster's control plane (skew policy).
      - uses: azure/setup-kubectl@v4
        with: { version: ${{ vars.KUBECTL_VERSION }} }   # e.g. 'v1.33.x' for a 1.32/1.33 cluster
      - name: Configure kubeconfig
        run: aws eks update-kubeconfig --name ${{ vars.EKS_CLUSTER }} --region ${{ vars.AWS_REGION }}
      # GKE alt: google-github-actions/get-gke-credentials@v2
      # AKS alt: az aks get-credentials --resource-group RG --name CLUSTER

      - name: Roll out
        run: |
          kubectl set image deployment/${{ inputs.deployment }} \
            app=${{ inputs.image }} --namespace=${{ inputs.namespace }}
          kubectl rollout status deployment/${{ inputs.deployment }} \
            --namespace=${{ inputs.namespace }} --timeout=300s

      # Canary metric gate. The runner is OUTSIDE the cluster, so do NOT curl an
      # in-cluster `http://prometheus:9090`. Use one of:
      #   - a controller that analyzes for you (Argo Rollouts / Flagger), or
      #   - your managed/external metrics API (Datadog, Grafana Cloud, AMP), or
      #   - `kubectl port-forward` to reach in-cluster Prometheus over localhost.
      - name: Analyze canary (port-forward to in-cluster Prometheus)
        if: inputs.analyze
        run: |
          kubectl -n monitoring port-forward svc/prometheus 9090:9090 &
          PF_PID=$!; trap 'kill $PF_PID' EXIT
          for i in $(seq 1 30); do
            ERROR_RATE=$(curl -s "http://localhost:9090/api/v1/query" \
              --data-urlencode 'query=sum(rate(http_requests_total{status=~"5..",deployment="canary"}[1m])) / sum(rate(http_requests_total{deployment="canary"}[1m]))' \
              | jq -r '.data.result[0].value[1] // "0"')
            if awk "BEGIN{exit !(${ERROR_RATE:-0} > 0.05)}"; then
              echo "Canary error rate ${ERROR_RATE} exceeds 5% — rolling back"
              kubectl rollout undo deployment/${{ inputs.deployment }} --namespace=${{ inputs.namespace }}
              exit 1
            fi
            echo "Canary healthy (error rate: ${ERROR_RATE})"; sleep 10
          done

      - name: Smoke tests
        if: inputs.base-url != ''
        run: |
          curl --retry 5 --retry-all-errors --retry-delay 3 -sf "${{ inputs.base-url }}/healthz"
          npm ci && npm run test:smoke -- --base-url="${{ inputs.base-url }}"

      - name: Auto-rollback + notify on failure
        if: failure()
        run: |
          kubectl rollout undo deployment/${{ inputs.deployment }} --namespace=${{ inputs.namespace }} || true
          curl -X POST "${{ secrets.SLACK_WEBHOOK }}" \
            -H 'Content-Type: application/json' \
            -d "{\"text\":\"${{ inputs.environment }}/${{ inputs.deployment }} deploy failed — auto-rolled back\"}"

> Prefer a progressive-delivery controller over hand-rolled canary bash. Argo Rollouts (Rollout CRD with analysis templates) and Flagger automate traffic shifting, metric analysis (Prometheus/Datadog), and automatic rollback in-cluster — so your pipeline just pushes the digest and watches kubectl argo rollouts status. The bash gate above is the from-scratch fallback when you have no controller.

---

Supply-Chain Security: SLSA Provenance + Keyless Signing

By 2026, signing artifacts and attaching verifiable provenance is table stakes, and admission controllers reject unsigned images. GitHub's native artifact attestations generate SLSA-style provenance and sign it with Sigstore keyless (Fulcio short-lived certs tied to the workflow's OIDC identity — no private keys to store or rotate). Targets SLSA Build Level 3 when run from a non-falsifiable build.

Generate provenance + sign at build time

  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
      id-token: write       # OIDC identity for keyless signing (Fulcio)
      attestations: write   # required by attest-build-provenance
    outputs:
      image: ${{ steps.out.outputs.i

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [san-npm](https://github.com/san-npm)
- **Source:** [san-npm/skills-ws](https://github.com/san-npm/skills-ws)
- **License:** MIT
- **Homepage:** https://skills-ws.vercel.app

Install and usage instructions live in the source repository linked above.