# Cicd Pipeline Patterns

> Use when designing or reviewing CI/CD pipelines — covers pipeline design (stages, jobs, parallelism), GitOps and IaC patterns, build optimization (caching, incremental builds, artifact management), deployment strategies (blue-green, canary, rolling update, recreate), security in pipelines (SLSA, SBOM, supply chain, secrets management), testing in pipelines (shift-left, test pyramid, flaky test ma…

- **Type:** Skill
- **Install:** `agentstack add skill-mickeyyaya-refactoring-skills-cicd-pipeline-patterns`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [mickeyyaya](https://agentstack.voostack.com/s/mickeyyaya)
- **Installs:** 0
- **Category:** [Developer Tools](https://agentstack.voostack.com/c/developer-tools)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [mickeyyaya](https://github.com/mickeyyaya)
- **Source:** https://github.com/mickeyyaya/refactoring-skills/tree/main/skills/cicd-pipeline-patterns

## Install

```sh
agentstack add skill-mickeyyaya-refactoring-skills-cicd-pipeline-patterns
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# CI/CD Pipeline Patterns

## Overview

Slow pipelines block delivery, misconfigured secrets become attack vectors, and poorly designed deployment strategies cause downtime. Use this guide when designing, reviewing, or improving CI/CD pipelines across GitHub Actions, GitLab CI, and general pipeline tooling.

**When to use:** Designing a new pipeline from scratch; reviewing an existing pipeline for speed or reliability; evaluating deployment strategies for a release; hardening supply chain security; investigating flaky builds or slow test suites.

## Quick Reference

| Pattern | Core Idea | Primary Red Flag |
|---------|-----------|-----------------|
| Pipeline Design | Stages, jobs, fan-out/fan-in with explicit dependencies | All jobs sequential, one massive job doing everything |
| GitOps / IaC | Git is the single source of truth for infra and config | Manual kubectl apply, config drift between environments |
| Build Optimization | Cache dependencies, reuse artifacts, skip unchanged work | `npm install` from scratch on every run, no layer caching |
| Blue-Green Deployment | Two identical envs; route traffic atomically | Deploy in place with downtime, no rollback path |
| Canary Deployment | Gradually shift traffic to new version | All-or-nothing deploy with no traffic splitting |
| Rolling Update | Replace instances incrementally, keep service alive | Recreate all pods at once, causing downtime |
| SLSA / SBOM | Provenance and software bill of materials for supply chain | No artifact signing, no dependency audit |
| Shift-Left Testing | Run tests as early and often as possible | Integration tests only, no unit tests in pipeline |
| Flaky Test Management | Quarantine and track unstable tests, never ignore them | Retrying flaky tests silently, masking real failures |
| Rollback & DR | Automated rollback triggers, tested recovery procedures | Manual rollback, untested restore procedures |

---

## Patterns in Detail

### 1. Pipeline Design — Stages, Jobs, and Parallelism

Well-structured pipelines define explicit stages (lint, test, build, deploy) with dependency graphs that maximize parallelism. Each job should do exactly one thing and produce a clear artifact or signal.

**Red Flags:**
- Single job that lints, tests, builds, and deploys in sequence
- No dependency graph — all jobs run sequentially by default
- Jobs downloading the same dependency multiple times
- No timeout per job — a hanging test blocks the whole pipeline
- Missing `needs` / `depends_on` declarations — implicit ordering assumptions

**GitHub Actions — parallel jobs with fan-in:**
```yaml
# .github/workflows/ci.yml
on:
  push: { branches: [main, "release/**"] }
  pull_request:

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci && npm run lint

  unit-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci && npm test -- --coverage
      - uses: actions/upload-artifact@v4
        with: { name: coverage-report, path: coverage/ }

  build:
    needs: [lint, unit-test]   # fan-in
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci && npm run build
      - uses: actions/upload-artifact@v4
        with: { name: dist, path: dist/ }

  deploy-staging:
    needs: build
    environment: staging
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with: { name: dist, path: dist/ }
      - run: ./scripts/deploy.sh staging
```

GitLab CI: equivalent via `stages:` + `needs:` + `rules:`; matrix via `parallel: { matrix: [...] }`.

---

### 2. GitOps and Infrastructure as Code

GitOps treats Git as the single source of truth. Every change to infrastructure or configuration is expressed as a pull request, reviewed, and reconciled automatically by an operator (Argo CD, Flux). Manual `kubectl apply` or console changes create drift and are immediately overwritten by the reconciler.

**Red Flags:**
- Config applied manually; Git repo does not reflect live state
- Infrastructure defined only in runbooks, not in code
- Separate processes for app deployment and infra provisioning (no single pane)
- No drift detection — live state can diverge from declared state
- Secrets stored in Git as plaintext instead of sealed/external secrets

**Argo CD application manifest:**
```yaml
# k8s/argocd/app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/my-app-config
    targetRevision: HEAD
    path: k8s/overlays/production
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true      # remove resources deleted from Git
      selfHeal: true   # revert manual changes automatically
    syncOptions:
      - CreateNamespace=true
```

**Terraform pipeline (GitHub Actions):**
```yaml
# plan job: terraform init + plan -out=tfplan → upload artifact
# apply job: needs: plan, environment: production (manual approval gate) → download + apply
jobs:
  plan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      - run: terraform init && terraform plan -out=tfplan
        env: { TF_VAR_environment: staging }
      - uses: actions/upload-artifact@v4
        with: { name: tfplan, path: tfplan }
  apply:
    needs: plan
    environment: production
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with: { name: tfplan }
      - run: terraform apply tfplan
```

Cross-reference: `feature-flags-progressive-delivery` — progressive rollouts via GitOps-managed flag configs.

---

### 3. Build Optimization — Caching, Incremental Builds, Artifact Management

Every minute shaved from a pipeline is a minute developers spend on code instead of waiting. The three levers are: dependency caching (avoid re-downloading), layer/build caching (avoid re-compiling unchanged code), and artifact reuse (avoid re-building what already passed).

**Red Flags:**
- `npm install` / `pip install` / `go mod download` on every run with no cache
- Docker image rebuilt from `FROM` on every commit because no layer caching
- Test artifacts not uploaded — downstream jobs re-run tests already passed
- Build matrix with no cache key scoping — one branch pollutes another's cache
- Artifacts with no expiry — storage cost grows unbounded

**Dependency caching (GitHub Actions):**
```yaml
- uses: actions/setup-node@v4
  with:
    node-version: 20
    cache: npm           # built-in: caches ~/.npm keyed by package-lock.json hash

# For monorepos — scope cache key to workspace
- uses: actions/cache@v4
  with:
    path: |
      ~/.npm
      apps/api/node_modules
      apps/web/node_modules
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
    restore-keys: ${{ runner.os }}-npm-
```

**Docker layer caching with BuildKit:**
```yaml
- uses: docker/build-push-action@v5
  with:
    context: .
    push: true
    tags: ghcr.io/org/app:${{ github.sha }}
    cache-from: type=registry,ref=ghcr.io/org/app:buildcache
    cache-to: type=registry,ref=ghcr.io/org/app:buildcache,mode=max
```

**Incremental build — skip unchanged packages:**
```bash
#!/usr/bin/env bash
set -euo pipefail
BASE=${BASE_BRANCH:-origin/main}
CHANGED=$(git diff --name-only "$BASE"...HEAD | xargs -I{} dirname {} | sort -u)
for PKG in packages/*/; do
  PKG_NAME=$(basename "$PKG")
  echo "$CHANGED" | grep -q "^packages/$PKG_NAME" \
    && npm run build --workspace="$PKG" \
    || echo "Skipping $PKG_NAME (unchanged)"
done
```

---

### 4. Deployment Strategies

The right deployment strategy depends on your tolerance for downtime, traffic splitting capability, and rollback speed requirements.

**Red Flags:**
- Recreate strategy on a production service with no maintenance window
- Blue-green deployed but old (blue) environment torn down immediately — no rollback window
- Canary with no metrics gate — traffic shifted based on time, not signal
- Rolling update with `maxUnavailable: 100%` — equivalent to recreate
- No smoke test after deploy before serving traffic

**Blue-Green Deployment (GitHub Actions + AWS ECS):**
```yaml
deploy-blue-green:
  runs-on: ubuntu-latest
  steps:
    - run: aws ecs update-service --cluster prod --service my-app-green --force-new-deployment
    - run: aws ecs wait services-stable --cluster prod --services my-app-green
    - run: curl --fail https://green.internal.example.com/health  # smoke test
    - run: |
        aws elbv2 modify-listener \
          --listener-arn "$LISTENER_ARN" \
          --default-actions Type=forward,TargetGroupArn="$GREEN_TG_ARN"
    # Keep Blue running for 30 min rollback window; tear down in post-deploy job
```

**Canary Deployment (Argo Rollouts — essential fields):**
```yaml
# kind: Rollout  spec.strategy.canary
canaryService: my-app-canary
stableService: my-app-stable
steps:
  - setWeight: 10
  - pause: { duration: 5m }
  - analysis: { templates: [{ templateName: success-rate }] }
  - setWeight: 50
  - pause: { duration: 10m }
  - setWeight: 100
```

**Rolling Update (Kubernetes — essential fields):**
```yaml
spec:
  replicas: 6
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2        # allow 2 extra pods during rollout
      maxUnavailable: 0  # never take a pod down before new one is ready
  template:
    spec:
      containers:
        - name: app
          readinessProbe:
            httpGet: { path: /health, port: 8080 }
```

Cross-reference: `feature-flags-progressive-delivery` — combine canary with feature flags for traffic splitting at the application layer.

---

### 5. Security in Pipelines — SLSA, SBOM, Supply Chain, Secrets Management

**Red Flags:**
- Secrets stored in repository or build logs
- Third-party actions pinned to a mutable tag (`@v3`) instead of a commit SHA
- No artifact signing — anyone could inject a tampered artifact
- No dependency audit — known-vulnerable packages ship to production
- Pipeline permissions are `permissions: write-all` by default
- No SBOM generated — no visibility into what is in the deployed artifact

**Least-privilege permissions + SHA pinning:**
```yaml
permissions:
  contents: read
  id-token: write  # only for OIDC

# WRONG — mutable tag, supply chain risk
- uses: actions/checkout@v4
# CORRECT — immutable SHA
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683  # v4.2.2
```

**SBOM generation and artifact signing (cosign + syft):**
```yaml
- uses: anchore/sbom-action@v0
  with:
    image: ghcr.io/org/app:${{ github.sha }}
    artifact-name: sbom.spdx.json
    output-file: sbom.spdx.json

- name: Sign image with cosign (OIDC keyless)
  run: cosign sign --yes ghcr.io/org/app:${{ github.sha }}

- name: Attest SBOM
  run: |
    cosign attest --yes \
      --predicate sbom.spdx.json \
      --type spdxjson \
      ghcr.io/org/app:${{ github.sha }}
```

**Secrets management — OIDC + Vault:**
```yaml
- uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::123456789:role/github-actions-deploy
    aws-region: us-east-1

- uses: hashicorp/vault-action@v3
  with:
    url: https://vault.example.com
    method: jwt
    path: jwt/github
    secrets: |
      secret/data/prod/db DB_PASSWORD | DB_PASSWORD ;
      secret/data/prod/api API_KEY | API_KEY
```

**Dependency audit:**
```bash
#!/usr/bin/env bash
set -euo pipefail
npm audit --audit-level=high
npx license-checker --onlyAllow "MIT;Apache-2.0;BSD-2-Clause;BSD-3-Clause;ISC"
trivy image --exit-code 1 --severity HIGH,CRITICAL "ghcr.io/org/app:${IMAGE_TAG}"
```

Cross-reference: `security-patterns-code-review` — SAST, secrets detection, and dependency vulnerability patterns.

---

### 6. Testing in Pipelines — Shift-Left, Test Pyramid, Flaky Test Management

**Red Flags:**
- Only integration or E2E tests in CI — slow feedback, hard to isolate failures
- E2E tests run on every commit to every branch — unnecessary cost and time
- Flaky tests retried silently — masking intermittent failures in production code
- No coverage threshold enforced — coverage regresses unnoticed
- Tests that require a live database on the unit test stage

**Shift-left — test pyramid in CI:**

```
         /\
        /E2E\         — run on main/release branches only (slow, costly)
       /------\
      /Integr. \      — run on PRs, keyed to changed services
     /----------\
    / Unit Tests  \   — run on every commit, every branch (fast, cheap)
   /--------------\
```

**Unit tests with coverage gate (GitHub Actions):**
```yaml
unit-test:
  runs-on: ubuntu-latest
  timeout-minutes: 10
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with: { node-version: 20, cache: npm }
    - run: npm ci && npm test -- --coverage --coverageThreshold='{"global":{"lines":80}}'
```

**Integration tests with service container (GitHub Actions):**
```yaml
integration-test:
  services:
    postgres: { image: postgres:16-alpine, env: { POSTGRES_DB: testdb, POSTGRES_USER: test, POSTGRES_PASSWORD: test } }
  steps:
    - run: npm ci && npm run db:migrate && npm run test:integration
      env: { DATABASE_URL: postgres://test:test@localhost:5432/testdb }
```
GitLab CI: equivalent via `services:` block with `alias:` and job-level `variables:`.

**Flaky test quarantine:**
```bash
#!/usr/bin/env bash
set -euo pipefail
# Stable tests — failure fails the build
npx jest --testPathIgnorePatterns=".*\\.flaky\\.test\\.ts" --ci
# Quarantined tests — collect results, do not fail build
npx jest --testPathPattern=".*\\.flaky\\.test\\.ts" --ci || {
  echo "FLAKY_FAILURES=true" >> "$GITHUB_ENV"
  curl -s -X POST "$FLAKY_TRACKER_URL" \
    -H "Content-Type: application/json" \
    -d "{\"branch\": \"$GITHUB_REF_NAME\", \"run\": \"$GITHUB_RUN_ID\"}"
}
```

**Playwright E2E — main branch only, with retries:**
```yaml
e2e:
  runs-on: ubuntu-latest
  if: github.ref == 'refs/heads/main'
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with: { node-version: 20, cache: npm }
    - run: npm ci && npx playwright install --with-deps
    - run: npx playwright test --retries=2
    - uses: actions/upload-artifact@v4
      if: failure()
      with: { name: playwright-report, path: playwright-report/ }
```

Cross-reference: `testing-patterns` — unit, integration, and E2E test structure patterns; contract testing for microservices.

---

### 7. Pipeline Anti-Patterns

| Anti-Pattern | Description | Fix |
|-------------|-------------|-----|
| **Long-Running Pipeline** | Single pipeline takes >30 min | Split into fast (lint/unit) and slow (E2E) tiers |
| **Snowflake Configuration** | Each service has its own hand-crafted pipeline | Extract reusable workflow templates or shared includes |
| **Manual Gates Everywhere** | Every stage requires human approval | Automate quality gates; manual approval only at prod boundary |
| **Mutable Artifacts** | Same artifact tag overwritten on every build | Tag artifacts with commit SHA; treat tags as immutable |
| **God Job** | One job lints, tests, builds, scans, and deploys | Single responsibility per job; explicit `needs` graph |
| **No Timeout** | Jobs hang indefinitely on network or test failures | Set `timeout-minutes` on every job |
| **Secret in Logs** | `echo $SECRET_TOKEN` prints secrets to build log | Never echo secrets; use `::add-mask::` if unavoidable |
| **Test After Build** | Tests run only after a slow Docker build | Run unit tests before the build; fail fast on code issues |
| **Pipeline as Documentation** | Complex logic buried in YAML | Extract logic into scripts; keep YAML as thin orchestration

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [mickeyyaya](https://github.com/mickeyyaya)
- **Source:** [mickeyyaya/refactoring-skills](https://github.com/mickeyyaya/refactoring-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-mickeyyaya-refactoring-skills-cicd-pipeline-patterns
- Seller: https://agentstack.voostack.com/s/mickeyyaya
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.