Ci Cd

CI/CD Architect

Design fast, reliable, and secure CI/CD pipelines across GitHub Actions, GitLab CI, and CircleCI. Follows Google's DevOps capabilities and DORA metrics.

Decision Framework

Before building a CI/CD pipeline, answer:

• Where is the code hosted? → If GitHub, start with GitHub Actions. If GitLab, use GitLab CI. On-prem? Consider self-hosted.
• What's the deployment target? → Cloud (use OIDC), on-prem (use self-hosted runner), multi-cloud (use environment-specific jobs)
• Is the team size 1-3? → Simple single-workflow. 10+? → Separate build, test, deploy workflows with artifact passing.
• Do you need matrix builds (multiple OS/versions)? → Yes for libraries, no for single-platform apps.
• Is the deploy target production? → Require manual approval gates. Non-prod: automatic on merge.

Workflow

Step 1: Choose platform

| Platform | Best for | Config location | |----------|----------|----------------| | GitHub Actions | OSS, GitHub ecosystem | .github/workflows/*.yml | | GitLab CI | Self-hosted, monorepos | .gitlab-ci.yml | | CircleCI | Performance, Docker | .circleci/config.yml |

Decision: If the project is on GitHub.com, use GitHub Actions. If self-hosted GitLab, use GitLab CI. If maximum performance needed, use CircleCI.

Step 2: Generate pipeline

Use the scaffold script with your platform and stack:

scripts/generate-pipeline.sh --platform github --language node --e2e --docker
scripts/generate-pipeline.sh --platform gitlab --language python --docker
scripts/generate-pipeline.sh --platform circle --language go --e2e

This generates a production-ready pipeline with:

• Lint → typecheck → test (sharded) → build → docker → deploy
• Caching (npm/pip/go, Docker layers)
• OIDC auth (no static secrets)
• Environment gates (staging → production)

Step 3: Configure caching

| Cache type | GitHub Actions | GitLab CI | CircleCI | |-----------|---------------|-----------|----------| | npm/pip/go | actions/cache with lockfile hash | cache:key: with lockfile hash | save_cache / restore_cache | | Docker layers | docker/build-push-action GHA cache | Docker layer caching on self-hosted | Remote Docker engine cache | | Playwright browsers | npx playwright install chromium | before_script cache | Custom Docker image with browsers | | Build artifacts | upload-artifact / download-artifact | artifacts: section | persist_to_workspace |

If the build is consistently >15 min: add more shards or parallelize independent jobs.

Test sharding

# GitHub Actions: split tests across N parallel runners
strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: npx vitest --shard=${{ matrix.shard }}/${{ strategy.job-total }}

In GitLab CI, use the parallel: key: parallel: 4 and CI_NODE_INDEX / CI_NODE_TOTAL env vars.

Decision: Use sharding when test suite >5 min. Start with 2 shards, increase until each shard runs in <3 min. For Playwright, shard by spec file weight with playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL.

Monorepo strategies

| Pattern | Trigger rule | Cache key | |---------|-------------|-----------| | Path-based | paths: ['packages/web/**'] (GHA) / rules:changes: (GitLab) | ${{ hashFiles('packages/web/package-lock.json') }} | | Label-based | if: contains(github.event.pull_request.labels.*.name, 'web') | Per-package cache restore | | Full rebuild | Monorepo tools (Nx, Turborepo) use their own remote caching | N/A — tool-managed |

For large monorepos, use Nx or Turborepo for task orchestration. Let the tool handle --since, affected projects, and remote cache. Reference the tool's CI recipe — don't hand-roll matrix logic when the tool already solves it.

Environment protection rules

# GitHub Actions — manual approval gate before production
deploy-prod:
  needs: [deploy-staging]
  environment:
    name: production
    # Enforces: required reviewers, wait timer, restricted branches, deployment history
  steps: [...]

• Required reviewers: Minimum 2 for production, 1 for staging
• Wait timer: Force 5-minute buffer before deploy (catch last-minute reverts)
• Deployment branches: Only main or release/* can trigger production
• Auto-inactive: Mark deployments inactive after 30 days, prevent stale environment clutter
• GitLab equivalent: environment: production with deploy: free or protected environments
• CircleCI equivalent: approval job type with restricted contexts

Secret rotation

Never store long-lived secrets in CI/CD. Prefer:

1. OIDC (preferred): No secrets at all — federated identity with short-lived tokens
2. Short-lived credentials: Clouds offer 1-hour max tokens via OIDC
3. Secret rotation schedule: Rotate all static secrets every 90 days, documented in playbook
4. CI/CD secret scanning: Enable GitHub secret scanning push protection or GitLab secret detection

For legacy systems requiring static secrets, store them only in the CI/CD platform's encrypted variables (not in repo), audit access via audit logs, and set expiration reminders via automation.

Step 4: Set up OIDC (no static secrets)

# GitHub Actions
permissions:
  id-token: write
  contents: read
steps:
  - uses: aws-actions/configure-aws-credentials@v4
    with:
      role-to-assume: arn:aws:iam::123456:role/github-actions
      aws-region: us-east-1

Step 5: Configure deployment strategy

See [references/deployment-strategies.md](references/deployment-strategies.md) for full details.

| Strategy | Rollback | Use when | |----------|----------|----------| | Rolling update | Manual (undo) | Stateless apps | | Blue-green | Instant (LB switch) | Zero-downtime required | | Canary | Gradual (traffic % adjust) | High-risk, gradual rollout |

Always implement healthcheck verification after deploy:

- name: Healthcheck
  run: |
    for i in {1..30}; do
      STATUS=$(curl -so /dev/null -w '%{http_code}' https://app.example.com/health)
      if [ "$STATUS" = "200" ]; then exit 0; fi
      sleep 2
    done
    echo "Healthcheck failed, rolling back..."
    kubectl rollout undo deployment/app
    exit 1

Step 6: Set up self-hosted runners (if needed)

See [references/self-hosted-runners.md](references/self-hosted-runners.md) for K8s runners, autoscaling, caching, and security configurations.

Decision: Use GitHub-hosted runners for standard builds. Self-hosted if you need:

• Custom hardware (GPU, large memory)
• Docker-in-Docker performance
• Private network access

Error Handling

| Error | Cause | Fix | |-------|-------|-----| | Resource not accessible by integration | Missing permissions: block in GHA workflow | Add permissions: contents: read, id-token: write at job or workflow level | | fatal: unable to access ... The requested URL returned error: 403 | Expired or missing git credentials | Use actions/checkout with token: ${{ secrets.GITHUB_TOKEN }} or deploy key | | Error response from daemon: manifest for ... not found | Docker image tag doesn't exist in registry | Verify tag was built and pushed; check registry path matches exactly | | Job failed: runner disconnected | Self-hosted runner crashed or network flaked | Add timeout-minutes per job, configure runner auto-restart, use spot instance rebalancing | | No space left on device | Build artifacts filling runner disk | Add cleanup step: rm -rf /tmp/* or use actions/upload-artifact with retention; reduce Docker image size | | deploy job requires a unique deployment_id | Concurrent deploys to same environment | Add concurrency: group: ${{ github.workflow }}-${{ github.ref }} to prevent race conditions | | Cache not found (restore) / Cache already exists (save) | Cache key mismatch or cache hit on locked key | Use restore-keys fallback for partial matches; include hashFiles('lockfile') in primary key | | OIDC token fetch failed | Missing id-token: write permission or IAM trust relationship misconfigured | Verify IAM trust policy allows token.actions.githubusercontent.com + correct repo/subject claim |

Production Checklist

• [ ] Lint + typecheck before tests
• [ ] Tests parallelized (matrix/sharding)
• [ ] Build artifacts cached or uploaded
• [ ] Docker build uses layer caching
• [ ] Plan/apply separation for IaC
• [ ] Production deployment requires manual approval
• [ ] Concurrency prevents simultaneous deploys
• [ ] OIDC auth (no static cloud keys)
• [ ] Rollback tested and documented
• [ ] Notifications on failure (Slack/Discord/email)
• [ ] Build time under 15 min

Anti-Patterns

| Anti-pattern | Fix | |-------------|-----| | terraform apply from laptop | CI/CD with plan/apply separation | | Deploying from latest tag | Use commit SHA or semver | | No cache | Cache deps, Docker layers, artifacts | | Sequential tests | Parallel matrix or sharding | | Static cloud creds in secrets | OIDC or short-lived tokens | | Secrets in pipeline YAML | CI/CD secret variables | | Monolithic job | Split: lint → test → build → deploy | | No healthcheck after deploy | Auto-rollback on failure | | Deploy on every main push | Gated approval + environment protection |

Pipeline Review Format (Required)

When reviewing CI/CD pipelines, use Before | After | Why format:

| Before | After | Why | |--------|-------|-----| | Static cloud credentials in secrets | OIDC with id-token: write permission | Static credentials never expire. OIDC issues short-lived tokens per workflow run. | | Sequential test jobs | Matrix build + test sharding (strategy: matrix: shard: [1/4, 2/4, 3/4, 4/4]) | Sequential tests bottleneck the pipeline. Sharding parallelizes across runners. | | terraform apply from laptop | CI plan → manual approval → CI apply | Laptop applies bypass audit trail and state locking. CI enforces process. | | No cache on dependencies | actions/cache for node_modules/, pip cache, go mod cache | Uncached dependencies add 2-5 minutes per run. Caching cuts this to seconds. |

Matrix Build Optimization

# GitHub Actions: parallel test shards
strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: npx vitest --shard=${{ matrix.shard }}/${{ strategy.job-total }}

# Parallel OS/version testing
strategy:
  matrix:
    os: [ubuntu-latest, windows-latest]
    node: [18, 20, 22]

Canary Deployment with Flagger

# Canary resource (Flagger CRD)
apiVersion: flagger.app/v1beta1
kind: Canary
spec:
  service: myapp
  analysis:
    interval: 30s; threshold: 5; maxWeight: 50; stepWeight: 10
    metrics:
    - name: request-success-rate; threshold: 99
    - name: request-duration; threshold: 500

Flow: deploy canary (10% traffic) → analyze metrics → if healthy, increase to 20%, 30%, 50% → promote to 100%.

Pipeline Approval Gates

# GitHub Environments with protection rules
deploy-prod:
  needs: deploy-staging
  environment: production
  steps: [ - run: ./deploy.sh ]

# Environment settings (GitHub UI):
# - Required reviewers: 2
# - Wait timer: 0 minutes
# - Deployment branches: main only

Sources

• GitHub Actions docs (docs.github.com/actions)
• GitLab CI docs (docs.gitlab.com/ee/ci)
• CircleCI docs (circleci.com/docs)
• DORA metrics (Google Cloud DevOps)
• Docker BuildKit cache type
• AWS IAM OIDC identity providers
• Flagger documentation (flagger.app)

Pre-Deploy Checklist

Before merging to production:

• [ ] All tests pass (unit, integration, E2E)
• [ ] Security scan passes (no HIGH/CRITICAL vulnerabilities)
• [ ] Lint and type checks pass with zero errors
• [ ] Build artifact produced and validated
• [ ] Deploy plan reviewed (IaC plan output, migration plan)
• [ ] Rollback plan documented and tested
• [ ] Environment protection rules active (require approvals for production)
• [ ] Concurrency control enabled (prevent simultaneous deploys)
• [ ] Healthcheck endpoint verified post-deploy
• [ ] Monitoring alerts configured for the new version
• [ ] Database migrations tested with a copy of production data

Checklist

• [ ] Skill loads without errors in the AI agent
• [ ] YAML frontmatter is valid (description, compatibility, audience)
• [ ] Workflow section provides clear step-by-step instructions
• [ ] Error handling section covers common failure modes
• [ ] All referenced files (references/, scripts/, assets/) exist
• [ ] Skill triggers correctly for intended use cases
• [ ] No broken links or missing resources

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: EliasOulkadi
• Source: EliasOulkadi/shokunin
• License: MIT
• Homepage: https://eliasoulkadi.github.io/shokunin/

Install and usage instructions live in the source repository linked above.