Cicd Pipelines

CI/CD Pipelines

Acknowledgement: Shared by Peter Bamuhigire, techguypeter.com, +256 784 464178.

Use When

• Use when designing or implementing a release pipeline that turns every commit into a release candidate progressing through automated stages with clear pass/fail gates.
• Use when picking GitHub Actions vs GitLab CI, federating CI to cloud or Vault via OIDC, or wiring blue/green / canary deploys into a workflow.
• Use when adding pipeline observability (DORA metrics, queue time, stage duration) or fixing anti-patterns like cache poisoning, mega-workflows, or rebuild-per-environment.

Do Not Use When

• Jenkins-on-Debian operations, plugin governance, or controller hardening — load references/cicd-jenkins-debian.md.
• Vault server architecture, PKI, exception governance, or compliance controls — load references/cicd-devsecops.md.
• High-level pipeline shape and stage-boundary design from a blank slate — start with references/cicd-pipeline-design.md, then return here for the engine-specific implementation.

Required Inputs

• Source-control engine (GitHub, GitLab CE, both) and runner story (self-managed Debian/Ubuntu, hosted, mix).
• Target deploy surface (Debian VPS, Kubernetes, ECS, mobile stores) and the chosen deployment strategy.
• Secret-management posture (Vault, AWS Secrets Manager, env secrets) and whether OIDC federation is already wired.

Workflow

• Read this SKILL.md first; load only the referenced files needed for the engine and deploy surface in front of you.
• Apply the stage-gate model in §1 before writing YAML — pipelines without explicit gates rot into mega-workflows.
• Produce the deliverable (workflow YAML, trust policy, deploy record schema) with assumptions and rollback plan made explicit.

Quality Standards

• Build once, deploy many: a single artefact (digest-pinned image, signed AAB, .ipa) is promoted through every environment.
• Least-privilege permissions: per workflow and per job; no static cloud keys when OIDC is available.
• Every production deploy emits a deployment record (git SHA, image digest, environment, actor, timestamp, run URL) to an append-only sink.

Anti-Patterns

• Echoing constructed strings containing secrets; pinning third-party actions to mutable tags; one mega-workflow per repo.
• Cache keys without a lockfile hash; rebuilding per environment; deploying without a concurrency: group on production.
• Long-lived static AWS_ACCESS_KEY_ID in repo secrets when OIDC federation is available.

Outputs

• Reusable workflow with versioned inputs/outputs; environment-specific deploy workflows; rollback workflow.
• OIDC trust policy, least-privilege IAM/Vault role binding, deployment-record schema and sink.
• Pipeline-observability dashboard spec (DORA quad + queue time + stage p95) wired into SigNoz/Prometheus.

Evidence Produced

| Category | Artifact | Format | Example | |----------|----------|--------|---------| | Release evidence | Pipeline configuration record | Markdown doc covering build, test, deploy stages plus secret references | docs/ci/pipeline-config.md | | Release evidence | Latest release run evidence | CI URL plus archived log of the most recent successful release | docs/ci/release-run-2026-04-16.md | | Release evidence | Deployment record sink schema | JSON schema + sample row | docs/ci/deployment-record.md |

References

• references/oidc-federation.md — OIDC → AWS, GCP, and Vault with bound-claim trust policies.
• references/pipeline-observability.md — DORA metrics, queue time, scraper sketch, dashboards.
• references/reference-architectures.md — three end-to-end pipelines (PHP/MySQL SaaS, Node.js/TS service, container library).
• references/anti-patterns.md — broken vs fixed examples.
• references/github-vs-gitlab.md — engine comparison and migration notes.
• references/github-actions-workflows.md, references/mobile-pipelines.md, references/ios-fastlane-pipeline.md, references/android-pipeline.md — domain templates.

Load Order

1. Load world-class-engineering and git-collaboration-workflow for the baseline.
2. Load references/cicd-pipeline-design.md for the high-level pipeline shape.
3. Load this skill for engine-specific implementation (GitHub Actions primary, GitLab CI secondary).
4. Pair with references/cicd-devsecops.md for secrets policy, references/cicd-jenkins-debian.md when the engine is Jenkins, deployment-release-engineering for rollout, observability-monitoring for post-deploy verification, cloud-architecture for traffic-shifting plumbing.

§1 What a CI/CD pipeline is and is not

The deployment pipeline is the canonical model: every commit produces a release candidate that progresses through automated stages, and any failed stage stops the candidate. Continuous Delivery (Humble & Farley, Addison-Wesley, 2010, ISBN 978-0-321-60191-9) frames the pipeline as the mechanism by which you build quality in rather than testing it in afterwards. The DevOps Handbook (2nd ed., Kim, Humble, Debois, Willis, IT Revolution, 2021, ISBN 978-1-950508-40-2) reinforces this with the First Way (flow from development to operations) and Second Way (fast feedback via short-lived automated pipelines).

Stage-gate convention used throughout this skill, derived from the canonical 5-stage model:

| Stage | Purpose | Target duration | |-------|---------|-----------------| | Commit | Compile, unit tests, lint, type-check | /:`) tied to repo permissions; ECR via OIDC for AWS-hosted services.

Multi-platform images only when the runtime differs (ARM Graviton, Apple Silicon dev) — it doubles build time. Sign with cosign keyless and emit provenance: true and sbom: true from docker/build-push-action so consumers can verify before deploy.

§5 Parallelism, fan-out / fan-in, manual approvals, concurrency

Four orchestration primitives:

• Fan-out: matrix strategy or multiple top-level jobs running in parallel.
• Fan-in: dependent jobs declared via needs: [job-a, job-b].
• Manual approval: environment with required reviewers — the deploy job pauses until approved.
• Concurrency: concurrency: { group: deploy-prod, cancel-in-progress: false } ensures only one prod deploy runs at a time. For PR previews use cancel-in-progress: true so superseded commits abandon in-flight runs.

Test parallelisation pattern (Playwright shards, then merge reports):

e2e:
  strategy:
    fail-fast: false
    matrix: { shard: [1/4, 2/4, 3/4, 4/4] }
  runs-on: ubuntu-24.04
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with: { node-version: '20', cache: 'npm' }
    - run: npm ci && npx playwright install --with-deps
    - run: npx playwright test --shard=${{ matrix.shard }} --reporter=blob
    - uses: actions/upload-artifact@v4
      with: { name: blob-report-${{ strategy.job-index }}, path: blob-report, retention-days: 1 }

merge-reports:
  if: always()
  needs: [e2e]
  runs-on: ubuntu-24.04
  steps:
    - uses: actions/download-artifact@v4
      with: { path: reports, pattern: blob-report-*, merge-multiple: true }
    - run: npx playwright merge-reports --reporter=html ./reports

Jest shards via jest --shard=1/4 across a matrix or --maxWorkers=50% on one large runner. Report merging is mandatory — reviewers need a single HTML report.

§6 Deployment strategies invoked from a pipeline

| Strategy | When to use | Key implementation knob | |----------|-------------|--------------------------| | Rolling | Stateless service, minor risk | Replicas updated N at a time | | Blue / green | Need atomic cutover, easy rollback | Two parallel environments + traffic switch | | Canary | High-risk change, gradual rollout | % traffic shift over time, automated abort on SLO breach |

Detailed traffic-shifting plumbing (Kubernetes Service swap, Argo Rollouts, AWS CodeDeploy, Nginx symlink swap on Debian VPS) lives in cloud-architecture and deployment-release-engineering. The pipeline's job is to invoke the strategy, watch the health gate, and trigger rollback on SLO breach.

Promotion is workflow_dispatch with an image_digest input — never a rebuild:

name: promote
on:
  workflow_dispatch:
    inputs:
      image_digest: { description: 'sha256:... to promote', required: true }
      target:
        description: 'Target environment'
        required: true
        type: choice
        options: [staging, production]
jobs:
  deploy:
    runs-on: ubuntu-24.04
    environment: { name: ${{ inputs.target }} }
    steps:
      - run: deploy.sh "${{ inputs.image_digest }}" "${{ inputs.target }}"

Rollback is the same workflow with the previous digest. Three rollbacks in a week on the same service freezes deploys until a post-mortem is filed. A nightly scheduled staging re-deploy detects drift between main HEAD and live.

§7 Pipeline observability

The four DORA metrics — deployment frequency, lead time for changes, change failure rate, mean time to restore — are the canonical indicators of pipeline health (The DevOps Handbook). Augment with:

• Queue time (run.created_at → run.started_at) — p95 80%.
• Workflow failure rate on main — < 10%.

Workflow timings come from GET /repos/{owner}/{repo}/actions/runs and /runs/{id}/jobs. A small scraper polls and ships line-protocol to the platform observability backend (SigNoz OTLP or Prometheus pushgateway). Schema and scraper pseudocode live in references/pipeline-observability.md.

Every successful production deploy emits one row to an append-only sink (S3 + Athena, BigQuery, or a Postgres deployments table). DORA metrics are computed from this table — not reconstructed from logs. Pair with observability-monitoring for SLO and alert design.

§8 GitHub Actions vs GitLab CI

For repos in the self-managed Debian/Ubuntu stack (Jenkins + GitLab CE + Nexus 3 + Vault), the canonical engine is GitLab CI on self-managed runners. GitHub Actions is used for repos already on GitHub.com — primarily mobile (iOS hosted macos-14) and OSS releases.

Quick-decision summary:

| Concern | GitHub Actions | GitLab CI | |---------|----------------|-----------| | Configuration | .github/workflows/*.yml (multiple) | .gitlab-ci.yml (single, with include:) | | Reuse | Reusable workflows + composite actions | include: + components | | Job graph | DAG via needs: | Stages-then-jobs (DAG via needs:) | | Secrets | Repo / env / org, OIDC, secrets: inherit | CI/CD variables (masked, protected, env-scoped), OIDC ID token | | Concurrency | concurrency: keyword | resource_group: |

Full comparison and migration notes (claim shapes for OIDC sub, matrix → parallel:matrix:, marketplace → components) live in references/github-vs-gitlab.md.

§9 Anti-patterns and pitfalls

Headline list — broken-vs-fixed examples in references/anti-patterns.md:

• Echoing secrets via constructed strings (URL with embedded token) bypasses GitHub's redactor.
• Cache keys without a lockfile hash poison subsequent builds with stale dependencies.
• The mega-workflow — one 800-line ci.yml — is unmaintainable; split per concern with reusable workflows.
• Missing permissions: block defaults to write-all on older repos; pin per workflow and per job.
• No concurrency: on production allows two simultaneous deploys to race on kubectl apply.
• Rebuilding per environment lets staging and production drift silently; build once on merge to main, promote the digest.
• Pinning third-party actions to mutable tags (@v1) lets a publisher repoint to a malicious commit; pin to a full SHA on security paths.
• Logging the entire environment can leak vended Vault tokens that the redactor never registered.

§10 Reference architectures

Three worked examples covered end-to-end in references/reference-architectures.md:

1. PHP/MySQL SaaS — composer install, PHPUnit, PHPStan, Trivy on container, OIDC → Vault → DB creds, blue/green via Nginx symlink swap on Debian VPS.
2. Node.js/TypeScript service — pnpm, vitest, tsc --noEmit, esbuild bundle, container build, push to GHCR, canary via Argo Rollouts.
3. Container image library — multi-arch BuildKit, SBOM via syft, keyless cosign signing — produces signed images consumed by both pipelines above.

The mobile pipelines (iOS Fastlane match/gym/pilot, Android gradlew assembleRelease + Fastlane supply) live in references/mobile-pipelines.md, references/ios-fastlane-pipeline.md, and references/android-pipeline.md.

Branch strategy and semantic versioning

• GitHub Flow (default for SaaS web/API): long-lived main, short feature branches merged via PR, deploys from main.
• Trunk-based (10+ engineers): very short branches (< 1 day), feature flags hide unfinished work, main always releasable.
• Release branches (mobile, firmware, versioned SDKs): main for ongoing work, release/v1.2 cut and tagged for ship.

Pick one per repo and document in CONTRIBUTING.md. Conventional commits drive semantic-release for tag, CHANGELOG, GitHub Release, npm publish on merge to main (feat: → minor, fix: → patch, feat!: or BREAKING CHANGE: → major). Config lives in .releaserc.json.

Verification

• Every workflow PR is reviewed by someone other than the author; actionlint is a required status check on any .github/workflows/*.yml change.
• Track DORA metrics off the deployment-record sink; alert when change-failure rate or queue time p95 breach the SLO.
• Codex and Codex users on the same repo work from the same files — nothing platform-specific in workflow YAML.

Companion Skills

• references/cicd-pipeline-design.md — high-level pipeline shape, stage boundaries, gate design, FinOps.
• references/cicd-devsecops.md — secrets policy, Vault server architecture, scan thresholds, exception governance.
• references/cicd-jenkins-debian.md — when the CI server is Jenkins, not GitHub Actions or GitLab CI.
• deployment-release-engineering — rollout, canary, blue/green, post-deploy verification.
• kubernetes-platform — ArgoCD GitOps and progressive delivery for multi-tenant SaaS on Kubernetes.
• observability-monitoring — deploy markers, SLO burn, MTTR, dashboards.
• cloud-architecture — traffic-shifting plumbing for the deployment strategies invoked here.
• ios-development, android-development — app build config upstream of the mobile pipelines.

Sources

• Continuous Delivery, Humble & Farley, Addison-Wesley, 2010, ISBN 978-0-321-60191-9.
• The DevOps Handbook, 2nd ed., Kim, Humble, Debois, Willis, IT Revolution, 2021, ISBN 978-1-950508-40-2.
• GitHub Actions docs: docs.github.com/actions; Reusing workflows: docs.github.com/en/actions/using-workflows/reusing-workflows.
• Automatic token authentication: docs.github.com/en/actions/security-guides/automatic-token-authentication.
• GitLab CI docs: docs.gitlab.com/ci; Docker Build cache: docs.docker.com/build/cache.
• Trivy: aquasecurity.github.io/trivy; semantic-release: semantic-release.gitbook.io; Fastlane: docs.fastlane.tools.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: peterbamuhigire
• Source: peterbamuhigire/skills-web-dev
• License: MIT
• Homepage: https://techguypeter.com

Install and usage instructions live in the source repository linked above.