Load Test

Quick Reference

| When you need... | Jump to | |------------------------------------------|---------------------------------------------| | Write a load test from scratch | §2 Gates -> §5 Checklist -> §6 Scenarios | | Review existing test script | §2 Gates -> §5.2 Script Quality | | Analyze test results | §2 Gates -> §5.3 Analysis -> load ref | | Choose between k6/vegeta/wrk | §6.1 Tool Selection | | Define SLOs for a service | §5.1 SLO Definition | | Debug why a test shows bad numbers | §7 Anti-Examples -> load analysis ref | | Capacity planning | §6.2 Scenario Selection -> breakpoint/soak | | Load generator OOM / high RSS | §7 AE-7 + k6-patterns §11 Memory Hygiene |

---

1 Scope

In scope: HTTP/gRPC service load testing, SLO definition, scenario design, script generation (k6 primary, vegeta, wrk), result analysis, bottleneck identification, capacity planning recommendations.

Out of scope: unit/micro-benchmarks (use go-benchmark), database-only benchmarks, browser/UI performance (Lighthouse), chaos engineering fault injection, infrastructure provisioning (Terraform/Pulumi).

---

2 Mandatory Gates

Gates are serial hard blockers. Failure at any gate stops all subsequent work.

Gate 1: Context Collection

Gather before proceeding. STOP if target service is unknown.

| Item | Example | Required | |-------------------|--------------------------------------------|----------| | Service endpoint | https://api.example.com/v1/orders | Yes | | Protocol | HTTP/1.1, HTTP/2, gRPC | Yes | | Current baseline | p50=12ms, p99=85ms, 2000 RPS | If known | | Deployment | k8s 3 replicas, 2 CPU / 4Gi each | If known | | Auth mechanism | Bearer token, API key, mTLS | If any | | Data dependencies | DB, Redis, external API | If known |

Gate 2: SLO-First

SLOs MUST exist before writing test scripts. Without SLOs, test results are meaningless numbers. If the user has no SLOs, help them define SLOs first.

STOP and define SLOs if none provided. Minimum SLO set:

• Latency: p50 and p99 targets (e.g., p50 5 endpoints
• Coverage: smoke + load + stress + soak, resource correlation, bottleneck report
• Force Deep if: soak test requested, breakpoint analysis, multi-service chain

---

4 Degradation Modes

When prerequisites are incomplete, produce explicitly-marked partial output.

| Available Data | Mode | Can Deliver | Cannot Claim | |------------------------------------|----------|--------------------------------------------------|---------------------------------| | Service spec + SLOs | Full | Script + scenario + analysis plan | Actual performance numbers | | Script only, no results | Script | Script review + improvement suggestions | SLO pass/fail verdict | | Results only, no SLOs | Partial | Statistical summary + anomaly flags | Pass/fail, capacity conclusions | | Results + SLOs | Analysis | Full SLO verdict + bottleneck analysis | Script improvements | | No service info, vague request | Planning | Generic scenario template + SLO questionnaire | Anything specific |

Mark degraded outputs: # DEGRADED: [reason] — [what's missing]

Never fabricate performance numbers. Never claim SLO compliance without data.

---

5 Load Test Checklist

5.1 SLO Definition

1. Latency targets are percentile-based — p50 and p99 minimum; p95 recommended.

Raw averages hide tail latency. A service with avg=20ms but p99=2s is broken.

1. Throughput target matches production traffic — use access logs or APM to

derive realistic RPS. Add 2-3x headroom for growth.

1. Error budget is explicit — "= 1k TPS, compute peak RSS via `maxVUs × 3 MB + Σ(Trend) × 80 B × rate ×

duration + body + tag overhead BEFORE running. If peak > 80% of generator RAM, tighten maxVUs (Little's Law in k6-patterns.md §2) or shorten duration. The 4 most common OOM causes (in order): oversized maxVUs when service saturates, --out csv= buffering, duplicated/diagnostic Trend metrics retaining all samples, dynamic-URL tag bucket explosion. All four are covered in k6-patterns.md §11`.

1. Environment matches production — or document differences explicitly.

A test on a single-replica staging env says nothing about 3-replica prod.

---

6 Scenario & Tool Selection

6.1 Tool Selection

| Tool | Best For | Language | Distributed | |-----------|---------------------------------------------|----------|-------------| | k6 | Scenario modeling, JS scripting, CI/CD | JS/TS | k6 Cloud | | vegeta | Constant-rate attacks, Go pipelines | Go CLI | Manual | | wrk | Raw throughput measurement, simple scripts | Lua | No |

Default to k6 unless: (a) user explicitly requests another tool, (b) constant-rate is the only requirement (vegeta), or (c) maximum raw throughput measurement (wrk).

6.2 Scenario Selection

| Goal | Scenario | Pattern | |-----------------------------|-----------------|-----------------------------------------------------| | "Does it work under load?" | Smoke | 1-5 VUs, 1 min — sanity check | | "Can it handle target RPS?" | Load | Ramp to target VUs, 3-5 min steady state | | "Where does it break?" | Stress | Ramp beyond target, find degradation point | | "What's the ceiling?" | Breakpoint | Step-increase VUs until failure — find absolute limit | | "Memory leaks? Pool drain?" | Soak | Moderate load, 30-60+ minutes — detect drift | | "Can it handle a flash sale?"| Spike | Sudden 10x burst, hold 1 min, drop — test recovery |

Select scenario based on the testing goal, not just "run some load". Multiple scenarios compose for Deep depth (smoke -> load -> stress -> breakpoint).

---

7 Anti-Examples

AE-1: Testing without warmup

# WRONG: measurement starts immediately
export default function() {
  http.get('http://api/endpoint');
}
// First 30s includes JVM startup, connection pool creation, cache cold starts
// Result: p99 inflated by 5-10x, meaningless numbers

# RIGHT: explicit warmup stage excluded from results
export const options = {
  scenarios: {
    warmup: { executor: 'constant-vus', vus: 10, duration: '30s',
              gracefulStop: '0s', tags: { phase: 'warmup' } },
    test:   { executor: 'ramping-vus', startTime: '30s',
              stages: [{ duration: '1m', target: 100 }] },
  },
  thresholds: {
    'http_req_duration{phase:test}': ['p(99)5000'],                        // throughput SLO
  },
};
// Output: p99=312ms FAIL (SLO:  JSON.parse(open('./users.json')));
export default function() {
  const user = users[Math.floor(Math.random() * users.length)];
  http.get(`http://api/users/${user.id}`);
}

AE-5: 30-second test declared "comprehensive"

# WRONG: "load test passed" after 30s
k6 run --vus 50 --duration 30s test.js
// Misses: GC major collections (every 2-3 min), connection pool exhaustion
// (builds up over minutes), memory leaks (invisible under 5 min),
// DB connection limit (pool fills gradually). This is a smoke test at best.

# RIGHT: duration matches what you're testing
// Smoke: 1 min (sanity only)    Soak: 30-60 min (leak detection)
// Load:  3-5 min steady state   Stress: until degradation observed

AE-6: Reporting averages as performance verdict

# WRONG: "average latency is 45ms, we're good"
// Average hides: p99=2.1s (1% of users wait 2+ seconds)
// Average hides: bimodal distribution (cache hit=5ms, miss=500ms)

# RIGHT: percentile-based analysis
// p50=12ms p95=45ms p99=180ms p99.9=890ms max=2.1s
// Verdict: p99=180ms = 1 min smoke, >= 3 min load/stress

### Standard (>= 4 of 5 must pass)

4. **Gradual ramp-up** — not instant full load
5. **Error rate monitored** — 4xx/5xx/timeout tracked separately
6. **Percentile latency reported** — p50/p95/p99 minimum, not just average
7. **Load generator not co-located** — separate from target
8. **Test data parameterized** — not single-value cache-hit bias

### Hygiene (>= 3 of 5 must pass)

9. **Environment documented** — infra specs, replica count, resource limits
10. **Baseline comparison** — delta from previous run or production metrics
11. **Resource metrics correlated** — CPU/mem/connections alongside latency
12. **Results archived** — raw data + summary stored for regression tracking
13. **Load-generator memory budgeted** — k6 scripts size `maxVUs` via Little's
    Law (`rate × healthy_p95 × 2`), opt-in diagnostic Trends, no `--out csv`,
    `discardResponseBodies` + per-request `responseType:'text'` override in
    setup. See `k6-patterns.md §2 + §11`. The load generator itself OOMing
    invalidates the run.

**Verdict**: Critical 3/3 AND Standard >= 4/5 AND Hygiene >= 3/5 = **PASS**

---

## 9 Output Contract

Every response MUST include these sections. Volume rules: FAIL items fully
detailed; WARN items up to 10; PASS items summary only.

### 9.1 Context Summary
Target service, protocol, deployment, SLOs — table format.

### 9.2 Mode & Depth
`Write | Review | Analyze` + `Lite | Standard | Deep` with rationale.

### 9.3 SLO Definition
Latency (p50/p99), throughput (RPS), error rate, availability.
If user-provided SLOs are incomplete, state what was assumed.

### 9.4 Scenario Design
Selected scenario type, rationale, VU/RPS targets, duration, stages.

### 9.5 Test Script or Script Review
Write mode: complete executable script with run command.
Review mode: findings with severity and fix suggestions.
Analyze mode: omit or reference original script.

### 9.6 Results Analysis (Analyze mode)
Percentile table (p50/p95/p99/p99.9/max), throughput, error breakdown.
SLO pass/fail for each metric. Trend analysis if multiple runs.

### 9.7 Bottleneck Assessment
Identified bottlenecks ranked by impact. For each: evidence, affected SLO,
recommended fix, expected improvement. If no bottleneck found, state why.

### 9.8 Recommendations
Prioritized next steps: fix bottleneck, run longer soak, add monitoring,
adjust SLO, scale infrastructure. Each with effort estimate (quick/medium/large).

### 9.9 Uncovered Risks
What this test did NOT cover. Mandatory — never empty. Examples: "soak test
not run — memory leak risk unvalidated", "only read endpoints tested — write
path capacity unknown", "single-region test — cross-region latency not measured".

**Scorecard appended**: `X/13 — Critical Y/3, Standard Z/5, Hygiene W/5 — PASS/FAIL`
+ data basis (script only | results available | full profiling).

---

## 10 Reference Loading Guide

| Condition                                   | Load                              |
|---------------------------------------------|-----------------------------------|
| Writing k6 script (Standard+)              | `references/k6-patterns.md`       |
| Writing vegeta attack (Standard+)          | `references/vegeta-patterns.md`   |
| Analyzing results, finding bottlenecks     | `references/analysis-guide.md`    |
| **k6 sizing maxVUs / OOM / RSS budgeting** | **`references/k6-patterns.md §2 + §11`** |
| Deep depth or multi-scenario               | All three references              |

Each reference has a table of contents. Load the relevant sections, not
the entire file, when only a specific pattern is needed.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [johnqtcg](https://github.com/johnqtcg)
- **Source:** [johnqtcg/awesome-skills](https://github.com/johnqtcg/awesome-skills)
- **License:** MIT
- **Homepage:** https://johnqtcg.github.io/awesome-skills/

Install and usage instructions live in the source repository linked above.