E2e Scenario Design From Prd

E2E Scenario Design From PRD

Turn a PRD into a shippable E2E test portfolio: 10-15 scenarios covering happy path → workflow correctness → compliance → security → performance → edge cases, with each scenario naming the right tool for its verification style.

The output is NOT a generic "test checklist". It's a scenario portfolio — every item is a runnable test spec worth writing, and anything that doesn't survive the priority cut gets explicitly dropped with a reason.

---

Input gathering — go wide before deep

The user should provide (or point you at) at least one of:

• A PRD / spec / design doc (markdown, Notion, Confluence, or wiki / pasted text).
• A list of user stories / personas.
• The current codebase (as a fallback if PRD is thin — you can reverse-engineer

the feature list from routers/views).

If nothing concrete is given, stop and ask — don't hallucinate requirements.

Before drafting any scenario, cast a wider net than "just the PRD":

1. README / landing copy — 3-5 promises customers buy. P0 seeds.
2. Top-level docs sidebar — real feature surface area. README alone

yields marketing-shaped scenarios and misses ~40% of real paths (autosave, search, bulk, folders, expiration).

1. Pricing / tier page — gating reveals what's promised per tier.
2. Settings / admin UI — every toggle is a promise; untested toggles

are live bombs.

1. API reference + webhook catalog — developer-contract surface; test

event delivery, not just CRUD.

Rule: 0 AND errorcount == 0 AND residualcount == 0`. Recurs until explicitly named, across teams and stacks.

How scenarios apply this

• For any ✓ / green tag, the scenario asks the backend API what specific

count/status drove that signal. If none exists, the badge is decorative — don't assert on it.

• Silent fallback is the enemy. If a component fails but emits pseudo-success

(LLM timeout → rule-score fallback), write an assertion that makes it LOUD (e.g. `llmerrorcount / llmcallcount 30 | Feature-grid primary; Promise-based becomes a subset |

The rest of this skill defaults to Promise-based. For Feature-grid or Hybrid, P0 still caps at 6-8; P1/P2 fan out per feature area.

---

Priority taxonomy (use these exact definitions)

P0 — Must-have. If any P0 fails, feature does NOT ship.

Reserved for invariants the product promises. Typical categories:

| Category | Example | |---|---| | Happy path | Core flow end-to-end, fresh user, golden inputs | | Data integrity | What comes out equals what should come out | | Security | Authorization, data isolation, authn | | Compliance hard red lines | PII redaction, audit logging, legal obligations | | Core algorithm correctness | If the product IS the algorithm (dedup / search / ranking) |

P0 count: 4-6 scenarios. >6 means over-classifying; demote some to P1.

P1 — Should-have. Block release of the FEATURE they test, not the whole product.

| Category | Example | |---|---| | Workflow correctness | Rollback / multi-step flows / versioning | | Config / admin flows | Hot-reload, rules engine, role management | | Audit / reporting | Whatever leadership looks at weekly | | Cross-format / cross-type coverage | Input variations |

P1 count: 4-6 scenarios.

P2 — Nice-to-have. File as backlog if they fail, don't block release.

| Category | Example | |---|---| | Performance / stress | Large files, high concurrency, long-running | | Edge cases | Boundary inputs, malformed data, special chars | | UX quality | Error message wording, progress feedback richness | | Observability-only | Log format, metric shape |

P2 count: 2-4 scenarios.

Total: 10-15 for Promise-based; 30-80+ for Feature-grid (see philosophy section). Within Promise-based: 15 means padding or shipping too big a feature at once.

---

Scenario template — copy this for every item

### Scenario NN: 

- **Priority**: P0 / P1 / P2
- **Category**: (from the tables above)
- **Goal** (1 sentence, user/business outcome — NOT implementation):
  > "新员工第一天上传 100 条数据,能看到 5 分钟内处理完成并可下载。"
- **Preconditions**: (test account, feature flag, fixture, env state)
- **Steps**: numbered, each actionable.

- **Assertions** — 3 columns, each row is ONE assertion:

  | Layer | What it checks | Example |
  |---|---|---|
  | Backend primary | Truth from API/DB, independent of UI | `task.passed_records == 100` |
  | UI display | What user sees, PAIRED with a backend row | `"通过: 100" 在 v1 行渲染` |
  | Counter-invariant | What should NOT happen | 0 console errors, no 500 responses |

  **If the backend row is missing for any UI row → you're testing display,
  not truth.** Go back and find the backend field that drives the UI.

- **Tool**: primary / secondary, with one-line reasoning.
- **Known risks / flakiness**: what could make this test flake.
- **Out of scope**: easily-confused adjacent scenarios you chose NOT to merge.

Every scenario MUST include goal + ≥2 assertion rows (at least 1 Backend) + tool choice with reasoning. If you can't name a primary Backend assertion, you don't understand the scenario yet — re-read the PRD.

---

Tool selection rules

Use Playwright MCP (preferred default) when:

• Assertions are deterministic (counts, IDs, API responses, DOM selectors).
• You need network interception (response body, status codes, headers).
• Test must be CI-runnable without a human in the loop.
• File upload, form filling, navigation, cross-checking UI vs API.
• Performance timing (stage durations, polling intervals).
• Scenario will be run >5 times over its lifetime (regression suite).

Use Claude in Chrome when:

• Judgment requires VISUAL evaluation no regex can reliably do:
• PII leakage scan across rendered pages
• Error message quality scoring ("does this sound like 人话?")
• UX friendliness rating (is the progress bar misleading?)
• Layout / overflow / truncation issues
• Scenario is exploratory (one-shot investigation, not CI regression).
• Looking at the page and describing it beats writing selectors.
• The "assertion" is a qualitative 1-10 score, not pass/fail.

Hybrid — both, when:

• Playwright drives deterministic parts (upload, API polling, counts).
• Chrome MCP takes screenshots + evaluates qualitative aspects at key moments.
• Reports combine structured JSON (Playwright) + visual observations (Chrome).

When NOT to use E2E at all

Some testing needs should NOT be E2E. Write the rejection into your portfolio doc so reviewers know you considered and rejected:

• Pure logic in isolation (regex patterns, score computation, formula) →

unit test. Fast, doesn't need browser.

• Load / throughput beyond single user → load test tool (k6 / locust).

E2E's single-session model makes this expensive and noisy.

• Contract between services → integration / contract test

(pact, api-schema). E2E masks which service is wrong when it fails.

• Auth / RBAC matrix of N roles × M endpoints → API-level matrix test,

not per-user E2E. E2E per cell is O(NM) runs; API test is O(1) per cell.

---

Design patterns (harvest from real projects)

• Unique staging: copy fixtures to /tmp/_${timestamp}. before

upload. Avoids dedup / version-chain state collision between runs.

• Cross-layer verification ("UI-lie detection"): every green badge the UI

shows → also fetch the API and assert the underlying counter.

• Counter invariants: assert total == passed + rejected + dropped.

Pipelines drop records silently otherwise.

• Finally-block cleanup: if a scenario mutates global state (rule configs,

user role, DB flags), restore in finally.

• Observation loops: for any stage >30s, poll every 10-30s and record

intermediate state, not just final. UX gaps (stuck progress, counters that jump from 0 to N) surface here.

• Structured result JSON: each spec writes results/.json with

{steps: [{name, passed, data}], screenshots, failures} for CI + reports.

Pitfalls that will bite

• LLM scoring is flaky — assert ranges / categories / fallback behavior,

never specific scores.

• Dedup collapses fixtures — bad-data fixtures with structurally

identical records get collapsed before your "≥N rejected" fires. Keep each record distinct.

• Same-filename → new version in some pipelines. Timestamp filenames or

scenarios cross-contaminate.

• Assertions voice: business truths ("通过率 > 80%") in Assertions,

implementation details in Steps.

• Single-actor blindspot — products with ≥2 roles (sender/signer,

admin/member, creator/viewer) can't be tested from one seat. A happy path run only as "creator" misses everything after handoff. Hard requirements for multi-actor products: (1) each actor ≥1 happy-path scenario from their viewpoint; (2) ≥1 cross-actor permission scenario (B cannot do what's reserved for A); (3) ≥1 asymmetric workflow in a SINGLE scenario — not two separate specs: (a) actor A performs an action, (b) actor B observes/responds within the same flow, (c) assert consistent state from both perspectives at the end. Anti-pattern: "Scenario 5: host creates event" + "Scenario 6: guest books event" — these are two separate happy paths, not asymmetric. Correct: one scenario where host sets booking-limit=1, guest A books, guest B attempts and sees "unavailable", host dashboard shows bookingCount=1. Catches cross-actor state-consistency bugs that single-actor specs never surface.

---

Workflow when invoking this skill

1. Read the full PRD first. No scenario writing until cover-to-cover.
2. Write feature inventory — 15-25 bulleted items, each a distinct

user-facing capability.

1. Identify 3-4 product-level invariants — what would make a customer

feel cheated if broken? Those are P0 seeds.

1. Draft 12-15 scenarios using the template. Over-produce, then cut.
2. Cut to 10-15 — weak items move to "Dropped" with a one-line reason.
3. Assign tools — default Playwright; upgrade to Chrome MCP only where

visual judgment is irreplaceable.

1. Sanity-check tier sizes — P0:4-6 / P1:4-6 / P2:2-4.
2. P0 self-review table — for each P0, if ≥2 YES, keep P0; else demote:

| Question | Keep P0 if answer is | |---|---| | 客户会投诉吗? | YES | | 监管会罚钱 / 合规红线吗? | YES | | 产品本身就是这个能力吗? (e.g. dedup-engine for a dedup product) | YES | | Ops 能 5 分钟人工验证吗? | NO (if yes, it's not a regression test) | | 只影响单个 feature,不阻塞整体发布吗? | NO (single-feature blocker → P1) |

---

Execution ordering

When the portfolio runs in CI:

• P0 happy-path first. If it fails, skip the rest — downstream is doomed.
• P0 security/compliance next. They don't depend on workflow state;

fail-fast saves minutes.

• P1 runs only if all P0 passed.
• P2 runs nightly / pre-release, not per-commit (too expensive per PR).
• Chrome-MCP scenarios do NOT enter regression. Exploratory evaluation.

Label them in the portfolio so CI skips them.

---

Anti-patterns (refuse to produce these)

• Scenarios whose goal is "verify that function X works" — unit territory.
• Scenarios with >15 steps — split them.
• Every assertion is "UI shows X text" — these drift when copy changes.

Pair every UI assertion with a backend-state equivalent.

• "Smoke test" as a separate P0 — the happy path IS your smoke test.
• Performance scenarios without a specific threshold — "pipeline should

complete quickly" is not a scenario; "100 records complete in 0 means signal-fidelity not applied — fix before handoff.

1. Buildable in ≤2h/scenario: if a scenario needs a week of infra, split

infra as separate deliverable and demote scenario to P2.

1. PM-readable goals: read each goal to the PRD author. "Why test that?"

means implementation-speak — rewrite as a user outcome.

If the output fails any of these, regenerate. Do NOT hand off a portfolio that fails self-test.

---

Sanity checklist before handoff

• [ ] Feature inventory present, ≤25 items.
• [ ] 10-15 total scenarios.
• [ ] P0: 4-6 / P1: 4-6 / P2: 2-4.
• [ ] Every scenario has: Priority, Category, Goal, Preconditions, Steps,

Assertions (≥2 rows, ≥1 Backend), Tool, Known risks, Out of scope.

• [ ] Every UI assertion row has a paired Backend row.
• [ ] 3-5 dropped candidates listed with reasons.
• [ ] Tool matrix at the end.
• [ ] ≥1 scenario uses Chrome MCP primary (visual judgment) OR explained why none.
• [ ] No scenario exceeds 15 steps.
• [ ] No scenario goal reads like "test that X function works".
• [ ] Coverage completeness audit — every feature-inventory item is

(a) in a scenario, (b) dropped-with-reason, or (c) deferred to unit/contract test. Target: ~80% covered / ~15% P2-or-deferred / ~5% dropped. >20% dropped → go back; portfolio is too thin. See Within-cluster check + Partial-heavy classification below.

• [ ] Within-cluster coverage check — for any cluster with >3 raw

capabilities (common: auth flows, integration providers, admin actions, payment methods), audit EACH capability individually. A single "auth cluster covered" check is insufficient if the cluster contains login, signup, logout, change-password, forgot-password, delete-account, 2FA, OAuth, SSO, email verification — these are 10 distinct capabilities that need 10 audit outcomes. Cluster-level audit is allowed ONLY when capabilities are structurally identical (see Partial-heavy classification below).

• [ ] Self-test four probes pass.

Partial coverage — legitimate vs true miss

When a cluster is marked "partially covered" by the audit above, classify which kind — the two have opposite remediation paths:

Legitimate "many-of-same-kind partial" (OK):

• Cluster contains N structurally-identical items (15 calendar integrations,

20 webhook event types, 10 language localizations).

• One item covered thoroughly + smoke-test the rest = valid coverage.
• Example: Google / Outlook / Apple calendar sync share the

OAuth → token → sync pattern; test Google deeply, smoke the others.

Distinct partial (TRUE MISS — fix before handoff):

• Cluster contains N structurally-different items; some covered, others omitted.
• Example: auth cluster with login (covered), change-password (NOT),

forgot-password (NOT) — different entry points, data shapes, and failure modes; each needs its own scenario.

Decision rule: if omitted capabilities share entry point, data shape, and failure mode with the covered one → legitimate partial. Otherwise → distinct partial; add scenarios before handoff.

If all boxes pass, output the portfolio. Otherwise iterate.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: CharmYue
• Source: CharmYue/claude-skill-e2e-scenario-design
• License: MIT

Install and usage instructions live in the source repository linked above.