E2e Testing

E2E Testing

Drive end-to-end tests through Playwright's MCP-backed Test Agents — Planner, Generator, Healer — released in Playwright 1.56 (Oct 2025). The user writes (or approves) a Markdown feature spec; agents generate the test, run it against a real browser via the accessibility tree, and self-heal when locators drift.

> This SKILL.md is a thin index. > Decision rules live in [rules/*.md](./rules) and load on demand. > Worked references (agent reference, MCP tool catalog, pyramid math) live > in [references/*.md](./references). > Literal boilerplate the skill emits lives in [templates/*.md](./templates). > Do not preload everything — load only what the current phase asks for.

---

When to use

Reach for this skill when any of the following is true:

• A feature has user-facing flow that integration tests cannot fully cover.
• A bug repros only through real navigation (multi-page, auth, real network).
• A flake needs a Healer pass instead of a manual locator hunt.
• The repo has no @playwright/mcp wiring yet and needs Phase 0 setup.

Do not reach for this skill when:

• A unit or component test would catch the same bug — defer to

[tdd](../../quality/tdd/SKILL.md) and the layer rule in [rules/layer-decision.md](./rules/layer-decision.md).

• The change is a pure refactor with no behavioural surface.
• You are adding test infrastructure unrelated to a real flow.

---

Phase 0 — Preflight (mandatory gate)

Before any agent loop, verify the repo is wired for Playwright Test Agents. Halt and ask the user before installing anything.

Run these checks (read-only):

# 1. Playwright + MCP server installed?
jq '.devDependencies | keys[]' package.json | grep -E '@playwright/(test|mcp)'

# 2. Test-agent artefacts present?
ls specs/ tests/seed.spec.ts playwright.config.ts 2>/dev/null

Decision table:

| State | Action | | ------------------------------------------- | ----------------------------------------------------------------------- | | Both deps present + artefacts exist | Proceed to Phase 1. | | Deps missing | Halt. Print install plan, ask permission before running. | | Deps present, artefacts missing | Halt. Print npx playwright init-agents --loop=claude, ask first. | | Playwright present but version .md ─┐ ├─→ Generator ─→ tests/.spec.ts ─→ run ─→ pass? │ │ no │ ▼ └──────────────────── Healer ←────────────── failing test │ ▼ patched test or data-testid` proposal

Two entry points:

1. **Spec already drafted by the user.**
   Skip the Planner.
   Run the Generator on `specs/.md`.
2. **App exists, no spec yet.**
   Run the Planner against the live app to draft `specs/.md`.
   User reviews the Markdown plan before generation.

Use the Markdown template in [`templates/spec.md`](./templates/spec.md).

### Locator ladder (when generating or healing)

The Generator and the Healer both walk the accessibility tree.
Pick locators in this order — never skip a rung:

1. `getByRole('button', { name: 'Save' })` — accessibility-tree native.
2. `getByLabel`, `getByPlaceholder`, `getByText` — user-facing strings.
3. `getByTestId('save-draft')` — escape hatch only.

`data-testid` is a source change, not a test workaround.
When the Healer cannot find a stable locator at rungs 1–2, propose a source
diff that adds `data-testid` to the component, and offer the diff for user
approval before patching the test.
Full rules and decision criteria: [`rules/locator-strategy.md`](./rules/locator-strategy.md).

---

## Phase 2 — Token-aware execution

Playwright MCP defaults to **snapshot mode** (accessibility tree, text-only).
Do not enable `--caps=vision` unless an explicit pixel-level concern exists.
Full rules: [`rules/token-budget.md`](./rules/token-budget.md).

Defaults the skill prescribes:

- Snapshot mode (no vision) for all agent calls.
- Run only the changed spec on iteration: `npx playwright test --last-failed`.
- Run the Healer **only on failure**, not on every save.
- Reuse `storageState` from `tests/seed.spec.ts` to skip auth on every run.
- Cap the heal loop at three attempts per failing test before escalating.

---

## Phase 3 — Verification

After the Generator produces a test:

1. Run the test once against the live app.
   It must pass on first run, or the Healer must converge in ≤ 3 attempts.
2. Invoke [`test-provenance-guard`](../../quality/test-provenance-guard/SKILL.md) on
   the generated file to ensure the test imports production code instead
   of a private re-implementation.
3. Open `playwright.config.ts` and confirm `trace: 'on-first-retry'` is set
   so a future failure produces a trace bundle.

If the heal loop fails to converge:

- Invoke `confidence(analysis)` on the test failure.
- If confidence is below 90%, escalate to the user with the trace, the spec,
  and the proposed locator changes — do **not** keep healing blindly.

---

## Decision flow at a glance

| Signal                                                       | Do                                                            |
| ------------------------------------------------------------ | ------------------------------------------------------------- |
| Bug fixable by a unit or component test                      | Use [`tdd`](../../quality/tdd/SKILL.md), not this skill.                 |
| Multi-page user flow, auth, or real network involved         | Spec-first feature flow (Phase 1).                            |
| Flaky existing test                                          | Healer pass only; do not rewrite without spec context.        |
| Locator unstable, no stable role / label                     | Propose `data-testid` diff (rule: locator-strategy).          |
| Repo missing Playwright or MCP                               | Phase 0 halt + ask permission.                                |
| Heal loop > 3 attempts                                       | Stop, run `confidence(analysis)`, escalate.               |
| Test passes on first run, never seen failing                 | Run `test-provenance-guard` before declaring done.            |

---

## Composes with

- [`tdd`](../../quality/tdd/SKILL.md) — owns the unit and component layers.
  This skill defers to it for anything below E2E.
- [`test-provenance-guard`](../../quality/test-provenance-guard/SKILL.md) — runs after
  Generator output to catch tests-by-construction.
- [`confidence`](../../quality/confidence/SKILL.md) — gate when the heal loop fails.
- [`holistic-analysis`](../../analysis/holistic-analysis/SKILL.md) — if a flow is failing
  for reasons no test rewrite can fix, step back instead of patching.
- [`playwright-trace-analyzer`](../../analysis/playwright-trace-analyzer/SKILL.md) —
  consume the trace produced by a failed test on retry.

---

## References

- [`references/playwright-agents.md`](./references/playwright-agents.md) —
  Planner / Generator / Healer reference, inputs, outputs, invocation.
- [`references/mcp-tool-catalog.md`](./references/mcp-tool-catalog.md) —
  the `@playwright/mcp` tool surface, grouped by category.
- [`references/pyramid-2026.md`](./references/pyramid-2026.md) — testing
  pyramid math in 2026, with the AI-generation caveat.

## Templates

- [`templates/spec.md`](./templates/spec.md) — feature-flow Markdown spec.
- [`templates/seed.spec.ts`](./templates/seed.spec.ts) — auth and storage
  bootstrap, produces `storageState`.
- [`templates/playwright.config.ts`](./templates/playwright.config.ts) —
  opinionated config: snapshot mode, traces on first retry, projects per
  browser, parallel CI defaults.
- [`templates/install-plan.md`](./templates/install-plan.md) — Phase 0 halt
  message with the exact commands to install Playwright + MCP.

---

## Anti-patterns (one-liner — full list in [`rules/anti-patterns.md`](./rules/anti-patterns.md))

- Writing E2E for logic a unit test catches.
- Running the Healer on every save.
- Patching the test with brittle CSS selectors instead of proposing a
  `data-testid` diff.
- Enabling `--caps=vision` without a pixel-level requirement.
- Ignoring Healer suggestions and keeping a `.skip()` in CI.
- Generating tests against a stub server, not the real app.

---

## Definition of done

- [ ] Phase 0 preflight passed or installs were user-approved.
- [ ] `specs/.md` exists and the user reviewed it.
- [ ] `tests/.spec.ts` passes against the live app.
- [ ] `test-provenance-guard` reports no violations on the new test.
- [ ] `playwright.config.ts` has `trace: 'on-first-retry'`.
- [ ] If a `data-testid` was added, it is in the source diff and committed.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [mthines](https://github.com/mthines)
- **Source:** [mthines/agent-skills](https://github.com/mthines/agent-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.