# Quality Playbook

> Run a complete quality engineering audit on any codebase. Derives behavioral requirements from the code, generates spec-traced functional tests, runs a three-pass code review with regression tests, executes a multi-model spec audit (Council of Three), and produces a consolidated bug report with TDD-verified patches. Finds the 35% of real defects that structural code review alone cannot catch. Wor…

- **Type:** Skill
- **Install:** `agentstack add skill-andrewstellman-quality-playbook-quality-playbook`
- **Verified:** Pending review
- **Seller:** [andrewstellman](https://agentstack.voostack.com/s/andrewstellman)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [andrewstellman](https://github.com/andrewstellman)
- **Source:** https://github.com/andrewstellman/quality-playbook/tree/main/plugins/quality-playbook/skills/quality-playbook

## Install

```sh
agentstack add skill-andrewstellman-quality-playbook-quality-playbook
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Quality Playbook Generator

## Plan Overview — read this first, then explain it to the user

Before reading any other section of this skill, understand the plan and its dependencies. Each phase produces artifacts that the next phase depends on. Skipping or rushing a phase means every downstream phase works from incomplete information.

**Phase 0 (Prior Run Analysis):** If previous quality runs exist, load their findings as seed data. This is automatic and only applies to re-runs.

**Phase 1 (Explore):** Run the documentation intake first. **Invocation form:** `bin/reference_docs_ingest.py` does `from bin import benchmark_lib` at import, so Python's `-m bin.reference_docs_ingest` resolves ONLY when `bin/` is reachable as a top-level package. Resolve the **install root** (the directory containing this SKILL.md) via the same install-location fallback list this skill uses for `references/` / `quality_gate.py` (the 10 canonical layouts documented below — install-root forms: ``, `.claude/skills/quality-playbook`, `.github/skills`, `.github/skills/quality-playbook`, `.cursor/skills/quality-playbook`, `.continue/skills/quality-playbook`, `.codex/skills/quality-playbook`, `.windsurf/skills/quality-playbook`, `.cline/skills/quality-playbook`, `.aider/skills/quality-playbook`). Then, from the target repo root, run `PYTHONPATH= python3 -m bin.reference_docs_ingest `. If `` is empty (QPB self-bootstrap, or a `setup_repos.sh` install where `bin/` is already at the target root) omit the `PYTHONPATH=` prefix — cwd already provides `bin/`. The bare `python -m bin.reference_docs_ingest ` works ONLY for those two layouts; an `install_skill.py`-layout adopter (bin/ inside the marker dir) otherwise gets `ModuleNotFoundError: No module named 'bin'`. This walks `reference_docs/` — `cite/` files produce `quality/formal_docs_manifest.json` records; top-level files are loaded as Tier 4 context via `reference_docs_ingest.load_tier4_context()`. Then explore the codebase in three stages: open exploration driven by domain knowledge, domain-knowledge risk analysis, and selected structured exploration patterns. Write all findings to `quality/EXPLORATION.md`. This file is the foundation — Phase 2 reads it as its primary input.

**Phase 2 (Generate):** Read EXPLORATION.md and produce the quality artifacts: requirements, constitution, functional tests, code review protocol, integration tests, spec audit protocol, TDD protocol. (`AGENTS.md` at the target's repo root is generated by the orchestrator AFTER Phase 6, not by you in Phase 2 — see "File 6" below for the contract.)

**Phase 3 (Code Review):** Run the three-pass code review against HEAD. Write regression tests for every confirmed bug. Generate patches.

**Phase 4 (Spec Audit):** Three independent AI auditors review the code against requirements. Triage with verification probes. After triage, the same Council runs the Layer-2 semantic citation check — one prompt per reviewer, structured per-REQ verdicts for every Tier 1/2 citation, output to `quality/citation_semantic_check.json`. Write regression tests for net-new findings.

**Phase 5 (Reconciliation):** Close the loop — every bug from code review and spec audit is tracked, regression-tested or explicitly exempted. Run TDD red-green cycle. Finalize the completeness report.

**Phase 6 (Verify):** Run self-check benchmarks against all generated artifacts. Check for internal consistency, version stamp correctness, and convergence.

**Phase 7 (Present, Explore, Improve):** Present results to the user with a scannable summary table, offer drill-down on any artifact, and provide a menu of improvement paths (iteration strategies, requirement refinement, integration test tuning). This is the interactive phase where the user takes ownership of the quality system.

Every bug found traces back to a requirement, and every requirement traces back to an exploration finding.

**The critical dependency chain:** Exploration findings → EXPLORATION.md → Requirements → Code review + Spec audit → Bug discovery. A shallow exploration produces abstract requirements. Abstract requirements miss bugs. The exploration phase is where bugs are won or lost.

**MANDATORY FIRST ACTION:** After reading and understanding the plan above, print the following attribution banner **exactly as written below — do NOT condense, abbreviate, summarize, reformat, or add a version number**. Reproduce the full 8-line canonical block including the two `===…===` rules, both taglines, the blank lines, and the license line, byte-for-byte matching `bin/_purpose.print_attribution_banner()` (the single source of truth). Then explain the plan in your own words — what you'll do, what each phase produces, and why the exploration phase matters most. Emphasize that exploration starts with open-ended domain-driven investigation, followed by domain-knowledge risk analysis that reasons about what goes wrong in systems like this, then supplemented by selected structured patterns. Do not copy the plan verbatim; paraphrase it to demonstrate understanding.

```
================================================================================
  Quality Playbook -- by Andrew Stellman
  https://github.com/andrewstellman/quality-playbook

  AI code review is good. Quality engineering is better.
  Because code that looks right can still do the wrong thing.

  Licensed under the Apache License, Version 2.0
================================================================================
```

Generate a complete quality system tailored to a specific codebase. Unlike test stub generators that work mechanically from source code, this skill explores the project first — understanding its domain, architecture, specifications, and failure history — then produces a quality playbook grounded in what it finds.

## How to run this — self-encoded invocation contract

If the operator hands you this skill (or points you at any QPB-installed target) and says **"Run the Quality Playbook"** — possibly with a hint like "this is a bootstrap run" or "run on itself" or "self-audit" — this section tells you exactly what to do. The operator should not need to provide additional instructions; the canonical invocation, the defaults, the guardrails, and the output contract all live here.

### Pick your execution mode

QPB ships in two execution shapes. Pick the one that matches your runtime — the wrong choice produces the codex-on-codex indirection pathology.

| Mode | When this is you | What you do |
|------|------------------|-------------|
| **A. Skill-direct (UI-context)** | You are a coding agent (Claude Code, Cursor, Copilot, Codex desktop, etc.) handed this skill in your own chat. Your runtime IS the reasoning loop — you read files, you write files, you decide. | Walk through Phase 1 → Phase 6 yourself using the externalized phase prompts in `phase_prompts/`. Write artifacts into the target's `quality/` directory directly. No subprocess, no runner. |
| **B. Runner-driven (CLI-automation)** | The operator is invoking `python3 -m bin.run_playbook` deliberately — to batch across multiple targets, drive a headless CI run, or fan out per-phase work to a different model than the one reading this prose. | The orchestrator spawns a CLI agent (`claude`, `copilot`, `codex`, or `cursor`) per phase. You (or whoever is reading this) are the operator-side control loop, not the per-phase reasoner. |

**Both modes use the same phase prompt content** — the `phase_prompts/*.md` files at the repo root are the single source of truth, loaded by `bin/run_playbook.py::_load_phase_prompt` and read directly by Mode A walkthroughs. The only thing the two modes differ on is WHO drives — you (Mode A) or the orchestrator subprocess-spawning a CLI agent (Mode B).

**When in doubt, default to Mode A.** If the operator wanted runner-driven invocation they would have run the runner themselves; if they pasted "Run the Quality Playbook" into your chat, they want you to drive. The Mode B section below tells you what to do *if* the operator explicitly invokes the runner.

**Documented Mode A vs Mode B asymmetries — by design.** Three behaviors deliberately differ between the modes because the modes serve different purposes (Mode A is the adopter-facing walkthrough on a target repo; Mode B is the QPB-internal runner used for benchmarking + headless CI). The phase prompts are identical (single source of truth — see paragraph above); the divergences are in the *driver* obligations, not the *audited contract*:

1. **Phase 0 entry contract — install validator (Mode A only).** Mode A's mandatory first action is to run the install validator. **Invocation form is install-location-aware:** resolve the **install root** (the directory containing this SKILL.md) via the same install-location fallback list this skill uses for `references/` / `phase_prompts/` (the 10 canonical layouts documented below), then run **`python3 /bin/qpb_validate.py `**. For a clone / self-bootstrap (install root is the QPB clone, `bin/` reachable from cwd) the bare `python3 bin/qpb_validate.py ` works. For an `install_skill`-layout adopter (channel or manual install — `bin/` inside the marker dir, e.g. `.github/skills/quality-playbook/bin/`) you MUST use the install-root-prefixed form; running bare `bin/qpb_validate.py` from the target repo root fails (the validator lives under the marker dir, not at repo root). The adopter has just installed QPB into their target repo and must confirm canonical layout before any phase. Mode B (the QPB clone running its own runner) operates from the canonical install by construction; the validator would always succeed against the QPB clone itself, so the check would be vacuous. Adopters running Mode A get the install correctness gate; Mode B operators have it already.
2. **End-of-run archive (Mode B only).** Mode B's `_finalize_iteration` calls `archive_lib.archive_run` after Phase 6, materializing `quality/previous_runs//` for benchmark replay + cross-run diffing. Mode A produces artifacts directly into the target's live `quality/` for the adopter to inspect, commit, or re-run on top of; there is no archive partition because there is no benchmark loop. Adopters who want longitudinal archives invoke Mode B explicitly (`python3 -m bin.run_playbook --full-run `).
3. **Phase 6 auditor-prompt sub-agent dispatch (Mode A only).** `phase_prompts/phase6.md` routes Mode A through `phase_prompts/phase6_auditor.md` via a fresh-context sub-agent spawn (the A-13 hybrid). Mode B's per-phase CLI subprocess is *already* a fresh context (the structural separation the auditor exists to provide), so `phase6.md:5-7` tells Mode B to execute the verification inline and skip the auditor spawn. `phase_prompts/phase6_auditor.md` is therefore only loaded under Mode A; the file's header documents this routing explicitly.

These asymmetries are pinned by `bin/tests/test_mode_a_b_parity_documented.py`. If a future change wants to align them, update the pin test, the rationale here, and add the missing behavior to the appropriate side.

### Mode A — skill-direct walkthrough (UI-context)

The operator's prompt is just **"Run the Quality Playbook"** (or "run on itself", "self-audit", etc.). You drive every phase inline.

**Mode A means YOU execute the skill.** If you are an interactive coding agent (Claude Code, Cursor, Copilot UI, Codex desktop, any tool where the operator is monitoring your chat output in real-time), DO NOT:

- Spawn a sub-skill via your `quality-playbook` skill-invocation mechanism.
- Spawn a sub-agent via your Task tool / Agent tool / similar.
- Invoke `python3 -m bin.run_playbook` — that's Mode B, the CLI-automation path.
- Read `agents/quality-playbook.agent.md` or `agents/quality-playbook-claude.agent.md` and follow their "spawn sub-agents per phase" pattern — those files are AUTOMATION ONLY (batch CI runs, headless invocations) where per-phase context-window isolation is necessary at the cost of operator chat visibility.

**Phase 0 entry contract (MANDATORY first action): run the QPB install validator.** Resolve the install root via the install-location fallback list (per the Phase 0 numbered step above), then run `python3 /bin/qpb_validate.py ` — bare `python3 bin/qpb_validate.py ` works only when cwd already contains `bin/` (QPB clone / self-bootstrap); `install_skill`-layout adopters (channel install — `bin/` under `.github/skills/quality-playbook/`, etc.) MUST prefix the install root. Paste every `event=` line; do not proceed until `event=validation_complete status=ok` — see `AGENTS.md` "Mode A entry sequence". Non-negotiable — skipped/fabricated validation and bare-path-from-repo-root invocation are both documented adopter failure modes.

**EXCEPTION: Phase 6 verification MUST be delegated to a fresh-context sub-agent.** Sub-agent delegation is forbidden for EXECUTION (Phases 1-5) because hiding work from operator chat defeats real-time monitoring. Sub-agent delegation is MANDATED for VERIFICATION (Phase 6) because shared-context executor-as-verifier has empirically fabricated PASS verdicts against failing gates across multiple benchmark runs despite repeated rounds of strengthened prompt-level witness language. The principle is the same as the rule above — operator chat should carry the truth — but the mechanism is opposite: visibility for execution; isolation for verification (a fresh auditor has none of the executor's memory/anchoring/completion-reward bias). See `phase_prompts/phase6.md` + `phase_prompts/phase6_auditor.md` for the sub-agent delegation protocol. This exception is scoped strictly to Phase 6 verification — the no-sub-agent rule still holds for all Phase 1-5 execution.

The failure mode that motivated this rule: a shared-context session ran phases 1-6, fabricated the gate-verdict log (`quality/results/quality-gate.log` was hand-written with a "PASS" tally while the actual gate showed 14 FAIL, GATE FAILED), and reported PASS to the parent. The Phase 6 witness contract (your obligation to quote the gate's verdict lines verbatim) cannot enforce when a sub-skill owns the witness — only the parent's chat carries operator-readable evidence.

YOU read SKILL.md and the phase prompts. YOU execute Phases 1-5 yourself and write artifacts to `quality/`. Phase 6 verification is delegated to the fresh-context auditor sub-agent per the exception above (the auditor invokes `quality_gate.py`); you paste its verbatim verdict. Your chat IS the trust trail — execution stays visible, verification stays bias-isolated.

**You have shell access — probe before claiming unavailability.** Mode A means an interactive coding agent (Claude Code, Cursor, Copilot UI) running commands on the operator's actual machine via your Bash tool — NOT a sandboxed runtime. A common failure is writing "the sandbox can't run X" reflexively, conflating your model-level "I can't natively execute" with the harness Bash tool you DO have. A `which ` / ` --version` probe costs one line; assuming-and-being-wrong forfeits the entire Phase 5 evidence trail.

For each phase 1..6, in order:

1. **Load the phase prompt.** Read `phase_prompts/phaseN.md` (resolve via the same install-location fallback list documented for `references/` below). For `phase1.md`, substitute `{seed_instruction}` (the prelude that says "skip Phase 0/0b" — empty string when seeds are allowed) and `{role_taxonomy}` (the taxonomy block rendered from the role taxonomy below). For `phase2.md` through `phase6.md`, the file is pure-literal — read it verbatim.
2. **Execute the phase per the prompt.** Read the inputs the prompt names, do the analysis, write outputs into the target's `quality/` directory.
3. **STOP at the end-of-phase boundary.** Every phase prompt ends with an "IMPORTANT: Do NOT proceed to Phase N+1" instruction. Honor it. The operator advances to the next phase by saying so.

You are responsible — without the orchestrator's structural backstop — for the same source-unchanged invariant the runner enforces:

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [andrewstellman](https://github.com/andrewstellman)
- **Source:** [andrewstellman/quality-playbook](https://github.com/andrewstellman/quality-playbook)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** yes
- **Shell / process execution:** yes
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: flagged — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-andrewstellman-quality-playbook-quality-playbook
- Seller: https://agentstack.voostack.com/s/andrewstellman
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.