# Quality Playbook

> Run a complete quality engineering audit on any codebase. Derives behavioral requirements from the code, generates spec-traced functional tests, runs a three-pass code review with regression tests, executes a multi-model spec audit (Council of Three), and produces a consolidated bug report with TDD-verified patches. Finds the 35% of real defects that structural code review alone cannot catch. Wor…

- **Type:** Skill
- **Install:** `agentstack add skill-andrewstellman-quality-playbook-skills`
- **Verified:** Pending review
- **Seller:** [andrewstellman](https://agentstack.voostack.com/s/andrewstellman)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [andrewstellman](https://github.com/andrewstellman)
- **Source:** https://github.com/andrewstellman/quality-playbook/tree/main/repos/pdf-1.5.4/.github/skills

## Install

```sh
agentstack add skill-andrewstellman-quality-playbook-skills
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Quality Playbook Generator

## Plan Overview — read this first, then explain it to the user

Before reading any other section of this skill, understand the plan and its dependencies. Each phase produces artifacts that the next phase depends on. Skipping or rushing a phase means every downstream phase works from incomplete information.

**Phase 0 (Prior Run Analysis):** If previous quality runs exist, load their findings as seed data. This is automatic and only applies to re-runs.

**Phase 1 (Explore):** Run the v1.5.3 documentation intake first (`python -m bin.reference_docs_ingest ` to walk `reference_docs/` — `cite/` files produce `quality/formal_docs_manifest.json` records; top-level files are loaded as Tier 4 context via `reference_docs_ingest.load_tier4_context()`). Then explore the codebase in three stages: open exploration driven by domain knowledge, domain-knowledge risk analysis, and selected structured exploration patterns. Write all findings to `quality/EXPLORATION.md`. This file is the foundation — Phase 2 reads it as its primary input.

**Phase 2 (Generate):** Read EXPLORATION.md and produce the quality artifacts: requirements, constitution, functional tests, code review protocol, integration tests, spec audit protocol, TDD protocol, AGENTS.md.

**Phase 3 (Code Review):** Run the three-pass code review against HEAD. Write regression tests for every confirmed bug. Generate patches.

**Phase 4 (Spec Audit):** Three independent AI auditors review the code against requirements. Triage with verification probes. After triage, the same Council runs the v1.5.3 Layer-2 semantic citation check — one prompt per reviewer, structured per-REQ verdicts for every Tier 1/2 citation, output to `quality/citation_semantic_check.json`. Write regression tests for net-new findings.

**Phase 5 (Reconciliation):** Close the loop — every bug from code review and spec audit is tracked, regression-tested or explicitly exempted. Run TDD red-green cycle. Finalize the completeness report.

**Phase 6 (Verify):** Run self-check benchmarks against all generated artifacts. Check for internal consistency, version stamp correctness, and convergence.

**Phase 7 (Present, Explore, Improve):** Present results to the user with a scannable summary table, offer drill-down on any artifact, and provide a menu of improvement paths (iteration strategies, requirement refinement, integration test tuning). This is the interactive phase where the user takes ownership of the quality system.

Every bug found traces back to a requirement, and every requirement traces back to an exploration finding.

**The critical dependency chain:** Exploration findings → EXPLORATION.md → Requirements → Code review + Spec audit → Bug discovery. A shallow exploration produces abstract requirements. Abstract requirements miss bugs. The exploration phase is where bugs are won or lost.

**MANDATORY FIRST ACTION:** After reading and understanding the plan above, print the following message to the user, then explain the plan in your own words — what you'll do, what each phase produces, and why the exploration phase matters most. Emphasize that exploration starts with open-ended domain-driven investigation, followed by domain-knowledge risk analysis that reasons about what goes wrong in systems like this, then supplemented by selected structured patterns. Do not copy the plan verbatim; paraphrase it to demonstrate understanding.

> Quality Playbook v1.5.3 — by Andrew Stellman
> https://github.com/andrewstellman/quality-playbook

Generate a complete quality system tailored to a specific codebase. Unlike test stub generators that work mechanically from source code, this skill explores the project first — understanding its domain, architecture, specifications, and failure history — then produces a quality playbook grounded in what it finds.

### Locating reference files

This skill references files in a `references/` directory (e.g., `references/iteration.md`, `references/review_protocols.md`). The location depends on how the skill was installed. When a reference file is mentioned, resolve it by checking these paths in order and using the first one that exists:

1. `references/` (relative to SKILL.md — works when running from the skill directory)
2. `.claude/skills/quality-playbook/references/` (Claude Code installation)
3. `.github/skills/references/` (GitHub Copilot flat installation)
4. `.github/skills/quality-playbook/references/` (alternate Copilot installation)

All reference file mentions in this skill use the short form `references/filename.md`. If the relative path doesn't resolve, walk the fallback list above.

## Why This Exists

Most software projects have tests, but few have a quality *system*. Tests check whether code works. A quality system answers harder questions: what does "working correctly" mean for this specific project? What are the ways it could fail that wouldn't be caught by tests? What should every developer (human or AI) know before touching this code?

Without a quality playbook, every new contributor (and every new AI session) starts from scratch — guessing at what matters, writing tests that look good but don't catch real bugs, and rediscovering failure modes that were already found and fixed months ago. A quality playbook makes the bar explicit, persistent, and inherited.

## What This Skill Produces

Nine files that together form a repeatable quality system:

| File | Purpose | Why It Matters | Executes Code? |
|------|---------|----------------|----------------|
| `quality/QUALITY.md` | Quality constitution — coverage targets, fitness-to-purpose scenarios, theater prevention | Every AI session reads this first. It tells them what "good enough" means so they don't guess. | No |
| `quality/REQUIREMENTS.md` | Testable requirements with project overview, use cases, and narrative — generated by a five-phase pipeline (contract extraction → derivation → verification → completeness → narrative) | The foundation for Passes 2 and 3 of the code review. Without requirements, review is limited to structural anomalies (~65% ceiling). With them, the review can catch intent violations — absence bugs, cross-file contradictions, and design gaps that are invisible to code reading alone. | No |
| `quality/test_functional.*` | Automated functional tests derived from specifications | The safety net. Tests tied to what the spec says should happen, not just what the code does. Use the project's language: `test_functional.py` (Python), `FunctionalSpec.scala` (Scala), `functional.test.ts` (TypeScript), `FunctionalTest.java` (Java), etc. | **Yes** |
| `quality/RUN_CODE_REVIEW.md` | Three-pass code review protocol: structural review, requirement verification, cross-requirement consistency | Structural review alone misses ~35% of real defects. The three-pass pipeline adds requirement verification and consistency checking — backed by experiment evidence showing it finds bugs invisible to all structural review conditions. | No |
| `quality/RUN_INTEGRATION_TESTS.md` | Integration test protocol — end-to-end pipeline across all variants | Unit tests pass, but does the system actually work end-to-end with real external services? | **Yes** |
| `quality/BUGS.md` | Consolidated bug report with patches | Every confirmed bug in one place with reproduction details, spec basis, severity, and patch references. The single source of truth for what's broken and how to verify it. | No |
| `quality/RUN_TDD_TESTS.md` | TDD red-green verification protocol | Proves each bug is real (test fails on unpatched code) and each fix works (test passes after patch). Stronger evidence than a bug report alone — maintainers trust FAIL→PASS demonstrations. | **Yes** |
| `quality/RUN_SPEC_AUDIT.md` | Council of Three multi-model spec audit protocol | No single AI model catches everything. Three independent models with different blind spots catch defects that any one alone would miss. | No |
| `AGENTS.md` | Bootstrap context for any AI session working on this project | The "read this first" file. Without it, AI sessions waste their first hour figuring out what's going on. | No |

Plus output directories: `quality/code_reviews/`, `quality/spec_audits/`, `quality/results/`, `quality/history/`.

The pipeline also generates supporting artifacts: `quality/PROGRESS.md` (phase-by-phase checkpoint log with cumulative BUG tracker), `quality/CONTRACTS.md` (behavioral contracts), `quality/COVERAGE_MATRIX.md` (traceability), `quality/COMPLETENESS_REPORT.md` (final gate), and `quality/VERSION_HISTORY.md` (review log). Phase 7 can additionally generate `quality/REVIEW_REQUIREMENTS.md` (interactive review protocol) and `quality/REFINE_REQUIREMENTS.md` (refinement pass protocol) for iterative improvement.

The two critical deliverables are the requirements file and the functional test file. The requirements file (`quality/REQUIREMENTS.md`) feeds the code review protocol's verification and consistency passes — it's what makes the code review catch more than structural anomalies. The functional test file (named for the project's language and test framework conventions) is the automated safety net. The Markdown protocols are documentation for humans and AI agents.

### Complete Artifact Contract

The quality gate (`quality_gate.py`) validates these artifacts. If the gate checks for it, this skill must instruct its creation. This is the canonical list — any artifact not listed here should not be gate-enforced, and any gate check should trace to an artifact listed here.

| Artifact | Location | Required? | Created In |
|----------|----------|-----------|------------|
| Formal docs manifest (v1.5.3) | `quality/formal_docs_manifest.json` | Yes | Phase 1 (`bin/reference_docs_ingest.py`) |
| Requirements manifest (v1.5.3) | `quality/requirements_manifest.json` | Yes | Phase 2 |
| Use cases manifest (v1.5.3) | `quality/use_cases_manifest.json` | Yes | Phase 2 |
| Bugs manifest (v1.5.3) | `quality/bugs_manifest.json` | If bugs found | Phase 3/4/5 |
| Citation semantic check (v1.5.3) | `quality/citation_semantic_check.json` | Yes | Phase 4 (Layer 2 Council) |
| Exploration findings | `quality/EXPLORATION.md` | Yes | Phase 1 |
| Quality constitution | `quality/QUALITY.md` | Yes | Phase 2 |
| Requirements (UC identifiers) | `quality/REQUIREMENTS.md` | Yes | Phase 2 |
| Behavioral contracts | `quality/CONTRACTS.md` | Yes | Phase 2 |
| Functional tests | `quality/test_functional.*` | Yes | Phase 2 |
| Regression tests | `quality/test_regression.*` | If bugs found | Phase 3 |
| Code review protocol | `quality/RUN_CODE_REVIEW.md` | Yes | Phase 2 |
| Integration test protocol | `quality/RUN_INTEGRATION_TESTS.md` | Yes | Phase 2 |
| Spec audit protocol | `quality/RUN_SPEC_AUDIT.md` | Yes | Phase 2 |
| TDD verification protocol | `quality/RUN_TDD_TESTS.md` | Yes | Phase 2 |
| Bug tracker | `quality/BUGS.md` | Yes | Phase 3 |
| Coverage matrix | `quality/COVERAGE_MATRIX.md` | Yes | Phase 2 |
| Completeness report | `quality/COMPLETENESS_REPORT.md` | Yes | Phase 2 (baseline), Phase 5 (final verdict) |
| Progress tracker | `quality/PROGRESS.md` | Yes | Throughout |
| AI bootstrap | `AGENTS.md` | Yes | Phase 2 |
| Bug writeups | `quality/writeups/BUG-NNN.md` | If bugs found | Phase 5 |
| Regression patches | `quality/patches/BUG-NNN-regression-test.patch` | If bugs found | Phase 3 |
| Fix patches | `quality/patches/BUG-NNN-fix.patch` | Optional | Phase 3 |
| TDD traceability | `quality/TDD_TRACEABILITY.md` | If bugs have red-phase results | Phase 5 |
| TDD sidecar | `quality/results/tdd-results.json` | If bugs found | Phase 5 |
| TDD red-phase logs | `quality/results/BUG-NNN.red.log` | If bugs found | Phase 5 |
| TDD green-phase logs | `quality/results/BUG-NNN.green.log` | If fix patch exists | Phase 5 |
| Integration sidecar | `quality/results/integration-results.json` | When integration tests run | Phase 5 |
| Mechanical verify script | `quality/mechanical/verify.sh` | Yes (benchmark) | Phase 2 |
| Verify receipt | `quality/results/mechanical-verify.log` + `.exit` | Yes (benchmark) | Phase 5 |
| Triage probes | `quality/spec_audits/triage_probes.sh` | When triage runs | Phase 4 |
| Code review reports | `quality/code_reviews/*.md` | Yes | Phase 3 |
| Spec audit reports | `quality/spec_audits/*auditor*.md` + `*triage*` | Yes | Phase 4 |
| Recheck results (JSON) | `quality/results/recheck-results.json` | When recheck runs | Recheck |
| Recheck summary (MD) | `quality/results/recheck-summary.md` | When recheck runs | Recheck |
| Seed checks | `quality/SEED_CHECKS.md` | If Phase 0b ran | Phase 0b |
| Run metadata | `quality/results/run-YYYY-MM-DDTHH-MM-SS.json` | Yes | Phase 1 (created), Throughout (updated) |

**Sidecar JSON lifecycle:** Write all bug writeups *before* finalizing `tdd-results.json` — the sidecar's `writeup_path` field must point to an existing file, not a placeholder. Similarly, run integration tests and collect results before writing `integration-results.json`.

### Sidecar JSON Canonical Examples

**`quality/results/tdd-results.json`** — the gate validates field names, not just presence:

```json
{
  "schema_version": "1.1",
  "skill_version": "1.5.3",
  "date": "2026-04-12",
  "project": "repo-name",
  "bugs": [
    {
      "id": "BUG-001",
      "requirement": "REQ-003",
      "red_phase": "fail",
      "green_phase": "pass",
      "verdict": "TDD verified",
      "fix_patch_present": true,
      "writeup_path": "quality/writeups/BUG-001.md"
    }
  ],
  "summary": {
    "total": 3, "confirmed_open": 1, "red_failed": 0, "green_failed": 0, "verified": 2
  }
}
```

`verdict` must be one of: `"TDD verified"`, `"red failed"`, `"green failed"`, `"confirmed open"`, `"deferred"`. `date` must be ISO 8601 (YYYY-MM-DD), not a placeholder, not in the future.

**`quality/results/integration-results.json`:**

```json
{
  "schema_version": "1.1",
  "skill_version": "1.5.3",
  "date": "2026-04-12",
  "project": "repo-name",
  "recommendation": "SHIP",
  "groups": [{ "group": 1, "name": "Group 1", "use_cases": ["UC-01"], "result": "pass", "tests_passed": 3, "tests_failed": 0, "notes": "" }],
  "summary": { "total_groups": 12, "passed": 11, "failed": 1, "skipped": 0 },
  "uc_coverage": { "UC-01": "covered_pass", "UC-02": "not_mapped" }
}
```

`recommendation` must be one of: `"SHIP"`, `"FIX BEFORE MERGE"`, `"BLOCK"`. `uc_coverage` maps UC identifiers from REQUIREMENTS.md to coverage status.

### Run Metadata

Every playbook run creates a timestamped metadata file at `quality/results/run-YYYY-MM-DDTHH-MM-SS.json`. This enables multi-model comparison and run history tracking.

**Lifecycle:** Create this file at the start of Phase 1. Update `phases_completed`, `bug_count`, and `end_time` as each phase finishes. The final update happens after the terminal gate.

```json
{
  "schema_version": "1.0",
  "skill_version": "1.5.3",
  "project": "repo-name",
  "model": "claude-sonnet-4-6",
  "model_provider": "anthropic",
  "runner": "claude-code",
  "start_time": "2026-04-16T10:30:00Z",
  "end_time": "2026-04-16T11:45:00Z",
  "duration_minutes": 75,
  "phases_completed": ["Phase 0b", "Phase 1", "Phase 2", "Phase 3", "Phase 4", "Phase 5"],
  "iterations_completed": ["gap", "unfiltered", "parity", "adversarial"],
  "bug_count": 12,
  "bug_severity": { "HIGH": 2, "MEDIUM": 5, "LOW": 5 },
  "gate_result": "PASS",
  "gate_fail_count": 0,
  "gate_warn_count": 2,
  "notes": ""
}
```

**Required fields:** `schema_version`, `skill_version`, `project`, `model`, `start_time`. All other fields are populated as the run progresses. `model` should be the exact model string (e.g., `"claude-sonnet-4-6"`, `"gpt-4.1"`, `"claude-opus-4-6"`). `runner` identifies the tool used to execute the playbook (e.g., `"claude-code"`, `"copilot-cli"`, `"cursor"`, `"cowork"`). `duration_minutes` is computed from `end_time - start_time`. If the model or runner cannot be determined, use `"unknown"`.

## How to Use

**The playbook is designed to run one phase at a time.*

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [andrewstellman](https://github.com/andrewstellman)
- **Source:** [andrewstellman/quality-playbook](https://github.com/andrewstellman/quality-playbook)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** yes
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** yes

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: flagged — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-andrewstellman-quality-playbook-skills
- Seller: https://agentstack.voostack.com/s/andrewstellman
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.