Docs List Undocumented Prs

Skill: docs-list-undocumented-prs

Description

Scans merged GitHub PRs and produces a documentation-coverage audit report. For each PR, the skill determines whether docs are required, checks whether they exist, and grades coverage against a four-level rubric. Results are batched in groups of 20 with a persistent state file so that already-audited PRs are never re-processed.

Trigger: User says "doc audit", "documentation audit", "which PRs need docs", "show me undocumented changes", "what needs to be documented", "docs coverage", or similar phrasing.

---

Phase 0 — Load Persistent State

Read .docs-audit-state.json from the repo root. Create the file if it does not exist.

State file schema:

{
  "repo": "owner/repo",
  "checked_prs": {
    "42": {
      "title": "Add ZStream.throttle operator",
      "docs_required": "yes",
      "classification_gate": "YES-2",
      "classification_reason": "New public Scala file added: src/main/scala/zio/stream/ZStream.scala",
      "status": "Not Documented",
      "checked_at": "2026-05-19T10:00:00Z"
    },
    "41": {
      "title": "Fix HttpClient connection leak",
      "docs_required": "no",
      "classification_gate": "NO-7",
      "classification_reason": "Bug fix label with no new public files added",
      "status": null,
      "checked_at": "2026-05-19T10:00:00Z"
    }
  }
}

• checked_prs: map of PR number (string) → { title, docs_required, classification_gate, classification_reason, status, checked_at }
• docs_required: "yes" | "no" | "uncertain"
• classification_gate: ID of the gate that fired (e.g., "YES-2", "NO-7", "OVERRIDE-BREAKING", "UNCERTAIN")
• classification_reason: Human-readable one-line explanation of why the gate fired
• status: Documentation rubric grade from Phase 4 (or null when docs_required = "no")

Initialization rules:

• If the file does not exist, initialize it with:

``json { "repo": "", "checked_prs": {} } ``

• If the file exists but the repo field does not match the current repo, warn the user and ask whether to reset or abort.
• If --reset was passed as the argument, confirm with the user before clearing checked_prs.

Gitignore reminder: Remind the user to add .docs-audit-state.json to .gitignore if it is not already listed there.

---

Phase 1 — Collect Next Batch of 20 Unchecked PRs

Step 1a — Detect the repo

git remote -v

Parse owner/repo from the remote URL (handles both SSH and HTTPS formats).

Step 1b — Fetch candidate PRs

Fetch 40 merged PRs to have room to filter already-checked ones:

gh pr list --state merged --limit 40 \
  --json number,title,labels,mergedAt \
  --repo 

If a base-ref argument was given (e.g., origin/main, v1.0.0), cross-reference with git log:

git log ..HEAD --oneline --merges

Extract PR numbers from merge commit messages and restrict the gh results to those PRs.

Step 1c — Filter and select

1. Remove PR numbers already present in checked_prs.
2. Take the first 20 remaining candidates (newest first).
3. Handle edge cases:

• 0 unchecked remain: Print "All recent PRs have been checked. Here's a cumulative summary:" followed by aggregated stats from checked_prs. Stop.
• 1–19 unchecked remain: Proceed with the available count and note it in the output.
• All 40 fetched are already checked: Paginate — increase --limit (e.g., --limit 80, --limit 120, etc.) and repeat until 20 unchecked are found or the list is exhausted.

---

Phase 2 — Process Each PR One by One

For each PR in the batch, print a progress line before processing:

> [N/20] Processing PR #: 

Fetch full PR details in a single call (metadata):

gh pr view  --repo  \
  --json number,title,body,labels,mergedAt,commits,author

Then fetch file details with statuses (added/modified/deleted/renamed):

gh api repos/{owner}/{repo}/pulls/{N}/files --paginate \
  --jq '[.[] | {path: .filename, status: .status, additions: .additions, deletions: .deletions}]'

Combining both calls gives complete PR context with file-level status information.

Run Phase 3 (classification) and Phase 4 (grading) for this PR before advancing to the next.

---

Phase 3 — Does This PR Require Documentation?

Classify the PR using a hierarchical, priority-ordered gate system. Extract preliminary variables, check overrides, evaluate NO gates (first match wins), then YES gates (first match wins), then default to UNCERTAIN.

Preliminary Variable Extraction

Before evaluating any gate, compute these derived facts from the PR data:

File categories:

• files_test: Paths matching *Test.scala, *Spec.scala, *Suite.scala, or under src/test/
• files_main_scala: .scala files under src/main/, excluding files_test
• files_internal: files_main_scala whose path contains /internal/, /impl/, or /private/
• files_public_main: files_main_scala minus files_internal
• files_new_public_main: files_public_main with file status = added
• files_infra: Paths matching *.yml, *.yaml, .github/**, Dockerfile*
• files_build: Paths matching build.sbt, project/**, *.sbt

Title and label facts:

• cc_prefix: Conventional commit type from title (e.g., feat, fix, chore, refactor, test, ci, build, docs, perf, style, revert) — extract if title matches ^(feat|fix|chore|refactor|test|ci|build|docs|perf|style|revert)(\(.+\))?!?:
• cc_breaking: True if title contains !: (breaking change marker)
• is_bump: True if title matches (case-insensitive): ^(bump|upgrade|update).*\b(to v?\d|version|dep) OR starts with build(deps)
• is_revert: True if title starts with Revert "

Evaluation Algorithm

1. Overrides (bypass all gates):
   - If cc_breaking = true → REQUIRES_DOCS = yes, GATE = "OVERRIDE-BREAKING"
   - If labels include "documentation-needed" → REQUIRES_DOCS = yes, GATE = "OVERRIDE-DOCS-NEEDED"

2. NO gates (first match wins → REQUIRES_DOCS = no):
   - Evaluate gates NO-1 through NO-9 in order

3. YES gates (first match wins → REQUIRES_DOCS = yes):
   - Evaluate gates YES-1 through YES-5 in order

4. Fallback (all gates failed):
   - REQUIRES_DOCS = uncertain, provide explanation

Override Conditions (checked first)

| Override | Condition | Gate ID | |----------|-----------|---------| | Breaking change | cc_breaking = true | OVERRIDE-BREAKING | | Docs explicitly needed | Labels include documentation-needed | OVERRIDE-DOCS-NEEDED |

NO Gates (any single gate is sufficient to return NO; evaluated in order)

| Gate ID | Name | Condition | Rationale | |---------|------|-----------|-----------| | NO-1 | Dependency update | is_bump = true OR labels include renovate or dependabot OR (cc_prefix = "build" AND files_main_scala is empty) | Dependency version bumps never introduce user-facing API changes. Renovate and dependabot labels are definitive. | | NO-2 | CI/infrastructure only | files_public_main is empty AND files_infra ∪ files_build is non-empty AND (labels include ci/infrastructure OR cc_prefix = "ci" OR cc_prefix = "build") | Changes that touch only workflow files, Docker configs, build system metadata cannot affect public API. Requires corroboration from label or commit prefix. | | NO-3 | Test-only changes | All changed files ⊆ (files_test ∪ files_infra ∪ files_build) AND files_public_main is empty | Changes that do not touch public source code require no documentation updates. | | NO-4 | Conventional fix: without new public files | cc_prefix = "fix" AND files_new_public_main is empty | A fix: prefix signals a bug correction to existing behavior. If no new public files are added, existing documentation remains valid. | | NO-5 | Conventional chore/refactor/test/ci/docs/style/perf/revert | cc_prefix ∈ {chore, refactor, test, ci, docs, style, perf, revert} AND files_new_public_main is empty | These conventional commit types signal maintenance, not API evolution. No new public files added means no new public surface. | | NO-6 | Internal-only refactor | files_public_main is empty AND files_internal is non-empty AND (labels include refactor or internal) | Changes touching only internal packages (*/internal/*, */impl/*, */private/*) are implementation details not visible in public API documentation. Requires refactor or internal label for corroboration. | | NO-7 | Bug fix label without new public files | Labels include bug, fix, or bugfix AND files_new_public_main is empty AND labels do NOT include any YES-gate labels (feature, enhancement, api-change, breaking-change) | Bug fix label combined with no new public files means existing behavior is corrected, not extended. | | NO-8 | Chore label with no source changes | Labels include chore or internal AND files_public_main is empty | Chore label + no public source changes = metadata/config update requiring no documentation. | | NO-9 | Revert commit | is_revert = true | Reverting a commit undoes changes already in the codebase. The original PR's documentation is no longer needed. |

YES Gates (any single gate is sufficient to return YES; evaluated in order after NO gates fail)

| Gate ID | Name | Condition | Rationale | |---------|------|-----------|-----------| | YES-1 | Explicit feature commit | cc_prefix = "feat" | The conventional commit feat: type is the authoritative signal that new functionality was introduced. | | YES-2 | New public source file added | files_new_public_main is non-empty | A .scala file added to src/main/ outside of internal packages is an extension of the public API surface. | | YES-3 | Breaking change label | Labels include breaking-change or api-change | Explicit breaking-change or api-change labels indicate API contract changes requiring documentation updates. | | YES-4 | Feature/enhancement label with source changes | Labels include feature, enhancement, or new-feature AND files_main_scala is non-empty | Feature or enhancement labels combined with actual source changes indicate new or extended functionality. Requiring source changes prevents false positives from mislabeled chore PRs. | | YES-5 | Public file modified with interface-change keywords | files_public_main is non-empty AND title contains (whole-word, case-insensitive match): introduce, deprecate, breaking, or API | Modification to public files combined with interface-change language suggests an API evolution requiring documentation. Whole-word matching prevents false positives like "add" in "added test" or "add CI". |

UNCERTAIN — No Gate Matched

If no gate fires (neither NO nor YES), classify as UNCERTAIN and include a brief explanation of what signals were observed. Examples:

• "Refactor label with public files modified — unclear if this is a breaking change or implementation detail"
• "No labels, both test and src/main files changed — requires manual review"
• "fix: prefix but new public file added — may be a silent feature addition"

---

Phase 4 — Grade Documentation Coverage

Skip this phase entirely if REQUIRES_DOCS = no.

Step 4a — Check for docs files in the PR

gh pr view  --repo  --json files \
  | jq -r '.files[].path | select(startswith("docs/"))'

List any docs/ paths that were modified or added in this PR.

Step 4b — Extract key symbols and search docs

Extract 3–5 key symbols from the PR title, body, and commit messages:

• PascalCase type names (e.g., ZStream, HttpClient)
• camelCase method names (e.g., throttle, mapZIO)

For each symbol, search the docs tree:

Use the Grep tool to search for each symbol across the docs/ directory (query: symbol name, path: docs/).

If docs/ does not exist in the repo, skip this step and mark all docs-required PRs as "Not Documented". Note the absence at the start of the report.

Step 4c — Assign documentation status

Apply the rubric below. Assign the highest level whose criteria are fully met.

| Status | Score | Criteria | |--------|-------|----------| | Well Documented ✅ | 3/3 | Dedicated doc page exists with working examples (mdoc blocks present), full API coverage, and cross-references to related features | | Partially Documented 🟡 | 2/3 | Mentioned in docs but incomplete: missing code examples, partial API coverage, or no cross-references | | Stub 🟠 | 1/3 | Doc file exists but is minimal — no examples, fewer than 15 meaningful content lines | | Not Documented 🔴 | 0/3 | No docs found — key symbols absent from entire docs/ tree |

---

Phase 5 — Persist State

After processing all PRs in the batch, write the updated state to .docs-audit-state.json before generating the report.

1. For each processed PR, add an entry to checked_prs with:

• title: PR title
• docs_required: "yes" | "no" | "uncertain" (from Phase 3 classification)
• classification_gate: Gate ID that fired (e.g., "YES-2", "NO-7", "OVERRIDE-BREAKING", "UNCERTAIN")
• classification_reason: Human-readable reason (e.g., "New public Scala file added: src/main/scala/zio/Foo.scala")
• status: Documentation rubric grade from Phase 4, or null if docs_required = "no" (Phase 4 skipped)
• checked_at: Current ISO 8601 timestamp

Use the Write tool to update the file atomically.

Never remove existing entries from checked_prs.

---

Phase 6 — Generate Report and Ask to Continue

Output a markdown-formatted report using the template below.

# PR Documentation Audit — Batch N
Generated: 
Batch: PRs #–# ( checked,  require documentation)
Total checked to date:  PRs across all runs

## This Batch Summary
| Status | Count |
|--------|-------|
| 🔴 Not Documented | N |
| 🟠 Stub | N |
| 🟡 Partially Documented | N |
| ✅ Well Documented | N |
| ⬜ Docs Not Required | N |

## PRs Requiring Documentation

### 🔴 Not Documented
#### #NNN — 
- **Merged:** 
- **Labels:** 
- **Why docs needed:** 
- **Suggested action:** `/docs-document-pr `

### 🟠 Stub
#### #NNN — 
- **Merged:** 
- **Labels:** 
- **Why docs needed:** 
- **Docs found:** 
- **Suggested action:** `/docs-enrich-section `

### ⚠️ Uncertain (Requires Review)
#### #NNN — 
- **Merged:** 
- **Labels:** 
- **Classification:** Gate ID 
- **Ambiguity:** 
- **Suggested action:** Review manually, then run `/docs-document-pr ` if needed

### 🟡 Partially Documented
#### #NNN — 
- **Merged:** 
- **Labels:** 
- **Why docs needed:** 
- **Docs found:** 
- **Suggested action:** `/docs-enrich-section `

### ✅ Well Documented
#### #NNN — 
- **Merged:** 
- **Labels:** 
- **Docs found:** 

---
*PRs with no documentation requirement are excluded from this report.*
*State saved to `.docs-audit-state.json` — already-checked PRs will be skipped next run.*

Sorting: within each status group, list most-recently-merged PRs first.

Uncertain PRs: include in the detailed section with a ⚠️ note explaining the ambiguity (e.g., "Mixed signals: label is refactor but src/main/ files were changed").

After displaying the report, if more unchecked PRs may exist, ask:

> "Processed N PRs in this batch. There may be more unchecked PRs. Continue with the next batch of up to 20?"

If the user confirms, loop back to Phase 1. If not, stop.

---

Implementation Checklist

When you invoke this skill:

• [ ] Phase 0: Read .docs-audit-state.json (or initialize it if missing)
• [ ] Phase 0: Validate that repo in state matches current repo; warn and ask if mismatched
• [ ] Phase 0: Handle --reset argument with explicit user confirmation before clearing state
• [ ] Phase 0: Remind user to add .docs-audit-state.json to .gitignore
• [ ] Phase 1: Detect owner/repo from git remote -v
• [ ] Phase 1: Fetch 40 merged PRs via gh pr list, filter already-checked, take first 20
• [ ] **Phase

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: zio
• Source: zio/zio-skills
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.