Doc Audit

doc-audit

This skill runs a comprehensive, repo-wide documentation truth audit. It's designed for occasional invocation — quarterly, before major refactors, after a long absence from a project — not for routine use. When it runs, it is exhaustive: it spawns many subagents, asks many questions, and produces atomic commits.

The skill operates in 7 sequential phases: Discovery → Claim extraction → Cross-reference → Triage → Resolution (interactive) → Apply (auto-commit on current branch) → CLAUDE.md trim. Each phase has its own subsection in this file.

The skill's "brain" is a cross-reference engine that compares verbatim quotes from docs against deterministic codebase facts and against other docs' verbatim quotes. Every finding shown to the user includes the original quote — never paraphrase. The user is the final arbiter for every discrepancy.

Reference docs in references/ are loaded on demand, not eagerly. Subagent prompts live in prompts/ and are inlined into subagent invocations. Helper scripts in scripts/ handle deterministic data extraction (Haiku-equivalent work without invoking an LLM).

When invoked

When this skill is invoked, do the following in order:

1. Determine the target project.

• Default: the current working directory.
• If --workspace= was passed, treat that path as a parent of multiple repos. Discover all child repos that contain a CLAUDE.md, present a leaderboard ranked by (file_size_bytes / 1024), and ask the user which to audit. Then iterate per repo.

1. Read or create .claude/audit-state/config.yml in the target project.

• If absent, create a default config (see references/move-target-detection.md for the auto-detect logic).
• Honor any user-set values: move_target, trim_target_kb, trim_target_lines, exclude_paths, include_comments, stale_mode.

1. Check working tree state via git status --porcelain.

• If dirty AND not --commit-dirty: warn the user, allow them to abort, otherwise stage-but-don't-commit at Phase 6.
• If on a protected branch (main, master, production) AND not --allow-protected-branch: bail with a warning before Phase 6.

1. Estimate run cost. Count docs (find . -name '*.md' -not -path './node_modules/*' -not -path './.git/*') and structured fact sources. Estimate tokens: #docs × 3000 + #facts × 500. If > 2M, ask the user to confirm.

1. Resolve mode flags:

• --report-only: stop after Phase 4.
• --resume: read existing state files, skip phases whose output is fresher than any source file.
• --include-comments: pass to Phase 2 prompts to expand scope.
• --stale=light|medium|deep: pass to Phase 3 (deep is default).
• --no-commit: skip Phase 6's git commit.
• --workspace=: handled in step 1.

1. Run phases 1-7 sequentially. See each phase's section below.

Orchestration

Phase 1 Discovery       → produces .claude/audit-state/discovery.json
Phase 2 Claim extraction → produces .claude/audit-state/claims.json
Phase 3 Cross-reference  → produces .claude/audit-state/findings.json
Phase 4 Triage           → produces .claude/audit-state/audit-report.md
Phase 5 Resolution       → updates .claude/audit-state/verdicts.json
Phase 6 Apply            → edits files + git commit on current branch
Phase 7 CLAUDE.md trim   → edits CLAUDE.md + new doc files + git commit

Subagent dispatch table:

| Phase | Model | Concurrency | Prompt template | |---|---|---|---| | 1 Discovery | sonnet × 1 | sequential | prompts/discovery.md | | 2 Doc claims | sonnet × N | parallel ≤ 8 | prompts/doc-claim-extraction.md | | 2 Code facts | (script) | parallel | scripts/extract-fact-sources.sh | | 3 Cross-ref | opus × 1 | sequential | prompts/cross-reference.md | | 5 Resolution | parent | sequential | (interactive — see Phase 5 section) | | 7 Trim | opus × 1 | sequential | prompts/trim-classification.md |

Idempotence rule: every phase reads its inputs and writes its outputs in .claude/audit-state/. Phases never write to source files except Phases 6 and 7. This means a --resume is always safe: re-running a phase reproduces the same output given the same inputs.

Verdict persistence: verdicts.json is the only state file that accumulates across runs. Findings have stable IDs (computed by scripts/finding-id.sh from verbatim quotes); when a source file changes, the quote changes, the ID changes, and the finding re-surfaces.

Phase 1 — Discovery

Purpose: catalog all docs and code-fact sources in the project. One subagent does the full scan and returns a structured catalog.

Subagent dispatch:

Spawn a Sonnet subagent with the prompt at prompts/discovery.md. The subagent's prompt is the stable cacheable prefix (the rules for what counts as a doc, classification rubric, output schema). The variable suffix is the project root path.

Inputs: project root path, exclude_paths from config.yml.

Subagent output schema (written to .claude/audit-state/discovery.json):

{
  "scanned_at": "2026-05-03T12:00:00Z",
  "project_root": "/path/to/repo",
  "docs": [
    { "path": "CLAUDE.md", "size_bytes": 34447, "classification": "claude-md" },
    { "path": "README.md", "size_bytes": 2104, "classification": "readme" },
    { "path": "narrow-scope-docs/deployment.md", "size_bytes": 8421, "classification": "runbook" }
  ],
  "fact_sources": [
    { "path": "package.json", "category": "package-manifest" },
    { "path": "pnpm-workspace.yaml", "category": "monorepo-shape" }
  ],
  "doc_folders": ["narrow-scope-docs/", "docs/"],
  "existing_doc_folder": "narrow-scope-docs/"
}

Failure handling: if the subagent fails or returns malformed JSON, retry once with a "your previous output was malformed; here it was: ; produce valid JSON" prompt suffix. If it fails twice, abort with a clear error.

Phase 2 — Claim extraction

Purpose: for each doc, extract every checkable claim with a verbatim quote. For each fact source, extract the deterministic facts.

2a — Doc claims (parallel Sonnet subagents)

For each entry in discovery.json#docs, spawn a Sonnet subagent with the prompt at prompts/doc-claim-extraction.md. Cap concurrency at 8 in flight. As each subagent completes, append its output to .claude/audit-state/claims.json#doc_claims.

Each claim has the schema:

{
  "doc_path": "CLAUDE.md",
  "doc_line": 42,
  "verbatim_quote": "Hono 4 + @hono/zod-openapi",
  "claim_type": "version",
  "subject": "hono",
  "value": "4"
}

claim_type is one of: path, version, count, port, command, service-id, architecture, behavior, cross-doc-reference.

2b — Code facts (deterministic, no LLM)

Run scripts/extract-fact-sources.sh . The script reads each entry in discovery.json#fact_sources and writes .claude/audit-state/claims.json#code_facts:

{
  "source_path": "package.json",
  "facts": [
    { "fact_type": "version", "subject": "hono", "value": "4.6.10" },
    { "fact_type": "script", "subject": "dev", "value": "vite" }
  ]
}

Idempotence: before invoking each subagent, check if claims.json#doc_claims already has entries for that doc_path AND if the doc's mtime is older than last-run.json#completed_at. If so, skip — already extracted.

Phase 3 — Cross-reference

Purpose: build the discrepancy graph by comparing claims against facts and against each other.

Subagent dispatch: spawn one Opus subagent with prompts/cross-reference.md. Provide it with the full claims.json and the rubrics from references/severity-rubric.md, references/finding-types.md, and references/confidence-rubric.md inlined into the prompt.

Stale modes (controlled by --stale=light|medium|deep, default deep):

• light: only direct path/version/count/port/command/service-id matches
• medium: light + cross-doc consistency checks (when two docs make claims about the same subject)
• deep: medium + LLM-inferred claims — the subagent re-reads each doc and identifies sentences making verifiable claims that didn't fit a structured claim_type

Output schema (written to .claude/audit-state/findings.json):

{
  "generated_at": "2026-05-03T12:30:00Z",
  "findings": [
    {
      "id": "abc123def456",
      "severity": "P0",
      "confidence": "high",
      "type": "drift",
      "subject": "hono version",
      "sources": [
        { "kind": "doc", "path": "CLAUDE.md", "line": 42, "quote": "Hono 4 + @hono/zod-openapi" },
        { "kind": "code-fact", "path": "package.json", "value": "hono@4.6.10" }
      ]
    }
  ]
}

id is computed by scripts/finding-id.sh from the concatenation of all sources[*].quote || sources[*].value. This makes the ID stable across re-runs as long as the underlying source text doesn't change.

type is one of: drift, contradiction, orphan-doc, orphan-code, duplication, bloat. See references/finding-types.md.

Phase 4 — Triage

Purpose: rank findings into a deterministic walkthrough order, write a human-readable report.

Logic (no subagent, parent does this):

1. Filter out findings whose id exists in verdicts.json AND whose applies_until_source_changes is true AND whose source text is unchanged.
2. Sort remaining findings by (severity, type, confidence) where:

• severity: P0

Generated: Total findings: (P0: , P1: , P2: ) Existing verdicts applied: findings auto-resolved

P0 (load-bearing — CLAUDE.md and high-traffic docs)

Drift ( findings)

[#1] hono version mismatch — high confidence

• CLAUDE.md:42 says: "Hono 4 + @hono/zod-openapi"
• package.json has: hono@4.6.10

[continued for each finding...]

If `--report-only` was passed, the skill stops here and tells the user where to find the report.
## Phase 5 — Resolution

**Purpose:** walk the user through unresolved findings and capture verdicts.

**No subagent.** The parent agent runs this interactively with the user.

**Walkthrough order:** the order findings appear in `audit-report.md` (P0 → P1 → P2, drift first within each).

**Per-finding interaction template:**

[ ] — confidence Source A: : quote: "" Source B: value:

What do you want to do? 1) Update to match (recommended: ) 2) Update the other side to match 3) Make the doc statement vague: 4) Skip — intentional discrepancy 5) Custom edit (you specify the exact change) 6) q — quit walkthrough, save progress

**The "recommended" choice is determined by:**

- For `drift`: usually update the doc to match code (code is the source of truth). Exception: if the doc is in the `move_target` folder (canonical docs), code might be the wrong one — flag it as ambiguous and ask without recommending.
- For `contradiction`: ask which doc is canonical; recommend updating the non-canonical one.
- For `orphan-doc`: ask whether the feature exists (recommendation: delete the doc claim) or is aspirational (recommendation: tag it with "aspirational" and skip).
- For `orphan-code`: recommend adding doc anchor to the most-relevant detected doc folder.
- For `duplication`: recommend keeping the canonical instance and replacing duplicates with a `see ` pointer.
- For `bloat`: not seen in Phase 5 (handled in Phase 7 trim).

**Verdict persistence:** after each user choice, append to `verdicts.json` immediately:

```json
{
  "": {
    "decision": "update-doc",
    "decided_at": "2026-05-03T13:00:00Z",
    "details": { "target": "doc-side", "applied": false },
    "applies_until_source_changes": true
  }
}

details.applied: false means the actual edit hasn't happened yet — that's Phase 6.

Quit handling: if the user picks q (or sends EOF), save what's accumulated, write a clear "resumable" note to last-run.json, and exit. --resume will pick up from the next unresolved finding.

Phase 6 — Apply

Purpose: turn verdicts into actual file edits and commit them atomically.

No subagent. The parent applies edits sequentially.

Logic:

1. Group verdicts by file path. For each file, build the full set of edits.
2. For each file, read current content (Read tool), apply edits in reverse line order to avoid invalidating later edit positions, write the result (Write/Edit tool).
3. Run git status to verify expected files are dirty.
4. If --no-commit was passed: stop. Print summary, list staged-but-uncommitted files.
5. Otherwise: git add then git commit -m '' with the message format below.
6. Update verdicts.json#.details.applied = true for every applied verdict.

Commit message format:

docs(audit): align  findings

 drift fixes
 contradiction fixes
 orphan fixes
 duplication fixes

Files touched: 
Audit run: .claude/audit-state/last-run.json

Working tree safety:

• Before applying, if git status shows pre-existing dirty files NOT in our edit set: refuse to commit (don't sweep up unrelated changes). Tell the user: stash or commit those first, then re-run with --resume.
• On protected branch with no --allow-protected-branch: refuse to commit, fall back to --no-commit behavior.

On failure: if git commit fails (hook rejection, sign-off issue), tell the user the exact error, leave the working tree as-is, and document next steps.

Phase 7 — CLAUDE.md trim

Purpose: with docs now aligned, identify CLAUDE.md sections that are duplicates of canonical docs (delete + pointer) or long-form policy (move to a new file in the doc folder). Trim until target size is hit.

Subagent dispatch: one Opus subagent with prompts/trim-classification.md. Provide it the full CLAUDE.md content, the list of all other docs (paths only, not content — let it ask for content if it needs to verify a duplicate), and the trim targets from config.yml.

Subagent output schema (.claude/audit-state/trim-plan.json):

{
  "current_size_kb": 34.4,
  "current_lines": 420,
  "target_size_kb": 8,
  "target_lines": 200,
  "decisions": [
    {
      "claude_md_section_lines": [40, 87],
      "section_summary": "Hard rule: every change updates narrow-scope-docs",
      "verdict": "STAY",
      "reason": "Load-bearing process rule, brief enough"
    },
    {
      "claude_md_section_lines": [200, 280],
      "section_summary": "Documentation Update Rules — full 6-step flow with detailed reasoning",
      "verdict": "MOVE",
      "move_to": "narrow-scope-docs/documentation-update-rules.md",
      "claude_md_replacement": "## Documentation Update Rules\n\nSee `narrow-scope-docs/documentation-update-rules.md`."
    },
    {
      "claude_md_section_lines": [350, 380],
      "section_summary": "Past-incident anecdote about recovery branch r1a-bitemporal-foundations",
      "verdict": "DELETE",
      "reason": "Anecdote belongs in changelog, not CLAUDE.md"
    }
  ]
}

verdict is STAY, MOVE, DELETE, or DUPLICATE_OF (with a duplicate_of_path field — replace with pointer).

User review: present the trim plan to the user as a structured menu (similar to Phase 5 walkthrough but per-section). They can accept/reject/modify each decision.

Apply:

1. For each MOVE verdict: write the section content to move_to, create the file with a header # (moved from CLAUDE.md, ). Replace the section in CLAUDE.md with the pointer.
2. For each DELETE verdict: remove the section from CLAUDE.md.
3. For each DUPLICATE_OF verdict: replace with See \\..
4. For each STAY: no action.
5. Verify CLAUDE.md size after edits ≤ trim_target_kb and line count ≤ trim_target_lines. If still over, surface remaining sections to the user with "still over budget — pick one to move/delete?" until target hit.
6. Auto-commit on current branch (same rules as Phase 6) with message:

docs(audit): trim CLAUDE.md from KB to KB (%)

 sections moved to 
 sections deleted (covered by canonical docs)
 duplicate sections replaced with pointers

Audit run: .claude/audit-state/last-run.json

Mode flags

|

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: MahmoudKhaledd
• Source: MahmoudKhaledd/claude-skill-doc-audit
• License: MIT

Install and usage instructions live in the source repository linked above.