Documentation Audit

/documentation-audit

Audit a TypeScript project's documentation against an opinionated baseline organised in four layers — project entry and onboarding, architectural and decision documentation, code-level documentation, operational documentation and drift — preceded by a diagnostic snapshot. Then offer to generate an implementation plan for the gaps.

The default mental model is a TypeScript and React application, but most checks apply equally to any TypeScript project (libraries, services, monorepos). The operational layer adapts to the project shape — library-only projects silently skip the deployment, rollback, and monitoring checks while still running drift detection.

How this differs from neighbouring audits

| Concern | Owner | | --- | --- | | Whether feature folders have a public-API doc comment or README.md | /architecture-audit (Layer 4 "public API documented") and /documentation-audit (Layer 2 "per-feature documentation") — both surface so a single fix passes both | | Whether tests describe user-visible behaviour | /testing-audit (its own concern; not documentation) | | Whether commit messages follow Conventional Commits | /quality-gates-audit | | Whether the README explains the project and gets a newcomer running | /documentation-audit | | Whether ADRs exist, are templated, and are kept current | /documentation-audit | | Whether public APIs carry TSDoc or JSDoc | /documentation-audit | | Whether runbooks, deployment docs, and rollback procedures exist | /documentation-audit | | Drift between docs and code (README scripts vs package.json, doc mtime vs code mtime, dead links) | /documentation-audit |

Static-first design with optional link-check enrichment

This skill is read-only. Two modes:

• Static (default). Read every Markdown file in the repository (.md, .mdx, excluding node_modules and ignored paths), package.json, source files (for TSDoc/JSDoc and TODO detection), and Git metadata for last-modified timestamps. Internal and relative links are always checked against the file system regardless of mode — no network needed for that.
• Static plus opt-in --with-link-check. Additionally HEAD-request external links found in documentation files to verify they resolve (HTTP 2xx or 3xx). The skill never POSTs, never authenticates, never follows redirects beyond the HEAD response. Timeouts and transient failures are tolerated; persistent failures across the run are reported.

The skill never modifies any documentation, source, or configuration file.

Usage

/documentation-audit                                # default: concise Top 5 + full report saved + ask about plan
/documentation-audit --worktree                          # create an isolated Git worktree, then run the audit there
/documentation-audit --learn                        # mid-level engineer teaching mode (detailed explanations + file/line examples)
/documentation-audit --teach                        # alias for --learn
/documentation-audit --with-link-check              # static plus HEAD-checks of external URLs
/documentation-audit --threshold-readme-min-lines=50        # override default 30
/documentation-audit --threshold-adr-staleness-days=365     # override default 180
/documentation-audit --threshold-feature-doc-coverage=70    # override default 50 (percent for partial)
/documentation-audit --threshold-public-api-doc-coverage=80 # override default 60 (percent for present)
/documentation-audit --threshold-todo-staleness-days=180    # override default 90
/documentation-audit --threshold-doc-staleness-days=365     # override default 180

💡 Pro tip: Add --worktree to run this audit in an isolated Git worktree.

The skill never accepts --apply. The implementation plan is descriptive Markdown.

The opinionated baseline

A check resolves to one of four statuses:

• present — the invariant holds.
• partial — most signals resolve, with a small number of exceptions, or the codebase shows mixed adherence to a soft check.
• missing — a structural prerequisite is absent (no README at all, for example).
• violation — the audit identified concrete documentation or drift that breaks the invariant.

Layer 0 is informational only and has no status. Layer 4's operational checks (deployment, rollback, monitoring, feature flags) are silently skipped when the project is library-only or no deployable target is detected.

Layer 0 — Diagnostic snapshot (always written, no pass/fail)

• Total documentation file count (.md and .mdx files, excluding node_modules and gitignored paths).
• Documentation-to-code line ratio (Markdown lines vs .ts plus .tsx lines).
• README presence and size in lines.
• LICENSE presence and detected SPDX identifier.
• ADR count and the most-recent ADR date.
• TODO/FIXME count, with breakdown by age bucket: less than 30 days, 30–90 days, more than 90 days.
• Detected Storybook (@storybook/* in devDependencies plus .storybook/ directory).
• Detected documentation-site framework: Docusaurus, Nextra, VitePress, MkDocs, Sphinx, custom, or none.
• Per-feature README coverage rate: number of feature folders with a README.md (or barrel-file documentation comment) divided by total feature folder count.
• External-link total count and, when --with-link-check is set, the count of unreachable external links.
• Project shape (informs the operational layer): deployable application, library, monorepo, or hybrid.

Layer 1 — Project entry and onboarding

| Check | Expectation | Violation signal | | --- | --- | --- | | README present | A README.md (or README.mdx) exists at the repository root. | No README at all. | | README is substantive | The README is at least the threshold lines (default 30; tunable via --threshold-readme-min-lines) and contains identifiable content beyond the project name and a logo. | README below the threshold. | | README covers onboarding essentials | The README contains sections (or clearly-headed prose) for: what the project is, prerequisites, install, run development server, run tests, build for production. The audit looks for header text and command blocks matching common patterns. Soft check — reported as partial when some essentials are present and others are missing. | None of the essentials present. | | README setup instructions match package.json scripts | Commands shown in the README install/run/test sections (npm run , pnpm , yarn , bun ) reference scripts that actually exist in package.json. | A README command references a script that no longer exists. | | LICENSE present | A LICENSE (or LICENSE.md/LICENSE.txt) file exists at the root with a recognisable license. | No license file. | | CONTRIBUTING.md present | When the project appears to accept external contributions (signals: presence of .github/PULL_REQUEST_TEMPLATE.md, public-repository markers, an explicit notice in the README), a CONTRIBUTING.md exists. Soft check — reported as partial when absent in a project that appears to accept contributions. | Public-facing project with no contributing guide. | | Required tool versions documented | One or more of .nvmrc, .tool-versions, package.json engines.node, package.json volta, or mise.toml is present. | None present. | | .env.example (or equivalent) present | When the project reads environment variables, a .env.example (or .env.sample, .env.template) documents the variables required to run the project. The example file does not contain real secrets — only placeholder values. | Project reads environment variables but no example file documents them. |

Layer 2 — Architectural and decision documentation

| Check | Expectation | Violation signal | | --- | --- | --- | | Architecture overview present | A top-level architecture document exists at one of the conventional paths: ARCHITECTURE.md, docs/architecture.md, docs/architecture/index.md. | No architecture overview at any conventional path. | | Architecture Decision Records present | An ADR directory exists (docs/adr/, docs/decisions/, adr/) with at least one ADR. Soft check — reported as partial when the directory exists but is empty. | No ADR infrastructure at all. | | Recent ADR activity | When ADRs are present, at least one was added or modified within the threshold (default 180 days; tunable via --threshold-adr-staleness-days). Long stretches with no ADRs in an active codebase suggest important decisions are being made undocumented. Soft check — reported as partial. | All ADRs older than the threshold in a project with recent code activity. | | ADRs follow a recognisable template | ADRs include identifiable sections for Status, Context, Decision, and Consequences (or the project's chosen template — the audit looks for header consistency across files). Soft check. | ADRs without a consistent structure. | | Per-feature documentation | Each feature folder (under src/features/, src/modules/, or the equivalent for the detected pattern) has a README.md or a documenting comment block at the top of its barrel file. Threshold for partial vs violation is the per-feature coverage rate (default partial ≥ 50%, violation "Generate an implementation plan for the documentation gaps? (yes/no)"

On yes, writes .architect-audits/documentation-audit/implementation-plan.md describing exactly which documents to create, which sections to add to existing docs, which TODOs to address (or close), which links to fix, and which TSDoc/JSDoc blocks to write — ordered by audience: onboarding documentation first (newcomer experience matters most), then architectural and operational, then code-level, then drift cleanup. The plan does not modify any project files.

On no, exits cleanly.

Implementation steps

Step 1 — Confirm the prerequisites

test -f package.json || { echo "documentation-audit: no package.json detected. This skill currently supports Node.js projects only."; exit 1; }

Step 2 — Detect project shape

Inspect the repository for shape signals:

• Deployable application: next.config.*, vercel.json, netlify.toml, Dockerfile, app/ or pages/ directory, src/main.tsx plus a build script, server entry points.
• Library: package.json with main/module/exports fields and no application entry; absence of deployment configuration.
• Monorepo: packages/ directory plus a workspace declaration in package.json or pnpm-workspace.yaml.
• Hybrid: signals from multiple categories.

Record the detected shape in metadata.json. The shape gates layer 4's operational checks.

Step 3 — Detect observability and feature-flag stacks

Scan dependencies for @sentry/*, @datadog/*, @grafana/* (where applicable), launchdarkly, @growthbook/*, statsig-js, configcat-js, unleash-client. Record matches in metadata.json. The detection seeds the feature-flag-documentation check and the monitoring-documentation check.

Step 4 — Walk documentation files

Enumerate .md and .mdx files outside node_modules and .gitignore. For each:

• Total lines.
• Last-modified timestamp from Git (the most recent commit touching the file).
• Heading structure.
• Fenced code blocks (capture commands inside).
• Link references: classify as internal (relative path or anchor) or external (absolute URL).

Special handling for README.md: extract command blocks for cross-referencing against package.json scripts. Special handling for ADRs: detect the ADR directory and parse each file for the template-section headers.

Step 5 — Walk source files

Enumerate .ts and .tsx files. For each:

• Identify exported public-API symbols (functions, hooks, components, types) and whether they carry a TSDoc/JSDoc block.
• Find TODO/FIXME comments. For each, run git blame to get the most recent modification date.
• Find blocks of commented-out code (multiple consecutive comment lines that parse as TypeScript when uncommented).
• Find empty /** */ blocks.

Use Graphify communities to sample broadly when present; otherwise sweep all source files.

Step 6 — Optionally HEAD-check external links

When --with-link-check is set, collect the unique set of external URLs from step 4 and issue HEAD requests:

• Timeout per URL: 5 seconds.
• Concurrency: at most 10 in flight.
• Acceptable: HTTP 2xx, 3xx (without following redirects).
• Tolerate transient failures: retry once, then mark as unreachable.

Record per-URL status in metadata.

Step 7 — Build the diagnostic snapshot

Aggregate the collected data into the items listed in Layer 0. Write snapshot.md and prepend the same content to findings.md.

Step 8 — Resolve each check

For each check in the active layer list, walk its detection logic. Layer 4's operational checks are skipped silently when the detected project shape is library-only with no deployable signal. Threshold-bearing checks compare aggregated values to the configured thresholds.

For each check, record evidence and up to ten representative samples plus a total count.

Step 9 — Write phase 1 outputs

Create .architect-audits/documentation-audit/ if needed. Write findings.md, findings.json, snapshot.md, metadata.json. Overwrite previous runs of these four; preserve implementation-plan.md unless the user agrees to regenerate it.

Step 10 — Print the concise chat summary and offer phase 2

Print a human-first, scannable summary in the chat. Do not print the full layered findings — those are written to disk in Step 9. The chat output has exactly this shape:

1. Short header — audit name, timestamp, and a one-line summary of the codebase state.
2. Top 5 Highest-Leverage Recommendations — ordered by architectural principles: test philosophy, maintainability, risk reduction, velocity, long-term health. For fewer than five findings, print what exists. For each recommendation (numbered 1–5):

• Title (one clear line).
• Why it matters (explain the principle in 1–2 sentences).
• Real consequences if ignored (honest downside for the team or project).
• Smallest high-leverage fix (exact next step, effort level, and which files to touch).
• At the end, add a lettered sub-list of concrete actions if useful (e.g. 2a, 2b) so the user can reply with "2b" or "1 and 3" to trigger implementation.

1. Bottom line: Full detailed audit report (layered findings, snapshot, metadata, implementation plan) → .architect-audits/documentation-audit/findings.md

When --learn or --teach is set, expand each recommendation into mid-level engineer teaching mode:

• For every item, explain as if teaching a mid-level engineer, pointing to specific files and line numbers from the current codebase.
• Use educational language: "Here's why this pattern bites teams in the long run…", "This is the exact mistake I see in most codebases at your stage…", "The fix is small but pays off huge because…".
• Include a short "What you'll learn from fixing this" section for each recommendation.
• Keep the numbered/lettered structure so the user can still reply with "2b" or "1 and 3".
• End with the same bottom-line link to the full report.

After printing, ask the single yes-or-no question: "Generate an implementation plan for the gaps identified above? (yes/no)" Do not proceed to phase 2 without an explicit affirmative.

Step 11 — Phase 2: generate the implementation plan

When the user agrees, build implementation-plan.md, ordered by audience:

1. Header — repository name, baseline version, project shape, observability stack, feature-flag stack, timestamp, total counts per layer.
2. Onboarding fixes (highest priority) — README expansion or rewrite, missing essentials, LICENSE addition, .env.example scaffolding, tool-version pin file. Per finding, the file to add or section to write, with a starter template.
3. **Architectural and decision-document

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: BenSheridanEdwards
• Source: BenSheridanEdwards/ArchitectPlaybook
• License: MIT

Install and usage instructions live in the source repository linked above.