Auditing Skills

Evaluate SKILL.md files against best practices for structure, conciseness, progressive disclosure, and effectiveness. Provide actionable findings with contextual judgment, not arbitrary scores.

To audit a skill:

1. Read reference documentation from creating-skills/
2. Read the target skill's SKILL.md and any subdirectories
3. Evaluate against best practices (YAML, Structure, Content, Anti-patterns)
4. Generate report with findings categorized by severity

• NEVER modify files during audit - ONLY analyze and report findings
• MUST read all reference documentation before evaluating
• ALWAYS provide file:line locations for every finding
• DO NOT generate fixes unless explicitly requested by the user
• NEVER make assumptions about skill intent - flag ambiguities as findings
• MUST complete all evaluation areas (YAML, Structure, Content, Anti-patterns)
• ALWAYS apply contextual judgment - what matters for a simple skill differs from a complex one

During audits, prioritize evaluation of:

• YAML compliance (name length, description quality, directive style with negative constraint)
• Pure XML structure (required tags, no markdown headings in body, proper nesting)
• Progressive disclosure structure (SKILL.md

MANDATORY: Read best practices FIRST, before auditing:

1. Locate the creating-skills skill and its references:

• Use Glob: .claude/plugins/cache/**/creating-skills/SKILL.md
• Then read its SKILL.md for overview

1. Read creating-skills/references/use-xml-tags.md for required/conditional tags, XML structure requirements
2. Read creating-skills/references/skill-patterns.md for YAML, naming, progressive disclosure patterns
3. Read creating-skills/references/core-principles.md for XML structure, conciseness, context window principles
4. Handle edge cases:

• If reference files are missing or unreadable, note in findings under "Configuration Issues" and proceed with available content
• If YAML frontmatter is malformed, flag as critical issue
• If skill references external files that don't exist, flag as critical issue and recommend fixing broken references
• If skill is

Check for:

• name: Lowercase-with-hyphens, max 64 chars, matches directory name, follows verb-noun convention (create-, manage-, setup-, generate-)
• description: Max 1024 chars, directive style (ALWAYS invoke + NEVER without), no XML tags

Check for:

• Progressive disclosure: SKILL.md is overview (

Check for:

• Conciseness: Only context Claude doesn't have. Apply critical test: "Does removing this reduce effectiveness?"
• Clarity: Direct, specific instructions without analogies or motivational prose
• Specificity: Matches degrees of freedom to task fragility
• Examples: Concrete, minimal, directly applicable

Check whether the skill provides operational wisdom, not just procedural steps:

Success Criteria Verifiability:

• Are success criteria concrete and testable? (commands to run, thresholds to check)
• Can you answer "did I succeed?" with a boolean, not a judgment call?
• ❌ Bad: "Task complete when migration is done"
• ✅ Good: "Coverage on src/foo.ts must be ≥86%. Run: pnpm test --coverage | grep foo.ts"

Verification Gates:

• Are there explicit "STOP and verify before proceeding" checkpoints?
• Do gates have pass/fail criteria with specific commands?
• ❌ Bad: "Verify coverage matches before removing legacy tests"
• ✅ Good: "GATE 2: Run pnpm test --coverage for both legacy and SPX. If delta >0.5%, STOP."

Failure Modes Documentation:

• Does the skill document what can go wrong in practice?
• Are failures from actual usage, not hypotheticals?
• Does each failure have: what happened, why it failed, how to avoid?
• ❌ Bad: No failure modes section
• ✅ Good: "Failure 1: Agent compared coverage per-story instead of per-file. Why: Multiple stories share one legacy file. Avoid: Always compare at legacy file level."

Example Concreteness:

• Do examples show real outputs with actual values?
• Can you compare your output to the example to verify correctness?
• ❌ Bad: "Coverage should match between legacy and SPX tests"
• ✅ Good: "Legacy: 24 tests, 86.3% on state.ts. SPX: 24 tests, 86.3% on state.ts. ✓ Match"

Procedural vs Operational Balance:

• Procedural = HOW to do steps
• Operational = how to KNOW you did it right
• Skills need both; flag if heavily imbalanced toward procedural

Flag these issues:

• markdownheadingsin_body: Using markdown headings (##, ###) in skill body instead of pure XML
• missingrequiredtags: Missing objective, quickstart, or successcriteria
• hybridxmlmarkdown: Mixing XML tags with markdown headings in body
• unclosedxmltags: XML tags not properly closed
• vague_descriptions: "helps with", "processes data"
• passive_description: Uses passive "Use when" instead of directive "ALWAYS invoke... NEVER X without this skill"
• toomanyoptions: Multiple options without clear default
• deeplynestedreferences: References more than one level deep from SKILL.md
• windows_paths: Backslash paths instead of forward slashes
• bloat: Obvious explanations, redundant content
• unverifiablesuccesscriteria: Success criteria that can't be tested with a command or boolean check
• noverificationgates: Complex multi-step skill without explicit stop-and-check points
• nofailuremodes: Skill lacks documentation of what went wrong in practice
• abstract_examples: Examples that show patterns but not concrete values/outputs

Apply judgment based on skill complexity and purpose:

Simple skills (single task,

Some skills were created before pure XML structure became the standard. When auditing legacy skills:

• Flag markdown headings as critical issues for SKILL.md
• Include migration guidance in findings: "This skill predates the pure XML standard. Migrate by converting markdown headings to semantic XML tags."
• Provide specific migration examples in the findings
• Don't be more lenient just because it's legacy - the standard applies to all skills
• Suggest incremental migration if the skill is large: SKILL.md first, then references

Migration pattern:

## Quick start → 
## Workflow → 
## Success criteria → 

Reference files in the references/ directory should also use pure XML structure (no markdown headings in body). However, be proportionate with reference files:

• If reference files use markdown headings, flag as recommendation (not critical) since they're secondary to SKILL.md
• Still recommend migration to pure XML
• Reference files should still be readable and well-structured
• Table of contents in reference files over 100 lines is acceptable

Priority: Fix SKILL.md first, then reference files.

What to flag as XML structure violations:

❌ Flag as critical:

## Quick start

Extract text with pdfplumber...

## Advanced features

Form filling...

````
✅ Should be:
```xml

Extract text with pdfplumber...

Form filling...

````

**Why**: Markdown headings in body is a critical anti-pattern. Pure XML structure required.

❌ Flag as critical:
```xml

1. Do step one
2. Do step two

Missing: `, , `

✅ Should have all three required tags:

What the skill does and why it matters

Immediate actionable guidance

How to know it worked

Why: Required tags are non-negotiable for all skills.

❌ Flag as critical:

PDF processing capabilities

## Quick start

Extract text...

## Advanced features

Form filling...

````
✅ Should be pure XML:
```xml

PDF processing capabilities

Extract text...

Form filling...

````

**Why**: Mixing XML with markdown headings creates inconsistent structure.

❌ Flag as critical:
```xml

Process PDF files

Use pdfplumber...

````
Missing closing tag: ``

✅ Should properly close all tags:
```xml

Process PDF files

Use pdfplumber...

````

**Why**: Unclosed tags break parsing and create ambiguous boundaries.

Flag when conditional tags don't match complexity:

**Over-engineered simple skill** (flag as recommendation):

```xml
Convert CSV to JSON
Use pandas.to_json()
CSV files are common...
Step 1... Step 2...
See [advanced.md]
Validate input...
Test with all models...

Why: Simple single-domain skill only needs required tags. Too many conditional tags add unnecessary complexity.

Under-specified complex skill (flag as critical):

Manage payment processing with Stripe API
Create checkout session
Payment completed

Why: Payment processing needs security_checklist, validation, error handling patterns. Missing critical conditional tags.

Examples of operational effectiveness issues to flag:

❌ Flag as critical for complex skills:

Task is complete when:
- All stories have SPX tests
- Coverage verified
- Legacy tests removed

Why it fails: "Coverage verified" is not testable. Verified how? What threshold? What command?

✅ Should be:

````xml

Task is complete when:

• All stories have SPX tests (verify: ls spx/.../tests/*.test.ts returns files for each story)
• Coverage parity confirmed (verify: both commands below show same % for target files)

```bash pnpm vitest run tests/legacy/... --coverage | grep "target.ts" pnpm vitest run spx/.../tests --coverage | grep "target.ts"

````

• Legacy tests removed via git rm (verify: git status shows deletions staged)

Threshold: Coverage delta must be ≤0.5%. If larger, STOP and identify missing tests.

```` Why it works: Every criterion has a verification command and a pass/fail threshold.

❌ Flag as critical for multi-step skills:

1. Read DONE.md files from worktree
2. Create SPX tests matching DONE.md entries
3. Verify coverage matches
4. Remove legacy tests with git rm
5. Create SPX-MIGRATION.md

````

**Why it fails**: No stop points. Agent could remove legacy tests before verifying coverage.

✅ Should be:

```xml
1. Read DONE.md files from worktree
2. Create SPX tests matching DONE.md entries

**GATE 1**: Before proceeding, verify:
- [ ] SPX test count matches DONE.md entry count
- [ ] All SPX tests pass: `pnpm vitest run spx/.../tests`
If gate fails, fix tests before continuing.

3. Verify coverage matches (run both, compare percentages)
4. Remove legacy tests with git rm

**GATE 2**: Before committing, verify:
- [ ] `pnpm test` passes
- [ ] `git status` shows only expected changes
If gate fails, do not commit.

5. Create SPX-MIGRATION.md

Why it works: Explicit gates prevent proceeding with broken state.

❌ Flag as recommendation for complex skills: Skill has detailed workflow but no `` section.

Why it matters: Agents will make the same mistakes that previous agents made. Failure modes capture hard-won operational knowledge.

✅ Should include:

Failures from actual usage:

**Failure 1: Compared coverage at wrong granularity**
- What happened: Agent saw 39% coverage for one story and stopped, thinking migration failed
- Why it failed: Multiple stories share one legacy file; per-story coverage is meaningless
- How to avoid: ALWAYS compare at legacy file level, not story level

**Failure 2: Removed shared legacy file too early**
- What happened: Agent removed tests/integration/cli.test.ts after migrating story-32
- Why it failed: Stories 43 and 54 also contributed tests to that file
- How to avoid: Build legacy_file → [stories] map BEFORE migration. Only remove after ALL contributing stories migrated.

Why it works: Future agents learn from past mistakes without repeating them.

❌ Flag as recommendation:

Coverage should match between legacy and SPX tests.

Why it fails: What does "match" mean? What numbers? How do I compare my output?

✅ Should be:

Coverage must match. Concrete example from actual migration:

Legacy tests:
  tests/unit/status/state.test.ts (5 tests)
  tests/integration/status/state.integration.test.ts (19 tests)
  Total: 24 tests, 86.3% coverage on src/status/state.ts

SPX tests:
  spx/.../21-initial.story/tests/state.unit.test.ts (5 tests)
  spx/.../32-transitions.story/tests/state.integration.test.ts (7 tests)
  spx/.../43-concurrent.story/tests/state.integration.test.ts (4 tests)
  spx/.../54-edge-cases.story/tests/state.integration.test.ts (8 tests)
  Total: 24 tests, 86.3% coverage on src/status/state.ts

Verdict: ✓ Test counts match (24=24), coverage matches (86.3%=86.3%)

Why it works: Agent can compare their actual output to the example and know if they succeeded.

❌ Flag as critical for complex skills: Skill has detailed `` (450 lines of steps) but:

• `` is 3 lines of vague statements
• No ``
• No ``

Pattern: Heavy procedural, light operational = agents know HOW but not WHETHER they succeeded.

Why it matters: This is the most common skill failure mode. The skill tells you what to do but not how to verify you did it right. Agents follow steps, produce wrong output, and don't realize it.

✅ Balanced skill has roughly equal investment in:

• Procedural content (workflow, steps, commands)
• Operational content (success criteria, verification gates, failure modes, concrete examples)

Audit reports use severity-based findings, not scores. Generate output using this markdown template:

## Audit Results: [skill-name]

### Assessment

[1-2 sentence overall assessment: Is this skill fit for purpose? What's the main takeaway?]

### Critical Issues

Issues that hurt effectiveness or violate required patterns:

1. **[Issue category]** (file:line)
   - Current: [What exists now]
   - Should be: [What it should be]
   - Why it matters: [Specific impact on this skill's effectiveness]
   - Fix: [Specific action to take]

2. ...

(If none: "No critical issues found.")

### Recommendations

Improvements that would make this skill better:

1. **[Issue category]** (file:line)
   - Current: [What exists now]
   - Recommendation: [What to change]
   - Benefit: [How this improves the skill]

2. ...

(If none: "No recommendations - skill follows best practices well.")

### Strengths

What's working well (keep these):

- [Specific strength with location]
- ...

### Quick Fixes

Minor issues easily resolved:

1. [Issue] at file:line → [One-line fix]
2. ...

### Context

- Skill type: [simple/complex/delegation/etc.]
- Line count: [number]
- Estimated effort to address issues: [low/medium/high]

Note: While this subagent uses pure XML structure, it generates markdown output for human readability.

Task is complete when:

• All reference documentation files have been read and incorporated
• All evaluation areas assessed (YAML, Structure, Content, Anti-patterns, Operational Effectiveness)
• Contextual judgment applied based on skill type and complexity
• Findings categorized by severity (Critical, Recommendations, Quick Fixes)
• At least 3 specific findings provided with file:line locations (or explicit note that skill is well-formed)
• Assessment provides clear, actionable guidance
• Strengths documented (what's working well)
• Context section includes skill type and effort estimate
• Next-step options presented to reduce user cognitive load
• For complex/migration skills: Explicitly evaluated verifiable success criteria, verification gates, and failure modes

Before presenting audit findings, verify:

Completeness checks:

• [ ] All evaluation areas assessed (including operational effectiveness)
• [ ] Findings have file:line locations
• [ ] Assessment section provides clear summary
• [ ] Strengths identified

Accuracy checks:

• [ ] All line numbers verified against actual file
• [ ] Recommendations match skill complexity level
• [ ] Context appropriately considered (simple vs complex skill)
• [ ] Operational effectiveness evaluated proportionally (critical for complex/migration skills)

Quality checks:

• [ ] Findings are specific and actionable
• [ ] "Why it matters" explains impact for THIS skill
• [ ] Remediation steps are clear
• [ ] No arbitrary rules applied without contextual

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: outcomeeng
• Source: outcomeeng/claude
• License: MIT

Install and usage instructions live in the source repository linked above.