Eval Skills

Skill Evaluator

Discover all skills in the project, score them across 6 criteria, and infer the appropriate effort level based on content analysis.

When to Use

• New project: run once to establish baseline quality
• Before committing a skill to a team repo
• After bulk-importing skills from another project
• When adding effort fields for the first time
• When a skill doesn't auto-trigger and you want to diagnose why

What Gets Audited

All SKILL.md files and flat .md files found in:

• .claude/skills/**
• ~/.claude/skills/** (if requested)
• .claude/commands/** (legacy flat files, still valid)
• Any path passed as argument: /eval-skills ./my-skills-dir

---

Valid Frontmatter Fields

Claude Code skills follow the agentskills.io open standard, extended with Claude Code-specific fields. Flag any field not in this table as unsupported.

agentskills.io spec fields

| Field | Required | Notes | |-------|----------|-------| | name | No | Display label shown in skill lists. The command name always comes from the directory name, not this field. | | description | Recommended | Combined with when_to_use, truncated at 1,536 chars in context. First paragraph used if omitted. | | when_to_use | No | Additional trigger phrases and example requests. Appended to description in context; counts toward the 1,536-char cap. | | allowed-tools | No | Tools usable without per-use approval while the skill is active. Space-separated string or YAML list (both valid). | | license | No | License identifier (agentskills.io spec) | | compatibility | No | Compatibility constraints (agentskills.io spec) | | metadata | No | Arbitrary metadata object (agentskills.io spec) |

Claude Code extension fields

| Field | Required | Notes | |-------|----------|-------| | argument-hint | No | Hint shown during autocomplete. Example: [issue-number] or [filename] [format] | | arguments | No | Named positional args for $name substitution. Space-separated string or YAML list. Names map to positions in order. | | disable-model-invocation | No | true = user-only invocation. Removes skill from Claude's context and prevents preloading in subagents. | | user-invocable | No | false = hides skill from / menu but Claude can still auto-invoke it. | | disallowed-tools | No | Tools blocked while this skill is active. Cleared after the current message. | | model | No | Override model for this skill's turn only. Reverts to session model on next prompt. | | effort | No | Thinking effort: low, medium, high, xhigh, max. Overrides session effort for the turn. | | context | No | fork = runs skill in an isolated subagent context. The skill body becomes the subagent's prompt. | | agent | No | Which subagent type to use when context: fork is set. Options: Explore, Plan, general-purpose, or any custom agent in .claude/agents/. | | hooks | No | Skill-scoped lifecycle hooks. Same format as session hooks. | | paths | No | Glob patterns that limit when Claude auto-loads this skill. Same format as path-specific rules. | | shell | No | Shell for !backtick injection: bash (default) or powershell. |

Unsupported fields (flag and remove)

tags, category, keywords, usage, args are ignored by the runtime. Remove them to avoid confusion.

---

String Substitutions (check for correct usage)

If a skill references substitution placeholders, verify they use the correct syntax:

| Placeholder | Meaning | |-------------|---------| | $ARGUMENTS | Full argument string as typed | | $ARGUMENTS[N] or $N | Argument at position N (0-based) | | $name | Named argument declared in arguments: frontmatter | | ${CLAUDE_SESSION_ID} | Current session ID | | ${CLAUDE_EFFORT} | Active effort level (low / medium / high / xhigh / max) | | ${CLAUDE_SKILL_DIR} | Absolute path to the skill's directory; use it for referencing bundled scripts |

Incorrect placeholders ($1 when no arguments: field is declared, ${ARGS}, %ARGUMENTS%) are passed as literal text and silently break the skill.

---

Scoring Criteria (16 pts per skill)

| # | Criterion | Max | What is checked | |---|-----------|-----|-----------------| | 1 | name field | 1 | name field present and lowercase with hyphens only | | 2 | description | 3 | Present (1), has "Use when" / trigger phrasing (1), when_to_use field present for skills that need broad matching (1) | | 3 | allowed-tools | 2 | Present (1), scoped appropriately (1): not Bash without path scoping when read-only tools suffice | | 4 | effort | 3 | Present (1), appropriate for content per inference engine below (2) | | 5 | content structure | 5 | Has Purpose or When section (1), has concrete examples or usage (1), has clear workflow or steps (1), no placeholder text (1), no unsupported frontmatter fields (1) | | 6 | bonus | +2 | argument-hint present when skill takes args (1), ${CLAUDE_SKILL_DIR} used for bundled scripts instead of hardcoded paths (1) |

Thresholds:

• ✅ Good: >=13/16 (>=80%)
• ⚠️ Needs work: 10-12/16 (60-79%)
• ❌ Fix: 5 minutes
• Deep synthesis across entire codebase or multi-repo scope

Examples: /deep-research, /full-security-audit, /codebase-migration

Mismatch flag

If a skill has effort: already set but the inferred level differs, flag it: > ⚠️ Effort mismatch: declared low, inferred high. Skill spawns 4 sub-agents and performs security analysis.

---

Execution Instructions

Step 1: Discovery

# Find all SKILL.md files
find .claude/skills -name "SKILL.md" 2>/dev/null

# Find legacy flat command files
find .claude/commands -maxdepth 1 -name "*.md" ! -name "README*" 2>/dev/null

# If argument provided, use that path instead

Step 2: Parse each skill

For each skill file found:

1. Read the full file
2. Extract YAML frontmatter (between first --- and second ---)
3. Parse all recognized fields from the table above
4. Flag any unrecognized fields as potentially unsupported
5. Note presence/absence of each field
6. Read the body content for structure analysis and substitution placeholder correctness

Step 3: Score and infer

Apply the scoring criteria to each skill:

• Check frontmatter fields against the valid fields table
• Evaluate description quality: does it answer "when to use"? Is combined description + whentouse under 1,536 chars?
• Evaluate allowed-tools scope: is Bash used without path scoping when read-only tools suffice?
• Check context: fork skills for completeness; they must have an actionable task body and an agent field (or a reasonable default applies)
• Infer effort level from content analysis
• Compare inferred vs declared effort (if set)
• Evaluate content structure

Step 4: Output

Produce a structured report:

# Skills Audit: [project name or path]
Date: [today] | Scanned: N skills

## Summary
| Status | Count |
|--------|-------|
| ✅ Good (>=80%) | N |
| ⚠️ Needs work (60-79%) | N |
| ❌ Fix (<60%) | N |

**Effort coverage**: N/N skills have effort field set

---

## Per-Skill Results

### [skill-name] ([score]/16) [✅/⚠️/❌]

| Criterion | Score | Notes |
|-----------|-------|-------|
| name | ✅ 1/1 | ok |
| description | ⚠️ 2/3 | Missing when_to_use field |
| allowed-tools | ✅ 2/2 | Well-scoped |
| effort | ❌ 0/3 | Missing. Recommended: high |
| content structure | ⚠️ 3/5 | No examples section, unsupported field "tags" found |

**Effort inference**: `high`. Skill performs security analysis with adversarial reasoning.
  Signals: "threat", "attack surface", "vulnerability scoring" in content; spawns 4 agents

**Priority fixes** (ordered by impact):
1. Add `effort: high` to frontmatter
2. Add `when_to_use` with trigger phrases
3. Remove unsupported field `tags`
4. Add a concrete usage example section

---

After all skills: print a Fix Summary.

---

Fix Summary Format

At the end, print a ready-to-use patch block for all missing/mismatched effort fields:

## Recommended effort fields (copy-paste ready)

skill-name-1: effort: low     # mechanical scaffold
skill-name-2: effort: high    # security analysis, spawns agents
skill-name-3: effort: medium  # code review, bounded scope

And a 1-line count: N skills need effort field · N mismatches · N missing allowed-tools · N unsupported fields to remove

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: FlorianBruniaux
• Source: FlorianBruniaux/claude-code-plugins
• License: MIT

Install and usage instructions live in the source repository linked above.