Audit Skills

audit-skills

Heuristic scanner + structured checker for everything that should not ship. Fast (grep + JSON parse), false-positive-tolerant, never silently rewrites files. The agent classifies and the human decides.

When to use

• Before publishing a new skill / plugin
• Before merging a skill PR from another contributor
• After a refactor that touched multiple SKILL.md, plugin.json, README.md, or marketplace.json files
• After bumping plugin/marketplace conventions or rename rounds
• Periodic hygiene pass (monthly, before releases)
• After the official validator (claude plugin validate .) surfaces a warning that smells like a structural drift

Three axes

1. Leaks (personal info + internal references)

User-facing fields (description, keywords, frontmatter, README excerpts) where the content shouldn't be public.

Personal info (always blocking, regardless of catalog visibility):

• Real names of contributors (other than Yesterday org / public author handles)
• Private email addresses (look for @ not on documented public domains)
• Local filesystem paths that reveal a username (/Users//, /home//, C:\Users\\)
• Personal IPs, hostnames, machine names (e.g. kcma-d8, nas1, 192.168.x.x, *.lan, *.local)
• Personal API keys / tokens (sk-…, ghp_…, xoxb-…, anything matching common secret patterns)
• Browser bookmarks, session cookies, OAuth callback URLs with state tokens

Yesterday-internal (blocking in PUBLIC catalogs, expected in PRIVATE):

• Internal repo names ending in -internal (e.g. yopstack-internal, ystacks-internal)
• Internal infrastructure hostnames (*.yester.cloud is mostly public; *.yesterday.internal / *.ys.internal is not)
• Internal Slack channels, Linear project codes, internal Notion URLs
• Refactoring history phrases ("promoted from .-internal", "moved out of .-internal", "vendored from temp-home")
• Internal team member shorthand
• Yesterday-only dependency declarations (marketplace: "-internal") in plugins meant for the public catalog
• TODO/FIXME/XXX comments referencing internal Jira/Linear tickets

Migration-history framing (blocking in PUBLIC catalogs, OK in .ytstack/ history records):

• formerly X, previously X, predecessor, (renamed: X), migrated from
• Old plugin names that have been renamed (yopstack, yastack, ydstack, ydstack-extras, ystacks in the public Yesterday catalog)

2. Catalog consistency (structural drift)

Mechanical alignment checks across the repo. Failures here usually mean install will silently break or the marketplace will surface mismatched info.

| Check | Why it matters | | --- | --- | | marketplace.json.name matches every install command in READMEs / docs | Users copy-paste commands; mismatched marketplace name = "no such plugin" errors | | Bundle folder name === plugin.json.name | Inconsistency confuses maintainers; compile.mjs slug = folder name but marketplace surfaces plugin.name | | Every cross-marketplace dep target ({name, marketplace}) actually exists in that marketplace | Spec blocks cross-mp deps not allow-listed; install fails silently | | marketplace.json.allowCrossMarketplaceDependenciesOn covers every marketplace named in any bundle's dependencies[] | Spec rule: cross-mp deps from un-allowlisted marketplaces are blocked at install | | README plugin counts (X plugins total, per-section counts) match .compiled/marketplace.json | Doc rot when new skills are added but README isn't bumped | | Every plugin.json dependencies[] entry resolves: bare-string deps refer to a same-marketplace plugin; object-form deps refer to a real plugin in the named marketplace | Same as above | | compile.mjs + clean.mjs are byte-identical with the sister catalog | Drift between two compile pipelines wastes engineering time | | .gitignore covers .compiled/ and root marketplace symlinks | Otherwise generated artifacts leak into commits |

3. Plugin-spec compliance

Validate against the official schemas:

• Claude Code Plugins reference -- plugin.json schema, folder layout, slug conventions
• Claude Code Plugin marketplaces -- marketplace.json schema, source types, plugin entry fields
• Cursor Plugins -- .cursor-plugin/ mirror manifest, skills/rules layout

Plugin-manifest rules (.claude-plugin/plugin.json):

| Rule | Source | | --- | --- | | name is the only required field; kebab-case, no spaces | plugins-reference §"Required fields" | | version is optional; if set, plugin is pinned to that string until you bump it; if omitted, Claude Code uses the git commit SHA | plugins-reference §"Metadata fields" | | author documented fields: name (required), email (optional). url is not in the schema | plugins-reference §"Metadata fields" | | Other optional fields: $schema, description, homepage, repository, license, keywords, dependencies | plugins-reference §"Metadata fields" | | .claude-plugin/ contains only plugin.json. Skills, agents, commands, hooks, mcp, lsp folders MUST be at plugin root, not inside .claude-plugin/ | plugins-reference §"Standard plugin layout" warning | | Skills location: skills//SKILL.md at plugin root (or custom paths via skills field) | plugins-reference §"Skills" |

Marketplace-manifest rules (.claude-plugin/marketplace.json):

| Rule | Source | | --- | --- | | Required: name (kebab-case), owner (with name), plugins array | plugin-marketplaces §"Required fields" | | owner documented fields: name (required), email (optional). url is not in the schema | plugin-marketplaces §"Owner fields" | | Per-plugin entry required: name, source | plugin-marketplaces §"Required fields" | | Relative path sources must start with ./, never use .. | plugin-marketplaces §"Relative paths" | | allowCrossMarketplaceDependenciesOn lists every other marketplace that this catalog's plugins depend on; missing entries block install | plugin-marketplaces §"Optional fields" | | Reserved marketplace names: claude-code-marketplace, claude-code-plugins, claude-plugins-official, anthropic-marketplace, anthropic-plugins, agent-skills, knowledge-work-plugins, life-sciences | plugin-marketplaces §"Reserved names" |

Cursor parity:

| Rule | Source | | --- | --- | | Cursor mirror manifest at .cursor-plugin/plugin.json (per-plugin) and .cursor-plugin/marketplace.json (catalog) | cursor.com/docs/plugins | | Skills, rules, MCP servers in same default locations as Claude Code (skills/, rules/, mcp.json) | cursor.com/docs/plugins |

Where to look

Public-facing surfaces in this repo:

| Surface | Path glob | Why it's user-facing | | --- | --- | --- | | Skill frontmatter | skills/**/SKILL.md (top YAML block, esp. description) | Read by the model; surfaced in plugin listings | | Skill body | skills/**/SKILL.md | Loaded into the agent's context every time the skill fires | | Skill manifests | skills/**/.plugin.json | Symlinked into the marketplace; description is shown in /plugin install | | Bundle manifests | plugins/*/plugin.json (canonical) | Authoritative; .claude-plugin/ and .cursor-plugin/ symlink to it | | Marketplace header | marketplace.json | Top-level catalog metadata | | Compiled marketplace | .compiled/marketplace.json | Generated; should match sources | | Skill READMEs | plugins/*/README.md, skills/**/README.md | Often included verbatim in plugin install pages | | Skill references | skills/**/references/*.md | Loaded on-demand by skills; same exposure as body | | Catalog docs | README.md, CONTRIBUTING.md, AGENTS.md, NOTICE | Public install + contributor guide |

Procedure

1. Run axis 1 (leak scan) -- inline command below or references/leak-patterns.md regex set.
2. Run axis 2 (consistency scan) -- inline checks below.
3. Run axis 3 (spec compliance) -- where possible, run the official validator (claude plugin validate . from inside any `` or marketplace root) and supplement with the inline checks below.
4. Classify each hit as one of:

• LEAK -- clear personal/internal info, must be removed before publishing
• JARGON -- internal-sounding phrasing that should be rewritten in user-facing language
• INCONSISTENCY -- structural drift (renames, dep targets, doc counts)
• SPEC -- violates the Claude Code or Cursor plugin spec
• OK -- false positive

1. Report the triage table to the user, grouped by file. Quote the offending line. Suggest a replacement when obvious. Do NOT auto-fix.
2. For each non-OK finding, ask the user whether to apply the suggested fix (one-by-one) or batch.
3. Re-run the relevant axis after fixes to confirm zero blocking findings remain.

Inline scan commands

Axis 1: Leak scan

# from repo root. Use find+xargs (recursive, glob-safe) and `grep -e` so
# patterns starting with `-` aren't parsed as options.
PATTERNS='\b\w*-internal\b|/Users/[a-z]+/|/home/[a-z]+/|\bkcma-d8\b|\bnas1\b|\b192\.168\.|\b10\.0\.|\.local\b|\.lan\b|\btemp-home\b|promoted from .*-internal|vendored from temp-home|formerly\b|previously lived|migrated from|\(renamed:|TODO\([^)]+\)|\bFIXME\b|\bXXX:|sk-[A-Za-z0-9]{20,}|ghp_[A-Za-z0-9]{20,}|xox[baprs]-[A-Za-z0-9-]{10,}'

find skills plugins marketplace.json README.md AGENTS.md CONTRIBUTING.md NOTICE -type f \
  \( -name '*.md' -o -name '*.json' -o -name 'NOTICE' -o -name 'LICENSE' \) \
  -not -path '*/.compiled/*' \
  -not -path '*/node_modules/*' \
  -not -path '*/.agents/*' \
  -not -path '*/.ytstack/*' \
  2>/dev/null \
  | xargs grep -nE -e "$PATTERNS" 2>/dev/null

Excluded: .compiled/ (generated), .agents/ (this skill's own pattern docs), .ytstack/ (project memory is allowed to keep history records per the "no migration framing in user-facing files" rule).

Axis 2: Consistency checks

# from repo root.

# 2a. Marketplace name must match every install command
MP_NAME=$(node -e "console.log(JSON.parse(require('fs').readFileSync('marketplace.json','utf8')).name)")
echo "Marketplace name: $MP_NAME"
echo "Install commands referencing other names:"
grep -rn '/plugin install [^@]*@' README.md AGENTS.md CONTRIBUTING.md plugins/*/README.md 2>/dev/null \
  | grep -v "@$MP_NAME"

# 2b. Bundle folder name vs plugin.json name
echo "Folder/name mismatches:"
for f in plugins/*/plugin.json; do
  n=$(node -e "console.log(JSON.parse(require('fs').readFileSync('$f','utf8')).name)" 2>/dev/null)
  fold=$(basename $(dirname $f))
  [ "$n" != "$fold" ] && echo "  $fold -> $n"
done

# 2c. Cross-marketplace deps that point to non-existent plugins
# (requires sister marketplace .compiled/marketplace.json to compare against;
#  inline this check with the actual sister-repo path you cross-depend on)

# 2d. README plugin count vs compiled count
COMPILED=$(node -e "console.log(JSON.parse(require('fs').readFileSync('.compiled/marketplace.json','utf8')).plugins.length)")
README_CLAIM=$(grep -oE '[0-9]+ plugins total' README.md | head -1)
echo "Compiled count: $COMPILED   README claim: $README_CLAIM"

# 2e. compile.mjs / clean.mjs / .gitignore parity with sister catalog
# Adapt SISTER_PATH to your sister repo
SISTER_PATH=../yesterday-skills   # or ../skills, depending which side you are
diff -q compile.mjs $SISTER_PATH/compile.mjs
diff -q clean.mjs $SISTER_PATH/clean.mjs
diff -q .gitignore $SISTER_PATH/.gitignore

# 2f. .compiled/ / root marketplace symlinks gitignored
grep -E '^\.compiled/?$|^\.claude-plugin/marketplace\.json|^\.cursor-plugin/marketplace\.json' .gitignore

Axis 3: Spec compliance

# 3a. Run the official validator on the marketplace + every bundle
claude plugin validate .                          # marketplace check
for d in plugins/*/; do claude plugin validate "$d"; done

# 3b. plugin.json: required field present
for f in plugins/*/plugin.json skills/**/.plugin.json; do
  node -e "
    const d=JSON.parse(require('fs').readFileSync('$f','utf8'));
    if (!d.name) console.log('MISSING name:', '$f');
  "
done

# 3c. plugin.json: undocumented author fields
for f in plugins/*/plugin.json skills/**/.plugin.json; do
  node -e "
    const d=JSON.parse(require('fs').readFileSync('$f','utf8'));
    const ok=new Set(['name','email']);
    if (d.author) Object.keys(d.author).filter(k=>!ok.has(k)).forEach(k=>console.log('AUTHOR.$f has undocumented .'+k));
  "
done

# 3d. plugin.json: forbidden version pin (project policy: omit version, use git SHA)
for f in plugins/*/plugin.json skills/**/.plugin.json; do
  grep -l '"version"' "$f" 2>/dev/null | xargs -I{} echo "VERSION pin in {}"
done

# 3e. marketplace.json: required fields + undocumented owner fields
node -e "
  const m=JSON.parse(require('fs').readFileSync('marketplace.json','utf8'));
  for (const k of ['name','owner','plugins']) if (!m[k]) console.log('MARKETPLACE missing:', k);
  if (m.owner) Object.keys(m.owner).filter(k=>!['name','email'].includes(k)).forEach(k=>console.log('OWNER undocumented field: .'+k));
"

# 3f. relative source paths must start with ./, no ..
node -e "
  const m=JSON.parse(require('fs').readFileSync('.compiled/marketplace.json','utf8'));
  m.plugins.filter(p=>typeof p.source==='string').forEach(p=>{
    if (!p.source.startsWith('./')) console.log('SOURCE not ./-relative:', p.name, p.source);
    if (p.source.includes('..')) console.log('SOURCE contains ..:', p.name, p.source);
  });
"

Output format

Report a table per file. Example:

plugins/office/plugin.json
  L3: JARGON  "...live in ydstack-extras"
       -> office-extras (renamed plugin)

README.md
  L60: INCONSISTENCY  "13 plugins total: ... 8 standalone"
       -> compiled count is 14 / 9; bump and add new standalone to category section

plugins//plugin.json
  -:  SPEC     author has undocumented `url` field
       -> remove or replace with email (spec only documents name + email)

End the report with a one-line summary: LEAK, JARGON, INCONSISTENCY, SPEC across files. Suggest fixes? (y / per-file / skip).

What NOT to flag

• Public Yesterday brand mentions (Yesterday, Yesterday-AI, yester.cloud) -- intentionally public.
• Plugin descriptions that mention other PUBLIC plugins by name (e.g. office-extras referencing office) -- catalog cross-refs.
• README install commands that reference org names.
• Keyword arrays containing yesterday.
• Internal references in PRIVATE catalogs (sister yesterday-skills) -- flag for awareness, but they are intentional content there.
• Migration history living in .ytstack/DECISIONS.md -- intentional history record (project memory is allowed to keep what user-facing files cannot).

False-positive examples

• kcma-d8 inside a local-llm skill that legitimately documents that the home-network GPU server is reachable at that hostname for users who set up the same topology -- if the doc explains the assumption, classify as JARGON not LEAK.
• /Users/alex/ inside a code block that demonstrates path-handling -- annotate or replace with /Users//.
• The word "internal" in a sentence like "internal state of the function" -- pure English usage; the regex \b\w*-internal\b requires a hyphen prefix, so this should not match. If it does, tighten the regex.
• promoted from corrections in self-improvement docs -- pattern promoted from .*-internal was tightened to avoid this; flag any regression to promoted from bare.

Limits of this skill

• Heuristic. Will miss findings that don't match patterns and over-flag legitimate uses. The human verdict is the source of truth.
• Does not parse YAML / JSON semantically for axis 1 -- regex on raw bytes. A leak that spans multiple lines after a key may be partially shown.
• Axis 3 (claude plugin validate) is the closest thing to a definitive spec check; the inline JSON checks are supplemental.

-

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Yesterday-AI
• Source: Yesterday-AI/skills
• License: MIT
• Homepage: https://github.com/Yesterday-AI/skills

Install and usage instructions live in the source repository linked above.