# Truestack Skill Evaluation

> Evaluate and score Agent Skills — this set or any skill — on triggering accuracy,

- **Type:** Skill
- **Install:** `agentstack add skill-adtn0810-truestack-truestack-skill-evaluation`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [adtn0810](https://agentstack.voostack.com/s/adtn0810)
- **Installs:** 0
- **Category:** [Cloud & Infrastructure](https://agentstack.voostack.com/c/cloud-infrastructure)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [adtn0810](https://github.com/adtn0810)
- **Source:** https://github.com/adtn0810/truestack/tree/main/skills/truestack-skill-evaluation

## Install

```sh
agentstack add skill-adtn0810-truestack-truestack-skill-evaluation
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# truestack-skill-evaluation

A score is only worth what's behind it. This skill measures skill quality with evidence —
a deterministic lint you can re-run, plus a judged read and a trigger test — so "8/10" means
something checkable, not a guess. It exists because the easy failure here is exactly the one the
honesty contract forbids: asserting a confident number nobody verified.

## When to run
- The user asks to score / rate / audit / improve a skill or set.
- Right after writing or editing a skill (gate before shipping).
- Periodically, to catch drift, bloat, and trigger collisions as a set grows.

## Three layers (run in order; each is cheap)
1. **Static lint (deterministic).** Run `scripts/skill_lint.py ` (no Python on the
   box? `uv run --no-project scripts/skill_lint.py ` provisions one). It checks every
   `SKILL.md` for: valid frontmatter, `name` matching its folder, a description that says **what
   it does AND when to use it**, body within the token budget (**both** halves — lines and chars),
   `references/` that are actually linked (no orphans) and that every referenced file exists (no
   dead refs), and the obvious anti-patterns. Repeatable, no judgment — same input, same score;
   the lint itself is unit-tested by `scripts/test_skill_lint.py`.
2. **Semantic judge.** Read each `SKILL.md` and rate it against the rubric dimensions, grounded
   in the actual text — reward clarity and correct scope, **not length**. A long skill is a cost,
   not a virtue. Cite the line that justifies each deduction.
3. **Behavioral trigger test.** Two parts. (a) **Measured floor (deterministic):** run
   `scripts/trigger_eval.mjs` — it routes the committed `fixtures/trigger-cases.json`
   (prompt → expected skill, plus should-not-fire cases) by IDF-weighted keyword overlap and
   asserts each intended skill lands in the top-2; it exits 1 on a miss, so a description edited
   to stop matching its own triggers fails in CI. Keyword overlap only *approximates* the LLM
   router — it's a regression guard, not proof of live routing — so label it measured-but-approximate.
   (b) **Judged:** for cases the fixtures don't cover, write 3–5 should-fire and 2 should-not
   prompts per skill and reason about routing. Under-triggering (too timid) and over-triggering
   (grabs unrelated work, collides with a sibling) are both findings; add a regression case to the
   fixtures for any real miss you find.

## What it scores
Six dimensions, 0–10, weighted — full anchors, weights, anti-pattern catalog, and badge
thresholds in **`references/rubric.md`**:
- **Triggering accuracy** — does the description fire when it should and stay quiet when it shouldn't?
- **Scope calibration** — one clear job, not too broad (collides) or too narrow (never fires).
- **Token efficiency** — earns its context cost; progressive disclosure via `references/`.
- **Structural completeness** — frontmatter, body, references, handoff all present and consistent.
- **Robustness / anti-patterns** — none of: empty/vague description, missing "when", over-constrained MUST-walls, bloated body, orphan/dead reference, name mismatch, duplicated trigger surface.
- **Honesty adherence** — does the skill itself ground/verify/abstain rather than invent (for skills that produce claims)?

## Be honest about the number (this skill, of all skills)
Label each layer: the **static** score is deterministic; the **judge** and **behavioral** scores
are estimates — say so. Report only what you actually ran. Don't average away a Critical
finding (a dead reference or an empty description caps the skill until fixed). A measured **6/10
with three concrete fixes** is worth more than a flattering 9 — surfacing the gap is the point.

## Output
```
# Skill-eval report: 
## Scorecard
| Skill | Static | Judge | Behavioral | Score /10 | Badge | Top fix |
## Critical findings (must fix — cap the score)
## Per-skill notes (deduction → evidence → fix)
## Set-level: trigger collisions · total token budget · coverage gaps
Verdict: ship / fix-first — with the honest caveat on judged vs measured
```

## Explain it simply
Lead with the one-line verdict and the single highest-leverage fix. Show the scorecard as a
table, findings by severity — not prose. Make every deduction point to the line that caused it,
so a fix is obvious. After fixes, **re-run the lint** and show the before/after.

Fixes that change a skill's behavior or wording → that's editing the skill; re-run this eval to
confirm the score moved. For a brand-new skill, hold it to the same bar from the start: write it
against this skill's rubric (`references/rubric.md`) and run the lint before shipping.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [adtn0810](https://github.com/adtn0810)
- **Source:** [adtn0810/truestack](https://github.com/adtn0810/truestack)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-adtn0810-truestack-truestack-skill-evaluation
- Seller: https://agentstack.voostack.com/s/adtn0810
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.