AgentStack
Sign in Publish
MCP servers Skills Toolkits Why AgentStack Sell Docs
Sign in
SKILL verified MIT Self-run

Eval Capture

skill-ghostlygawd-recursive-harness-eval-capture · by GhostlyGawd

Snapshot a just-completed task into the regression corpus (evals/corpus/). Use after any task where the user accepted the result AND future harness changes could plausibly regress it — recurring task shapes, tasks that needed corrections before acceptance, anything involving taste the user articulated. Also use when /meta-retro reports a category with misses but no eval coverage. Without corpus g…

↗ Repository
— No reviews yet
0 installs
0 views
— view→install

Install

$ agentstack add skill-ghostlygawd-recursive-harness-eval-capture

✓ scanned · ✓ verified — works with Claude Code, Cursor, and more.

Security review

✓ Passed

No issues found. Passed automated security review. · v0.1.0 How review works →

  • ✓ Prompt-injection patterns
  • ✓ Secret / credential exfiltration
  • ✓ Dangerous shell & filesystem operations
  • ✓ Untrusted network calls
  • ✓ Known-malicious package signatures

What it can access

  • ✓ Network access No
  • ✓ Filesystem access No
  • ✓ Shell / process execution No
  • ✓ Environment & secrets No
  • ✓ Dynamic code execution No

From automated source analysis of v0.1.0. “Used” means the capability is present in the source — more access means more to trust, not that it’s unsafe.

Are you the author of Eval Capture? Claim this listing to set pricing, connect Stripe payouts, and keep 70% of every sale.
Sign up to claim

About

Eval Capture

The corpus is the only proof that harness vN+1 beats vN. Feed it.

When to capture (any one suffices)

  • The task shape recurs (third "write a migration" = the migration eval)
  • The user corrected you before accepting — the correction IS the rubric
  • The result encodes taste the user articulated ("never bury the lede")

Skip: one-offs, secrets/proprietary data (sanitize or skip), pure lookups.

Case anatomy: evals/corpus//

  • task.md — the request, verbatim enough to be re-runnable, with any needed

fixtures alongside.

  • exactly one grader:
  • check.py — objective: gets the output dir as argv[1], exits 0/1.

Prefer this whenever checkable; it's free to run forever.

  • rubric.md — subjective: 3-6 falsifiable criteria for the critic agent.

"Subject line under 60 chars" yes; "feels professional" no.

  • meta.json — {"date", "category", "source_session", "origin": "..."}

Rules

  • A correction-born case MUST encode the corrected behavior as the expected

one — that's the regression you're guarding against.

  • Keep cases hermetic: no network, no machine-specific paths, fixtures (fresh subagent

performs the task, no headless — ADR 0003) and confirm it passes today. A case that fails on day one is either a real finding (file it) or a bad rubric (fix it).

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

Install and usage instructions live in the source repository linked above.

Reviews

No reviews yet — be the first.

Write a review

Versions

  • v0.1.0 Imported from the upstream source.

More in Agent Skills