Calibration

Calibration

Self-awareness you can't audit is vibes. This protocol replaces it with a ledger.

Before acting

State, concretely, what success looks like — then log it:

harness predict --task "fix flaky auth test" \ --expect "root cause is async teardown; --result hit|miss --notes "actual cause was fixture ordering"

Score EVERY prediction, especially misses — misses are where the information is. The SessionStart banner shows your unscored count; keep it near zero.

Don't self-grade the user's acceptance

If a prediction's success clause includes the user accepting something ("user accepts", "<=2 rounds of correction", "no edits requested"), you may NOT score it until the user has actually responded to that thing. Self-declared acceptance is not an outcome — it is you grading from your own seat. Until the user reacts, the prediction stays pending (open debt), never a self-awarded hit.

Corollary: a prediction that bundles a mechanical clause (the loop runs) with an acceptance clause (the user accepts the design) can be a mechanical HIT and an acceptance MISS at once. Log them as TWO predictions up front (the outcome CLI scores one result per id), or wait for the user — a green mechanical result must not let you round the acceptance clause up to "hit".

provenance: session 3772bd2d, 2026-06-20 — scored prediction f156c20b a "hit" (its clause: "user accepts ... with <=2 rounds of correction") minutes BEFORE the user delivered a fundamental, design-invalidating correction. Acceptance was self-graded from inside Claude's own frame; the real outcome was a miss on that clause.

A load-bearing prediction gates shipping

If a prediction underwrites the DELIVERABLE — its core behavioral claim, the thing the user actually asked for — then while it is still pending or scored miss, the work is NOT done and must NOT merge. Green gates (tests, lint, a passing auditor, an eval run) verify the ARTIFACTS; they do not verify the CLAIM. Resolve it by exercising the REAL path end-to-end — the actual tool / trigger / environment, never a proxy — then score it before declaring done or merging. Shipping on a self-flagged "unverified" load-bearing claim is how a green PR delivers the opposite of the goal. (2026-06-20: merged a "rides into worktrees" change while its own prediction 55b1735b said that exact behavior was unverified; post-merge it scored a MISS and the whole goal had to be re-delivered.)

Reading your own stats

harness stats reports two lenses on the same scored log — confidence buckets (claimed vs. actual hit rate, auto-flagged on large drift) and a per-category hit-rate breakdown (no flag; it always prints both):

• Overconfident in a category → before acting in that category, list two

ways you could be wrong and check one. Note persistent drift in memory/calibration/notes.md with a date.

• Underconfident → stop hedging and over-asking; act, then verify.
• Misses clustering on one category → that's a missing skill or hook.

Route it (skill: routing-learnings).

Wrong-track detection in flight

While working, your prediction is the tripwire: the moment observed reality diverges from --expect (extra files ballooning, new failure class, the "quick fix" sprouting branches), stop and re-plan instead of pushing through. That divergence-noticing reflex is the intuition this harness trains.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: GhostlyGawd
• Source: GhostlyGawd/recursive-harness
• License: MIT

Install and usage instructions live in the source repository linked above.