Skill Authoring

Skill Authoring for Weaker Models

Institutional files are read by a model with zero context and less judgment than the author. Write for that reader. Every rule must survive being followed literally, without the author present.

1. The executable-rule format

An abstract demand ("keep quality high", "be careful", "verify appropriately") without a judgment criterion is noise — it costs context and changes nothing. Every load-bearing rule states:

• Trigger — the observable condition under which it applies.
• Steps — what to actually do, imperative, copy-pasteable where possible.
• Done — the completion definition; how the follower knows it worked.

And where misreading is costly or the judgment boundary is subtle, add:

• One positive and one negative example — the fastest way to transmit a

judgment boundary to a weaker model. The strongest negative example quotes a rationalization actually observed ("tests are probably fine — the change is small") and names why it fails.

• On failure — the next step when it does not work (retry differently,

escalate, mark unresolved), so failure does not improvise.

Placement is part of the format: an eligibility or refusal hard-exit gate precedes the first artifact-producing step, not mid-procedure. A refusal or scope check positioned after generation has begun gets blown past by mid-build momentum — the executor already has an artifact to protect and reads the gate as an obstacle. The same check asked first costs one sentence and holds. Smoke-measured fail-then-fix at the source (n=1 per cell): a weak-tier executor ran 68 tool calls and escaped its sandbox past a mid-procedure scope check; with the identical check moved before the first generation step, six tool calls, nothing generated, correct early exit. When a rule refuses or scopes the work — "refuse red-line domains", "no adapter when the sector is coding in disguise" — the skill's step order puts that test before the executor has produced anything; done when every eligibility or refusal test precedes the first artifact-producing step in the skill's ordering. A verification gate whose input IS the produced work (tests pass, a ship check) stays terminal — this rule moves eligibility and refusal checks, not verification. ❌ "Stage 4: before finalizing, confirm the sector needed an adapter at all" — by Stage 4 the adapter exists and the check reads as waste. ✅ the same sentence as Stage 1's first bullet, before any artifact-producing step. (unprobed in-house; external evidence — see Provenance.)

2. Ground truth only

• Verify every command, flag, path, and claim against the actual repo/system

before writing it down. A wrong runbook is worse than none, because it is trusted.

• Verifying the incident does not verify the prescription. Distilling an

incident into a rule is a lossy transform that can introduce a bug the incident never had: the rule cites a real failure yet prescribes a mechanism that itself fails on exactly the case it targets (git cherry for squash-merge residue on a multi-commit branch — its per-commit patch-ids never match the single squash commit, canonical rule in operational-rigor §2; "ack a webhook before durably recording it" — a post-2xx crash then loses the event; "peek-then-commit" a spend cap — a TOCTOU overspend race under concurrent fan-out). One reviewed batch of 27 incident-mined rules had 4 of exactly this shape, each passing the author's own self-review and caught only by a cross-family mechanism review before merge. So when a rule's fix is a specific mechanism — a command, protocol, or algorithm distilled from a failure — before it ships for an agent to execute verbatim: (1) fix the correct OUTCOME in advance for both its own motivating scenario — traced through the failure mode it names (the crash, the concurrent fan-out, the squash), not merely confirmed that the incident was real — AND the nearest variant with one property flipped (multi-commit → single-commit, crash → no crash, concurrent fan-out → one worker); (2) run the mechanism against both and confirm it matches each — correctness that flips across that boundary is the trap this catches (git cherry is wrong on the multi-commit squash it targets yet right on a single-commit branch); (3) get a cross-family mechanism review (cross-model-review) attacking the MECHANISM, not the prose. No second family available → cross-model-review §6's fallback (same-model fresh-context critic, gap recorded) applies here too.

• What cannot be verified is labeled unverified or user-must-provide —

never silently invented. Unproven ideas stay labeled open/candidate; no oversell.

• Embed the knowledge itself; do not make private paths or one person's

memory a load-bearing reference. A hard-coded machine-absolute path is worse than a broken link: a stale duplicate clone resolves silently to an outdated copy and gets trusted (more dangerous than a 404, which at least fails loud). Anchor to the VCS root (git rev-parse --show-toplevel) and verify the path prefix before reading.

3. Provenance and decay

• Date-stamp volatile facts (versions, flags, model names, defaults).
• Capability-negative claims rot the worst (unprobed — private incident

as shape; see Provenance). About to write "no such flag", "only works interactively", "the API can't do X" into an instruction file — or about to act on one already there: these are version-scoped observations that read as timeless rules. A stale positive claim is far likelier to be exercised and exposed the first time someone follows it; a stale negative fails silent — it steers every later session away from a capability that now exists, and nothing ever exercises it to expose the rot. One playbook's "model switching only works in the interactive UI; no flag" was actively wrong at the tool's current version and had been routing sessions into a degraded path. A negative about a hosted model's BEHAVIOR is the one class a version pin cannot hold — hosted endpoints drift behind unchanged strings; that class follows delegation-and-review §1's pinned-string rule, not this protocol: date-stamp the recorded claim where written, and any session acting on it — routing decision or not — re-probes at decision time before repeating or relying on the negative, the re-probe satisfied only under the pinned-string rule's own attribution and unknown-property-fallback clauses, carried verbatim: "an unattributed answer measures an unknown model, not the slug's", and probe unavailable or failing → "assume the ADVERSE plausible state for this decision"; on any wording disagreement, the pinned-string rule wins. Writing a tool-interface negative: pin it to the version and probe it was observed on — and a capability controlled server-side (an API feature, an account rollout, a remote configuration) additionally pins the instance/account and observation date, because it can flip with no version change. Acting on one: read its pin; the tool's version has changed, the pin is missing, or the capability is server-side and any pinned dimension (instance, account, configuration, or simply time since the dated observation) may have drifted → re-verify with one probe (--help for a local interface claim; a server-side capability probes against the CURRENT decision's resolved instance/account — re-probing the former pin is comparison evidence, never the acting gate, and a local help screen proves nothing about an account-controlled feature; an existence claim — a flag listed, a field accepted — settles on --help or a schema read, while a FUNCTIONAL claim needs a trial invocation exercising the claimed-absent capability, and a trial whose success would be consequential (a send, a delete, a purchase) runs as a safe synthetic or dry-run form, or under its own authorization (operational-rigor §2) — no safe form and no authorization → the capability stays unknown), recording the newly observed dimensions, before obeying it; probe unavailable or inconclusive → the capability is unknown, not absent — record that where the claim is used and do not repeat the negative as fact. Done: writing — the claim carries its version pin, the probe that observed it, and (server-side) its instance/account and date; acting — every applicable pinned dimension is matched current, or re-probed, or recorded unknown. ✅ "playbook says no flag (pinned v0.2.98); current binary v0.2.101 — --help lists the flag now (existence), a dry-run invocation accepted it (function); corrected the playbook in place." ❌ "the playbook says there's no flag, so drive it through the UI."

• Correct a stale rule in place — never append the correction below the old

line. A zero-context reader obeys whichever sentence it reads first, not the latest one.

• End each skill with a short provenance note and a one-line re-verification

command for anything that may drift. A skill without a re-verification path decays into exactly the stale-instruction problem it was meant to solve.

• **A merged upstream integration is not necessarily the end of the

campaign** (unprobed — the upstream half of the incident is verifiable in this repo's PR history, the sync half contributor-reported; see Provenance). Before diff-verifying a local file against "upstream final" and closing the sync, check for continuation on the synced surfaces — a maintainer's review can continue in follow-up PRs rather than concluding in the one that first merged, and at sync time those rounds may not have merged YET. The synced surfaces are every file the sync contract couples (the change-X-update-Y pairs), not only the file in hand. Check BOTH lists, each with its own invocation, on the SAME upstream repo and target branch lineage (a backport into another release branch is not a hit) — OPEN first, then MERGED, so a PR that merges between the two queries leaves the first set only by entering the second: ALL currently-open PRs (no creation-time bound — a follow-up opened BEFORE the anchor merged still counts; e.g. gh pr list --repo --base --state open), then PRs merged after the anchor by MERGE TIME, not PR number (e.g. gh pr list --repo --base --state merged --json number,mergedAt — the repo/base flags and the mergedAt field are load-bearing: an unflagged query can read the wrong fork or default branch, and PR numbers do not order by merge time). Each list is enumerated to EXHAUSTION — the tool's default page size (gh's is 30) silently truncates, and a date bound does not lift the cap: paginate until the last page is short, and record the total counted. "Touching" is decided from each candidate's CHANGED FILES read mechanically, with the per-PR query ALSO repo-scoped — PR numbers are repository-local, so an unflagged view from a fork checkout reads the wrong PR (e.g. gh pr view --repo --json files or gh pr diff --repo --name-only), never from titles or bodies — a continuation PR's title may carry no path token while it edits the synced file. File enumeration has its own caps (gh's files query returns the first 100; hosted diffs truncate around 300): verify the retrieved file count equals the PR's changedFiles total, and when completeness cannot be proven, treat that PR as TOUCHING (conservative) or keep the sync provisional. One OPEN+MERGED pass is a snapshot with blind windows at its edges — a PR can change state between any two queries — so REPEAT the pass until a full OPEN+MERGED pass adds NO new TOUCHING-OR-UNCLASSIFIED candidate versus the previous pass — every newcomer gets its changed-files classification, AND every still-open candidate is reclassified each pass — an open PR's files mutate with new commits (track head OIDs to skip provably-unchanged ones); a transition to touching-or-unclassified destabilizes, while non-touching classifications never do (else a busy repo livelocks into provisional despite zero synced-surface hits); each pass's merged query re-covers whatever the prior open query lost to a merge. Still unstable after three passes → record the sync provisional, no further queries owed. A rename touches when EITHER path side matches a synced surface — path-oriented file listings can hide the old path, so where the tool does not expose both sides, treat renames conservatively as touching. Any touching hit → do not close the sync as final: re-anchor to the newest touching merged state, RE-DIFF the local files against that new state, and re-run the checks, or — when touching rounds are still open — record the sync as provisional with the follow-up fold owed. The re-diff is a GATE, not a citation: final closure requires zero unexplained sync-contract differences (differences → fold them and re-run; unresolved → provisional). Done when the sync record cites the stable-pass checks (commands + date + totals) with ZERO TOUCHING HITS — the candidate lists may be nonempty — and a clean local diff against the anchor state; that makes the anchor safe AS OF the check, never forever; otherwise it carries the provisional label.

• When two files must agree, write the sync contract down ("change X → update

Y") in the canonical file. Prose inventories rot; prefer "read the directory" over hand-kept lists, and pin unavoidable lists with a rule or test. Do not paraphrase a load-bearing clause in a secondary location — quote it verbatim or point to the canonical copy (a paraphrase drifts silently), and the sync contract must name which file wins on disagreement.

• Package a set with its own honesty ledger. Alongside its START-HERE router

(§4), a multi-skill project library ships two more companion files — a MANIFEST (one line per skill → what it is + the evidence backing it, so the next maintainer can re-verify and knows what would falsify it) and an UNCERTAINTY register that quarantines everything not settled, each item bucketed and ending in a safe default; the three together are the packaging trio. A one-off handoff needs neither companion file — just an uncertainty / safe-default section when claims are unsettled. Bucket shapes and the trio: references/project-skill-templates.md.

4. Memory architecture

• One source of truth per fact. One canonical instruction file per repo;

other entry files include or point to it. Never maintain the same content in two places.

• The always-loaded file is a short router. CLAUDE.md/AGENTS.md holds

only what every session needs plus pointers; long content lives in load-on-demand skills/docs. Every always-loaded line taxes every future session — it must earn that.

• Fix log: one incident per file (problem / root cause / fix, with

frontmatter for search), written right after the incident while the cause is fresh. Batch-imported backlogs produce a pile, not a log.

• Memory, notes, and fix-log files never hold secrets — no keys, tokens, or

credentials; name where a secret lives, never its value.

• Compile, don't retrieve. When a fix-log entry reveals a default rule,

promote the rule into the standing instructions; the entry remains as the record of why. Retrieval re-derives the answer every session; compilation pays once.

• **Flipping a current-state order does not retire the old one on its

own — sweep the whole file** (unprobed — private incident as shape; see Provenance). The instruction-file analog of operational-rigor §3's call-site sweep: a flipped default is an interface change whose call sites are every older verdict block in the same file. Updating the top summary or the newest paragraph is not enough: an older evidence block can still carry its own bold imperative verdict ("KEEP X AS DEFAULT") lower i

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: F-e-u-e-r
• Source: F-e-u-e-r/opus-pack
• License: MIT

Install and usage instructions live in the source repository linked above.