Agent Acceptance Gate

agent-acceptance-gate

The last gate before merging multi-agent output. The reconciler describes what each agent did; the acceptance-gate decides whether the round is mergeable.

It runs the success_criteria declared in .coord/plan.yml, aggregates risks, optionally audits prose, checks budget, and produces a single PASS / FAIL / RETRY verdict per task and overall.

Why use this instead of "I'll grep it myself"

The gate replaces ad-hoc inline shell verification with one structured verdict. Concrete differences observed in production (measured-benefits.md R5 + Phase D):

| Inline shell verification | This skill (R5 measured) | |---|---| | Main session runs 5+ grep / diff commands, interprets each output, drafts a manual PASS/FAIL — ~10k tokens of shell-debugging in context | Subagent runs all checks + returns ~2k token structured report → ~5× token saving in main session | | Easy to forget a check ("did I grep for banned phrases this time?") | Preset YAML encodes the full check set — no missed checks | | Catches one drift case; misses cross-language echoes | cross_document_link_text_parity v0.2.3 caught 9+ real drift bugs the human audit missed |

The DeepSeek-R2 fabrication that almost shipped

2026-05-13 session: Codex generated a paragraph claiming "DeepSeek-R2 reaches 94.2% on GPQA Diamond" — pure fabrication. Manual grep on the diff missed it because the claim was internally consistent. Only a third-party reviewer agent doing live gh api search caught it.

The fact-check-frontier-models preset was built directly from this incident. Running the preset would have caught the fabrication automatically by hitting arxiv + GitHub for each (model, benchmark, %) triple in the diff.

→ F14: the preset existed by 2026-05-14, but a subsequent operator skipped it because "the task feels simple". The Phase D dogfood (49 files × 3 locales, 50k tokens inline vs 16k with gate = ~3× saving) proved the gate catches drift the operator misses. The cost of running the gate is much smaller than the cost of one missed drift.

Mandatory invocation triggers (no judgment call)

These triggers are mechanical — if any fire, invoke the matching preset before commit. Skipping is F14 territory.

| Trigger condition | MUST invoke | |---|---| | Diff touches ≥ 2 locale variants of same file stem (e.g., 06-x.md + 06-x.en.md + 06-x.zh-Hans.md) | --preset=multi-locale-mirror-sync --stem= | | Diff adds entries to any catalog file (project listings, framework comparisons) | --preset=catalog-entry-add --catalog-file= | | Diff touches a "frontier model" claim (model name within 3 lines of a benchmark %, e.g., "GPT-5.5 reaches 94% GPQA") | --preset=fact-check-frontier-models --file= --models= |

CLAUDE.md snippet to enforce the gate

## Acceptance gate rule (enforced)

Before any commit that:
- touches ≥ 2 locale variants of the same file stem → invoke
  `Skill("agent-collab-workspace:agent-acceptance-gate",
         args="--preset=multi-locale-mirror-sync --stem=")`
- adds catalog entries → invoke `--preset=catalog-entry-add ...`
- includes a frontier model + benchmark % claim → invoke
  `--preset=fact-check-frontier-models ...`

Skip only when the trigger does not apply (single-file diff, pure
mechanical rename, etc.). NEVER skip when the trigger fires —
this is the F14 anti-pattern that shipped the DeepSeek-R2
fabrication.

When the preset FAILS:
1. Read the FAIL reasons in the gate report.
2. Either fix-and-rerun (cheap drift) or re-delegate the task with
   tighter constraints (systemic drift).
3. Do NOT override the FAIL by hand. The whole point of the gate is
   that the operator's judgment failed earlier — adding more operator
   judgment on top defeats it.

When to use

Trigger phrases:

• "Run the acceptance gate."
• "Pre-commit check across multi-agent output."
• "Are we ready to commit this round?"
• "Verify all multi-agent output before I push."
• "Gate this round — go or no-go?"

Not for:

• Describing what each agent did → that's

agent-output-reconciler.

• Running the agents → delegate skills.
• Per-task acceptance during agent execution → that's the agent

skill itself (codex-delegate's wrapper checks its own task's acceptance).

• Single-agent runs without a .coord/plan.yml → just run

pytest and call it a day.

Inputs (auto-discovered)

1. .coord/plan.yml — round, tasks, success_criteria per

task, budget if declared, and context_policy if declared.

1. .ai/_log__.txt.result.json — token usage,

risks, files_changed. Filename uses double extension by design (codex-delegate appends .result.json to the log path; see examples/codex_log_001_*.txt.result.json.sample).

1. .coord/reconciliation_.md — reconciler's verdict; if

reconciler said "retry", gate respects that.

1. .coord/context_.md (optional, if agent-context-budget

ran) — declared per-task context budgets for this round. Gate checks that actual summary sizes / log tail counts honored the declared budgets. Absence is OK (the round may not have used the context-budget skill); presence means the gate enforces it.

1. For prose changes: if any result.json shows files_changed

matching *.md, *.tex, *.docx, the gate optionally invokes academic-writing-skills banned-word + claim-evidence audit. (Skipped silently if academic-writing-skills not installed.)

Presets (one-line invocation for common acceptance shapes)

Instead of hand-writing acceptance criteria every time, invoke a preset that codifies a tested set of checks:

| Preset | When to use | Invocation | |---|---|---| | multi-locale-mirror-sync | After zh-TW → en + zh-Hans mirror sync (or any N-locale fan-out) | agent-acceptance-gate --preset=multi-locale-mirror-sync --stem=stages/06-foo --required-terms="A,B,C" | | catalog-entry-add | Added entries to a catalog file | agent-acceptance-gate --preset=catalog-entry-add --catalog-file=resources/foo.md --new-entries="org/repo1,org/repo2" | | fact-check-frontier-models | Touched a frontier-model table | agent-acceptance-gate --preset=fact-check-frontier-models --file=stages/06-foo.md --models="GPT-5.5,Claude Opus 4.7" |

Preset YAMLs live in presets/. Each codifies failure modes observed in real dogfooding (see docs/observed-failure-modes.md). Don't hand-write checks if a preset covers your case.

Mandatory preset trigger conditions (gate auto-suggests if you don't specify):

• Diff touches ≥ 2 locale variants of the same file stem →

multi-locale-mirror-sync MUST be invoked.

• Diff adds entries to any file under resources/ matching catalog

shape → catalog-entry-add MUST be invoked.

• Diff touches a frontier-model claim (regex: model name within 3

lines of a benchmark %) → fact-check-frontier-models MUST be invoked.

Preset is mandatory when trigger fires (F14, 2026-05-14)

The presets above are not "consider running" — they are must run before commit when their trigger condition matches. The F14 incident (docs/observed-failure-modes.md) is the cautionary tale: a Phase D run on awesome-agentic-ai-zh touched 49 files across 3 locale variants (textbook multi-locale-mirror-sync trigger), skipped the preset, used a code-reviewer subagent instead, and shipped a drift the preset's cross_document_link_text_parity check was designed to catch.

Why skipping is tempting: when the work feels "just a title sweep, surely nothing can go wrong", the operator short-circuits the mandatory invocation. The presets exist precisely because that intuition is wrong — drift hides in the "obvious" cases.

Anti-patterns:

1. Replacing the preset with an ad-hoc code-reviewer subagent.

The subagent is a reasonable backup but cannot substitute for the codified checks, which encode observed failure modes. Run both, not one-instead-of-the-other.

1. "I'll run the preset later". The preset's diff-size check is

tied to the commit-staged diff; later means re-staging or running against history. Just run it before commit.

1. **Manually grep'ing for the same patterns the preset already

knows**. You'll miss one. The preset won't.

Enforcement options (in increasing strength):

• Documentation (current): this section + CLAUDE.md rule.

Held in 5 of 6 Phase B rounds, failed in Phase D.

• Pre-commit hook (recommended): mechanical check that any

mirror-diff commit prompts for preset invocation. Recipe in docs/observed-failure-modes.md F14.

• Block on missing preset run (strict): pre-commit hook fails

unless a .coord/acceptance_.md file exists in the commit. Use this in repos where the cost of drift is high (curriculum, public docs, anything user-facing).

Workflow

1. Identify round

Read .coord/plan.yml. Default to highest round. User can override.

2. Run each task's success_criteria

For each task with agent: codex or agent: gemini:

• Each success_criteria is either a runnable command or a

checkable assertion.

• Runnable command (pytest tests/auth, mypy src/, npm test):
• Execute it. Record exit code + last 20 lines of output.
• PASS = exit 0; FAIL otherwise.
• Checkable assertion ("src/auth/interfaces.py exists and defines AuthProvider ABC"):
• Translate to a verification (file existence, grep, AST check).
• Run it.
• PASS = assertion holds; FAIL otherwise.

For agent: claude tasks:

• success_criteria is usually "explicit YES/NO verdict in chat".
• Read the current Claude conversation for the most recent

statement matching the criterion. Mark PASS / FAIL based on whether Claude actually delivered the verdict.

3. Check the reconciler's recommendation

Read .coord/reconciliation_.md. If the reconciler's "Recommended action" section says anything other than "merge all" (e.g., "retry T2", "escalate", "manual merge needed"), the gate's verdict is at most CONDITIONAL PASS — the user is responsible for resolving the reconciler's flagged issue.

4. Aggregate risks

Concat all risks arrays from result.json files. Group by severity (gate makes its own call if not labeled — failed test = high; legacy compat concern noted but not breaking = medium).

5. Optional: prose audit

If any task changed *.md / *.tex files AND academic-writing-skills is installed:

• Invoke its banned-word audit on the changed files.
• Invoke its claim-evidence audit if .paper/claims.yml exists in

the project.

• Add results to the gate report.

If academic-writing-skills isn't installed, skip silently — don't fail the gate just because prose audit isn't available.

6. Cost / budget check

If .coord/plan.yml declared a budget.tokens:

• Sum tokens field across all result.json files for this round

(if present; some delegate wrappers don't write tokens — handle missing gracefully).

• PASS if under budget; FAIL with clear "you exceeded budget by X

tokens" if over.

If no budget declared, skip — don't invent one.

6.5. Context contract check

If .coord/plan.yml declares context_policy OR .coord/context_.md exists, enforce both:

From context_policy (plan-wide defaults):

• Each result summary must be at or below

result_summary_word_budget (default 250 words).

• Reconciliation and acceptance reports must reference raw logs by

path, not paste them.

• Failure diagnostics may include only the configured log tail

(default 50 lines).

• .coord/memory.yml entries must be promoted facts: decisions, open

questions, artifact pointers, or session outcomes. Long analysis in memory is a context violation.

From context_.md (per-task overrides, if present):

• Read the per-task task_packet_token_budget declarations.
• For each task, verify ` packet (the actual .ai/task_.md`)

did not exceed declared budget (rough char/word count, no need for exact tokenizer — flag at >120% of declared).

• Verify result_summary_word_budget per-task (overrides plan-wide

default if specified).

• Verify raw_logs_inline: path-only was honored — any log file

pasted inline in reconciliation / acceptance is a violation.

Debate caps (if .coord/debate_*.md files exist and are linked from plan.yml):

• Each per-turn Pro / Con argument ≤ 400 words.
• Total debate rounds ≤ 3 (unless plan.yml declares debate_rounds: N override).
• Final synthesis section ≤ 250 words.
• Total debate file size ≤ 8 KB.
• Violations: gate at most CONDITIONAL PASS; recommend compressing

the transcript before promoting any decision to memory.

Violations make the verdict at most CONDITIONAL PASS. If the violation hides acceptance evidence, mark FAIL and require a bounded summary rewrite.

6.6. Scope diff check (W1 — work boundary enforcement, file-level)

Compare git diff --name-only against each task's declared files_in_scope (from .coord/plan.yml task entries).

For each modified file F in the diff:

• If F appears in any task's files_in_scope → in-scope ✅
• If F is a "transitive fix" (e.g., anchor heading sync, mentioned in

task's result.md as intentional spillover with justification) → accept with WARN

• Otherwise → FAIL with message: `Scope violation: not in

any task's filesinscope; agent went outside brief`

This is the file-level enforcement for the W1 work boundary discipline. Without this check, brief writing "files in scope: [a, b, c]" is just guidance — the agent may still touch [d, e, f] and only manual diff review would catch it.

What this check does NOT verify (be honest):

• It does NOT verify the agent actually emitted the Confirmed scope:

echo block before editing. git diff --name-only only tells you which files got modified — it can't reconstruct whether the echo was the agent's first action. If you need echo verification, add an optional secondary check that greps the agent's result.md for the Confirmed scope: sentinel string.

• It does NOT catch scope violations WITHIN a permitted file (e.g.,

agent was told to edit foo.md section §3 but also edited §1). Use finer-grained acceptance criteria (per-section grep / line-range check) for that.

F11 + F12 specific catches:

• If diff touches resources/style-guide*.md AND the change replaces

a literal term in a contrast table → FAIL (F11 violation)

• If diff inserts a line matching pattern `^>?\s*(Attribution|Source|

Credits|Citation)s?:\s*` that wasn't requested in brief → FAIL (F12 violation)

7. Compose verdict

| Condition | Verdict | |---|---| | All successcriteria PASS, no risks, reconciler says merge, prose audit clean, budget ok | ✅ PASS | | All successcriteria PASS but reconciler flagged something | ⚠ CONDITIONAL PASS — user resolves reconciler's issue, then re-run gate | | Context contract violated but evidence is still checkable | ⚠ CONDITIONAL PASS — rewrite bounded summaries before next round | | Any success_criterion FAIL | ❌ FAIL — list which task / criterion | | Risks include unresolved blockers | ❌ FAIL — must address before merge | | Budget exceeded | ❌ FAIL — over budget (user explicitly OK can override by editing plan.yml) |

8. Write .coord/acceptance_.md

# Acceptance gate — round 1

**Verdict:** ⚠ CONDITIONAL PASS
**Run:** 2026-04-28T11:50:00Z
**Tasks gated:** 4
**Reconciliation report:** .coord/reconciliation_001.md

## Per-task results

### T1 — codex — extract-interfaces
- ✅ "src/auth/interfaces.py exists and defines AuthProvider ABC" — file present, grep matches.
- ✅ "no other source files modified" — git diff scope confirmed.

### T2 — codex — refactor-providers
- ❌ "pytest tests/auth/test_providers.py passes" — test_legacy_compat FAILED.
- ✅ "no imports of src.auth.legacy from other modules" — grep clean.

### T3 — gemini — review-doc-coverage
- ✅ "every public symbol in src/auth has a docstring" — gemini's report confirms.
-

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [WenyuChiou](https://github.com/WenyuChiou)
- **Source:** [WenyuChiou/agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills)
- **License:** MIT
- **Homepage:** https://github.com/WenyuChiou/agent-collab-skills

Install and usage instructions live in the source repository linked above.