AgentStack
SKILL verified MIT Self-run

Verify Claim

skill-temurkhan13-aufgaard-verify-claim · by temurkhan13

Verify whether an agent's stated outcome ("I committed and pushed", "tests pass", "I cleaned up the temp dir", "deployment succeeded") matches actual filesystem / git / test state. Catches the chiefofautism failure mode (agent confidently misreports what it did) AND the Codex sandbox-escalation case (agent acknowledges read-only constraint then violates it). Use when you suspect an agent's comple…

No reviews yet
0 installs
7 views
0.0% view→install

Install

$ agentstack add skill-temurkhan13-aufgaard-verify-claim

✓ scanned · ✓ verified — works with Claude Code, Cursor, and more.

Security review

✓ Passed

No issues found. Passed automated security review. · v0.1.0 How review works →

  • Prompt-injection patterns
  • Secret / credential exfiltration
  • Dangerous shell & filesystem operations
  • Untrusted network calls
  • Known-malicious package signatures

What it can access

  • Network access No
  • Filesystem access No
  • Shell / process execution No
  • Environment & secrets No
  • Dynamic code execution No

From automated source analysis of v0.1.0. “Used” means the capability is present in the source — more access means more to trust, not that it’s unsafe.

Are you the author of Verify Claim? Claim this listing to set pricing, connect Stripe payouts, and keep 70% of every sale.
Sign up to claim

About

Verify Claim (output-action divergence check)

Compare an agent's stated outcome against actual state. Returns one of: STATEMATCHESCLAIM (agent told the truth), STATEUNCHANGED (claimed action but nothing happened), STATEVIOLATEDCONSTRAINT (action happened but violated stated constraint), STATEMISMATCH (action happened but differently than claimed).

What to do

  1. Parse $ARGUMENTS for the claim text. The first quoted string is the claim. Optional --before-snapshot= argument loads a previously-captured before-snapshot if available.
  2. If no before-snapshot is provided, attempt to infer one from recent context: check git log, recent file mtimes, recent test runs.
  3. Call mcp__openclaw-output-vetter__verify_action_outcome with:
  • claim: the claim text
  • before_snapshot: the loaded snapshot (or empty if inferring from current state)
  • after_snapshot: take fresh snapshot of relevant paths NOW
  1. Render the verdict.

Verdict rendering

STATEMATCHESCLAIM (agent told the truth)

> ✅ **VERIFIED** — claim matches actual state
> **Claim:** ""
> **Evidence:**
> - 
> - 
> **Confidence:** HIGH/MEDIUM/LOW based on snapshot completeness

STATE_UNCHANGED (action claimed but nothing happened)

> 🔴 **UNCHANGED** — claim says action happened, state shows it didn't
> **Claim:** ""
> **What's missing:**
> - 
> - 
> **What likely happened:** 
> **Action:** Re-run the action, watch for hook / permission / sandbox errors. Run `/aufgaard:verify-claim` again after.

STATEVIOLATEDCONSTRAINT (Codex-style)

> 🔴 **CONSTRAINT VIOLATED** — agent acknowledged a constraint then violated it
> **Claim:** ""
> **Stated constraint:** 
> **Actual violation:** 
> **Severity:** CRITICAL — this is the alignment-relevant failure mode
> **Action:** Roll back the unauthorized changes via `git restore`. Investigate why the constraint was violated — sandbox-escape bug? prompt overrride? agent-config attack? Use `/aufgaard:vet-config` on agent-config files to rule out poisoning.

STATE_MISMATCH (action happened but not as claimed)

> 🟡 **MISMATCH** — action happened, but differently than claimed
> **Claim:** ""
> **What's different:**
> - 
> **Likely cause:** 
> **Action:** Reconcile — either accept the actual state and update downstream consumers, or roll back and try the original claim again.

ACTION_OUTCOME rule families (8 rules in output-vetter v1.1+)

  • ACTIONOUTCOME.STATEUNCHANGED — claimed action, no diff
  • ACTIONOUTCOME.STATEVIOLATED_CONSTRAINT — read-only / sandbox / approval-gate breached
  • ACTIONOUTCOME.MISSINGFILE — claim references a file that doesn't exist
  • ACTIONOUTCOME.MISSINGCOMMIT — claim references commit that's not in git log
  • ACTIONOUTCOME.MISSINGTEST_RESULT — claim "tests pass" but no test run found
  • ACTIONOUTCOME.PARTIALDIFF — diff exists but doesn't include claimed changes
  • ACTIONOUTCOME.UNINTENDEDDIFF — diff includes changes outside claimed scope
  • ACTIONOUTCOME.SCOPEDRIFT — actual scope of change is wider than claimed

Cite the actual rule returned. The catalog above is reference.

Style notes

  • Be specific about the evidence. "git log has no commits" beats "git status looks unchanged."
  • For STATEVIOLATEDCONSTRAINT, treat as CRITICAL and recommend immediate rollback. This is the chiefofautism / Codex-class incident pattern.
  • For STATE_UNCHANGED, the most useful diagnostic is usually "did a hook / permission / sandbox block the action silently?" — bias the recommendation toward checking those layers.

Edge cases

  • If no before-snapshot can be inferred (fresh session, no recent activity), state: "No before-snapshot available. Verifying against current state alone — limited confidence. For higher confidence on future verifies, the PreToolUse hook captures snapshots automatically when the plugin is fully active."
  • If $ARGUMENTS is empty: "Pass the agent's claim as a quoted argument: /aufgaard:verify-claim 'I committed the fix and pushed to main'"
  • If output-vetter MCP not loaded: "output-vetter MCP not available. Verify pip install openclaw-output-vetter-mcp and the plugin is loaded."
  • If the claim is too vague to verify (e.g. "I improved the code"), respond: "The claim is too vague for state-based verification. Re-ask the agent to state a specific verifiable claim — file modified, commit hash, test name passed, etc."

Footer CTA

---
Output-action divergence (P6.y) is one of 35 documented production-AI failure patterns. The [Production-AI MCP Suite Bundle](https://temurah.gumroad.com/l/production-ai-mcp-suite) ($29) includes the 8-page Field Reference PDF — covers the chiefofautism viral case + Codex sandbox-escalation + 698 real-world scheming incidents from the CLTR study (Oct 2025 - Mar 2026).

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

Install and usage instructions live in the source repository linked above.

Reviews

No reviews yet — be the first.

Versions

  • v0.1.0 Imported from the upstream source.