Logicprobe

Logic Probe

Documents are not truth — code is. Verify every verifiable claim before accepting or acting on any design.

Verification Depth (Plan-Mode Gate)

When loaded as a plan-mode verification gate, this skill's execution is mandatory. The model has no discretion to bypass it. Depth classification is gated on objective plan features extracted in Phase 0.

Phase 0: Feature Extraction (Mandatory)

Before any verification, output the plan's feature summary to context:

Plan features:
  Files: [N]
  Functions added/modified: [list or "none"]
  Behavioral claims: [none / "invariants listed" / "always/never/guaranteed assertions"]
  State machine changes: [none / describe topology delta]
→ Depth: LIGHTWEIGHT | STANDARD | ESCALATED

This step is NOT skippable — it creates an explicit, auditable record of what the plan claims before verification begins.

Depth Classification

| Plan Feature | Depth | |-------------|-------| | Single file, zero function signatures added/modified, no behavioral claims of any kind | LIGHTWEIGHT | | Multi-file, OR new/modified function signatures, OR implicit behavioral claims (invariants, equivalence assertions, "behavior is unchanged") | STANDARD | | "Always"/"never"/"guaranteed" language, OR state machine topology changes (≥1 state or ≥2 transitions modified) | ESCALATED |

"No behavioral claims" is narrow: if the plan asserts anything about behavior preservation — including listing invariants, claiming equivalence, or saying "refactoring only, no behavior change" — that IS a behavioral claim. The absence of the literal words "always"/"never" does NOT mean there are no behavioral claims.

Output Requirements

| Depth | Required | |-------|----------| | LIGHTWEIGHT | All 5 checklist items (file paths, API/type names, line numbers, behavioral claims, mechanism feasibility) answered in context with explicit results per item | | STANDARD | Phase 1-2: enumerate every verifiable claim → verify each against codebase with evidence | | ESCALATED | Full pipeline: Phase 1-5 including Logic Primitive Verification (Phase 2a + 2b, 14 checks) |

Plan Verification Block

After verification, append a summary to the plan file:

## Plan Verification

- **Depth**: [LIGHTWEIGHT / STANDARD / ESCALATED]
- **Scope**: [N] file paths, [M] API/type names, [K] line citations confirmed
- **Escalation**: [skipped — no behavioral claims detected] or [see transcript for N-check results]

This block is the audit trail. A future reviewer must be able to see what was verified and when.

Methodology

Phase 1: Enumerate Claims

Read the document fully. Extract every claim that is verifiable:

• Numeric claims (counts, sizes, frequencies)
• API/type/enum names
• File paths and line numbers
• Mechanism descriptions ("compile-time resolution", "static dispatch")

Phase 2: Verify Against Codebase

For each claim, run the relevant verification:

• Numeric claims: grep -c or grep -rn to get the real count
• API/type names: extract actual signatures from headers
• Enum/constant values: list actual values from BSP/config headers
• Mechanism feasibility: check language standard and compiler support

Phase 2 Trigger: Escalate to Logic Primitive?

If the document under review contains ANY of the following, escalate to [Logic Primitive Verification](#logic-primitive-verification) IMMEDIATELY:

• State machine or statechart with ≥3 states OR with guard conditions on transitions
• Protocol handshake, ACK/NACK, retry, or timeout sequence logic
• Claims using absolute language: "always", "never", "guaranteed", "all paths", "cannot", "impossible"
• Lock/unlock, alloc/free, start/stop paired operations where ordering matters
• Any logic where correctness depends on transition completeness or event ordering
• Refactoring that modifies state topology: splitting/merging states, adding/removing transitions, changing guard conditions, extracting sub-machines

Heuristic: If the transition-to-state ratio > 1.5 or any transition has a guard condition, the machine is complex enough to warrant verification — regardless of state count. For refactoring, trigger if the refactoring changes ≥1 state or ≥2 transitions from the original model.

Phase 3: Gap Analysis

Classify findings by severity:

1. Architecture-level: claims that make the design unimplementable (fake APIs, missing modules)
2. Mechanism-level: claims the language/compiler cannot fulfill
3. Consistency-level: internal contradictions across documents

Phase 4: Root Cause

For each error, identify why it happened:

• Wrong mental model (C++ constexpr thinking in C99)?
• Incomplete search scope?
• Copy-paste from other projects without verification?
• Misunderstanding of compiler/linker behavior?

Phase 5: Structured Output

Each finding includes:

• Exact location (file:line or section)
• What the document claims
• What the codebase actually contains (with evidence — grep output, line numbers)
• Correction direction

---

Logic Primitive Verification

When Phase 2 triggers escalation, do NOT proceed to Phase 3 until the verification pipeline below is complete. Trust models, not intuition.

Pipeline Overview

Document claims → Extract model → Runtime check:
  ├── DSH + `logicprobe_verify` tool available → build Model schema v1 (references/dsh-model-schema.md) → call the tool → structured report
  ├── Python available → fill in references/verification-harness.py → run → report
  └── No Python → Manual Verification Mode (see references/logic-verification-guide.md#manual-verification-mode)

Refactoring variant:
  Old code + Refactoring plan → Extract BEFORE model + AFTER model
    → Run pipeline on AFTER model (14 checks)
    → Compare BEFORE vs AFTER: behavioral preservation, regression, complexity delta
    → Flag any invariant that held in BEFORE but fails in AFTER

Refactoring Verification Mode

When the document under review is a refactoring plan (modifying existing state machine logic, not designing from scratch), adapt the pipeline:

1. Extract the BEFORE model from the existing codebase (not the plan — verify what the code actually does, not what the plan claims it does)
2. Extract the AFTER model from the refactoring plan
3. Show both tables to the user side by side and confirm the delta is intentional
4. Run Phase 2a + 2b on the AFTER model — same 14 checks as new design
5. Compare BEFORE vs AFTER:

| Check | Method | Severity if Violated | |-------|--------|:---:| | Behavioral preservation | Every event sequence accepted by BEFORE must also be accepted by AFTER (or explicitly removed per plan) | Error — regression | | Invariant continuity | Any invariant that held in BEFORE must hold in AFTER (unless the refactoring explicitly changes it) | Error — undocumented behavior change | | Deadlock regression | New states or transitions must not introduce deadlocks not present in BEFORE | Error | | Complexity claim | If plan claims "simpler": count states + transitions + guards. Is AFTER objectively simpler? | Warning — unsubstantiated claim | | Unreachable code | New states added in AFTER must be reachable (otherwise they're dead code from the start) | Warning |

1. Flag any behavioral delta not documented in the plan — the most common refactoring bug is an unintended side effect that the plan doesn't acknowledge

Detection step: Before generating any verification code, run python3 --version 2>&1 or python --version 2>&1. Check the output:

• Returns Python 3.x.y with x ≥ 6 → use Python harness
• Returns anything else (command not found, "Python was not found" Windows stub, version =1

ACTIVE | done | IDLE | - ERROR | cooldownelapsed | RECOVERING | - RECOVERING | reinitcomplete | IDLE | -

**CRITICAL**: Show this table to the user and ask for confirmation before generating the harness. The #1 failure mode of verification is extracting the wrong model. If the plan is ambiguous, flag it as a finding first — don't guess.

**Exception**: If the runtime reports `logicprobe interaction=auto`, do NOT call `ask_user_question`. Instead: (a) cite evidence for every extracted state/transition/guard, (b) round-trip the filled model back into a transition table and compare it with the extraction table, and (c) mark the report `UNCONFIRMED`.

### Code-Level Behavioral Suggestion

When the task is NOT document/plan review but involves code-level behavioral questions — e.g., the user is editing source files and asks:

- "check this timing sequence for bugs"
- "could this state machine deadlock here"
- "is this retry limit safe"
- "what happens if event X arrives during state Y"

→ **Proactively suggest** logicprobe as an optional verification pass. Do NOT escalate automatically — plan-level verification was likely already done. The suggestion is: "I can run a logic-primitive verification on this state machine to check for deadlocks, unreachable states, and boundary issues. Want me to?"

If the user says yes, extract the model from the existing code (not a plan document), and run the standard pipeline. Output the findings as suggestions, not requirements.

This covers the gap where behavioral verification is useful even when no design document is being reviewed.

### When NOT to Escalate

Skip logic-primitive verification when:

- The document makes no behavioral/logic claims (pure API listings, config tables, data schemas)
- The state machine has ≤2 states, no guards, and trivial transitions (IDLE↔ACTIVE)
- The claim is purely structural (file paths, type names, numeric constants) — Phase 2 grep verification is sufficient

---

## Rules

1. Never trust a document's claim without codebase verification.
2. Be honest about mechanism boundaries — if the language standard can't do it, say so.
3. Cite evidence with specific file:line references.
4. Don't fix during review — point the way, let implementation happen after approval.
5. **For behavioral claims: verify with code, not reasoning.** If a plan says "always", "never", or "guaranteed", generate and run a model. One counter-example is enough to refute a universal claim.
6. **Confirm the model before running it** — unless the runtime reports `logicprobe interaction=auto`. Extraction errors are the dominant failure mode of formal verification. In auto mode, substitute evidence-cited extraction + round-trip validation and mark the report `UNCONFIRMED`.
7. **Don't verify what the code already checks.** If the existing codebase has compile-time assertions, static analysis, or runtime checks for a property, cite those — don't re-verify in a Python model.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [AmethystLuna](https://github.com/AmethystLuna)
- **Source:** [AmethystLuna/logicprobe](https://github.com/AmethystLuna/logicprobe)
- **License:** MIT

Install and usage instructions live in the source repository linked above.