Confidence

Confidence Assessment

Rate your confidence that the current work fully solves the stated requirement.

> Multi-signal evaluation. A single LLM-confidence number is unreliable as > a stand-alone gate (token probability ≠ correctness). This skill combines > the LLM's dimensional scoring with deterministic rule checks the agent > must run alongside. The final score is gated on BOTH passing.

Contents

• [Mode Detection](#mode-detection)
• [Assessment Dimensions](#assessment-dimensions)
• [For plan mode](#for-plan-mode) — multi-signal: LLM scoring + rule checks (89% cap on failure)
• [For code mode](#for-code-mode)
• [For analysis mode](#for-analysis-mode)
• [Output Format](#output-format)
• [Score Thresholds](#score-thresholds)
• [Iteration Protocol (plan mode)](#iteration-protocol-plan-mode)
• [Auto-Fix (Fix Mode Only)](#auto-fix-fix-mode-only)

---

Mode Detection

Check the arguments: $ARGUMENTS

| Argument | Default | Validates | When to use | | ---------------- | ------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------- | | plan | | Implementation plan completeness | After Phase 1 planning, before autonomous execution | | code | yes | Code implementation correctness | After writing code, before PR | | analysis | | Analysis accuracy (root cause, refactor rationale, or skill gap) | During investigation, before proposing a fix, refactor, or skill-source diff | | bug-analysis | | Deprecated alias for analysis — behaves identically | Backwards-compatible; emit a one-line deprecation note in the report header |

If no argument is provided, default to code.

Alias handling. bug-analysis is accepted as a deprecated alias and resolves to analysis with identical dimensions, weights, thresholds, and Fix Mode behaviour. When invoked with the alias, prepend a single line to the report header: > Note: \bug-analysis\ is a deprecated alias for \analysis\. Update the caller when convenient. The alias keeps in-flight workflows and existing transcripts working through the transition; remove it after callers have migrated.

If arguments contain "fix" (e.g., code fix, plan fix, analysis fix), run in Fix Mode — after the review, automatically apply fixes for any concerns found.

---

Assessment Dimensions

For plan mode

Plan mode is multi-signal: LLM dimensional scoring + deterministic rule checks. Both must pass for the gate to clear.

Step 1 — LLM dimensional scoring

| Dimension | Weight | What to evaluate | | ---------------- | ------ | ---------------------------------------------------------------------------------------------------------------- | | Completeness | 40% | Are ALL Phase 0 requirements captured? Are the Core sections populated (and the Extended sections present where the task needs them)? Could a new session execute from this plan alone? Do NOT penalize an Extended section that is legitimately omitted because its Include when trigger does not apply (e.g. no Existing Code Survey on a modification-only plan). DO score down an Existing Code Survey whose "Searched for" column is vague ("looked around") or whose BUILD NEW verdicts show no searches — rule #10 checks presence, this dimension checks search quality. | | Feasibility | 30% | Is the technical approach sound? Are patterns consistent with the codebase? Are risks identified where applicable? | | No ambiguity | 30% | Are implementation steps specific enough to execute without interpretation? Are edge cases addressed where applicable? |

Step 2 — Deterministic rule checks (run via Bash)

Every check below MUST pass for plan mode. A single failed rule caps the gate at 89% regardless of LLM score — the agent must surface the failed rule and either fix the plan or escalate to the user. Rules #10 and #11 are conditionally applicable (their trigger condition is part of the rule); when the condition does not hold they pass vacuously with a logged note — they never silently skip.

| # | Rule check | Verification | | - | --------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | 1 | plan.md exists at the expected path | test -f .agent/$(git branch --show-current)/plan.md | | 2 | All Core sections present | grep -E '^## (TL;DR\|Requirements\|Decisions\|Acceptance Criteria\|Implementation Order\|File Changes\|Verification\|Progress Log)' plan.md \| wc -l ≥ 8 — these are the always-on Core tier. Extended sections (Background, Technical Approach, Patterns, Edge Cases, API, Tests, Dependencies, Risks) are include-when-needed and are NOT counted here. | | 3 | Acceptance Criteria section is non-empty | awk '/^## Acceptance Criteria/{f=1;next} /^###? /{f=0} f' plan.md \| grep -c '^- \|^[0-9]' ≥ 1 | | 4 | Every file in ## File Changes resolves OR is create | For each modify/delete row, git ls-files returns the path. Create rows skip this check. | | 5 | Every requirement is tagged [user-stated] or [inferred]| awk '/^## Requirements/{f=1;next} /^###? /{f=0} !f{next} /^[-0-9]/{n++} /\[user-stated\]/{t++} /\[inferred\]/{t++} END{exit n!=t}' plan.md exits 0 — the tag count must equal the requirement count (the range stops at ### Out of Scope, so excluded items are not counted). Single awk command with no shell pipes, so the table row executes verbatim. | | 6 | Every decision row has a Rationale column populated | `awk -F'[\|]' '/^## Decisions/{f=1;next} /^###? /{f=0} !f{next} !/^[\|]/{next} {r++} r 0) rows++} END {if (creates == 0) exit 0; exit (rows | X% | ... | | | X% | ... | | | X% | ... |

Deterministic rule checks (plan mode only — omit for code/analysis)

| # | Rule | Status | Evidence | |---|---------------------|-------------|---------------------------| | 1 | | ✓ pass / ✗ fail | | | ... | | | |

Combined gate

• Weighted LLM score: X%
• Rule checks: / (cap: 100% if all pass, else 89%)
• Final: X%

Calculate the weighted LLM score as a weighted average using the dimension weights above. For `plan` mode, the **Final** is `min(weighted_LLM_score, rule_check_cap)`. For `code` and `analysis` modes (including invocations via the deprecated `bug-analysis` alias), omit the rule-check section and `Final = weighted_LLM_score`.

**Be honest and critical — do not inflate scores. A low score with clear reasoning is more valuable than a false 95%. A failed rule check is non-negotiable — surface it even if the LLM score is high.**

---

## Score Thresholds

| Score         | Action                                                                                      |
| ------------- | ------------------------------------------------------------------------------------------- |
| **90-100%**   | Proceed — work is ready                                                                     |
| **70-89%**    | List specific concerns and what would raise confidence. If in Fix Mode, apply fixes.        |
| **Below 70%** | Recommend concrete next steps to validate or fix. Do NOT proceed with autonomous execution. |

---

## Iteration Protocol (plan mode)

When used as a quality gate before autonomous execution:

> If confidence is below 90%, do up to **2 iterations** of additional research, analysis, and evidence collection to raise the score.
> After each iteration, re-run the confidence assessment.
> If still below 90% after 2 iterations, present findings to the user and ask whether to proceed or refine further.

---

## Auto-Fix (Fix Mode Only)

**Skip this section entirely if not in Fix Mode.**

When running in Fix Mode (`plan fix`, `code fix`, `analysis fix` — or the deprecated `bug-analysis fix` alias), automatically address every concern that lowered your score:

### Simple Fixes (apply immediately)

Fix these without asking — they are low-risk and mechanical:

- Missing edge case handling with obvious implementation
- Missing null/undefined checks
- Off-by-one errors or incorrect boundary conditions
- Typos in strings, comments, or variable names
- Missing return types or type annotations where the type is clear
- Small logic errors with an unambiguous correction
- (plan mode) Missing sections, incomplete requirements, vague implementation steps

After applying each fix, briefly note what was changed (one line per fix).

### Complex Fixes (plan, then apply)

For issues requiring more thought:

- Missing test coverage for uncovered paths
- Incomplete implementations (missing cases, unhandled states)
- Architectural concerns or incorrect abstractions
- (plan mode) Fundamental approach issues, missing technical design

For each, output:

[Issue title]

Why: [1-sentence explanation] Fix plan:

1. [Step 1]
2. [Step 2]

Files involved: [list]

Then execute the plan.

### Post-Fix Re-Assessment

After all fixes are applied:

1. Re-run the confidence assessment with updated scores
2. List what was fixed and how each fix improved the score
3. If confidence is still below 90%, list remaining concerns that could not be auto-fixed

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [mthines](https://github.com/mthines)
- **Source:** [mthines/agent-skills](https://github.com/mthines/agent-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.