Ultra Review

Ultra Review

Coordinate a fleet of independent AI agents through a multi-round adversarial review of a code change. Reviewers investigate in parallel under narrow scopes, then cross-examine each other's findings — survivors are the defensible ones.

Designed for multi-agent collaboration across heterogeneous LLMs (Claude, Codex, Aider, custom harnesses), with the orchestrator composing the fleet and digesting results rather than reviewing code itself.

Optimizes for depth and adversarially-confirmed signal, not speed.

Credits: generalized from the original .NET-focused skill concept in PlatformPlatform by Thomas Jespersen.

---

When to invoke

• Interactive mode — the user explicitly typed "ultra review" or "ultra-review". Run the full interview (STEP 1) and present results in chat (STEP 10).
• Autonomous mode — another agent (e.g., a team-lead orchestrator at feature completion) invokes this skill programmatically with scope, risk hotspots, size, and confidence policy already specified. Skip STEP 1's interview. Skip STEP 10's chat presentation. Always write TASKS.md; return its path to the caller.

If you cannot tell which mode you are in, ask once. Default to interactive.

---

Host capability requirements

This skill is portable across LLM platforms (Claude Code, Codex CLI, Aider, Cursor, custom agent harnesses, etc.). It assumes:

| Capability | Required? | Used for | |---|---|---| | Filesystem read/write | Required | Writing artifacts to disk; agents communicate via files, not return values | | Code search (grep / ripgrep / equivalent) | Required | Every reviewer needs to navigate the codebase | | Version control (git or equivalent) | Required | Computing the diff under review | | Parallel sub-agent / delegated-task execution | Strongly preferred | Each round launches N agents at once; without this, fall back to sequential execution and accept the wall-clock cost | | Structured question / multi-choice prompt to user | Preferred | STEP 1 interview; degrades to plain-text questions if unavailable | | Web research (search + fetch) | Preferred | Verifying external assumptions, API contracts, CVE data | | Read-only database query | Optional | Verifying data-shape assumptions for backend reviews | | Domain integrations (payment, CI, issue tracker, observability, etc.) | Optional | Whichever the host platform exposes; agents use what's available |

Mapping to your host: wherever this document says "spawn a sub-agent" or "delegated reviewer", invoke your platform's task-delegation primitive (Claude: Task tool with an appropriate subagent_type; Codex: spawned agent; custom harness: whatever your equivalent is). Wherever it says "code-search capability", use your platform's grep/find/ripgrep tool. None of the workflow depends on a specific tool name.

If parallel sub-agent execution is not available, run agents sequentially. Same artifacts written. Same workflow. Just slower.

---

Glossary

• Orchestrator — the agent running this skill. Designs the review, launches reviewers, digests results. Does not read findings until the final digest step.
• Reviewer agent (or just "agent") — a delegated sub-agent assigned one narrow scope. Reads code, finds problems, writes findings to disk, returns a one-line triage. Never returns findings as prose.
• Scope — a one- to two-line description of an area and an angle. NOT a checklist. Names where to look, not what to find.
• Round — one parallel wave of agent work. This workflow has four: Discovery, Cross-review, Finalization, Digest.
• Confidence — categorical, not numeric: Certain / Likely / Possible. See Core principles.
• High-impact area — domains where false negatives are worse than false positives (security, data loss, privacy, regulatory, financial correctness). Subject to looser confidence rules.
• Affinity cluster — an orchestrator-side grouping of reviewers used to inform Round 2 assignment. Never shown to agents. Never a hard partition.

---

Core principles

• Depth, not efficiency. Spend the tokens, time, and tool calls needed. This skill is invoked when the cost of missing a defect outweighs the cost of running a thorough review.
• Generic by design. Roster, clusters, and focus areas are co-designed with the user every time, from the diff. There is no fixed catalog of agent types.
• 80/20 effort split. Each agent spends ~80% of its effort on the two or three highest-risk subareas of its scope (deep dive), ~20% sanity-scanning the rest.
• No cap on agent count. Match the change. A 20K-line PR touching money, third-party integrations, and schema changes may warrant 25+ agents. A small UI fix may warrant 4. Do not impose an arbitrary ceiling.
• Multiple agents on hot areas. Assign 3–4 agents per high-risk area from different angles ("correctness of the math", "behavior under partial failure", "what happens during a rolling deploy"). Overlap is expected and strengthens signal — when two independent agents land the same issue, that is a strong confirmation.
• Agents run independent and parallel. No mid-flight coordination. No splitting work between agents during a round. Independent investigation produces independent evidence.
• False-positive hunt in Round 2. Reviewers in Round 2 try to disprove findings, not validate them. Survivors must be defensible.
• Confidence is categorical, not numeric:
• Certain — verified. The agent can reproduce, cite exact code paths, and quote evidence.
• Likely — strong evidence, one step short of full verification. The agent must state what would close the gap.
• Possible — plausible from patterns or partial evidence; not verified. The agent must state the gap.
• Confidence policy (set in interview or by caller):
• Certain only — drop weaker findings.
• Allow Likely and Possible — keep weaker findings, each with a "Why not Certain" note.
• High-impact exception — reviewers flagged as covering high-impact areas may always keep Likely and Possible findings regardless of policy. Better to surface a possible breach than drop it.

---

Output structure

All artifacts live under a stable, machine-readable path:

//ultra-review//
├── CONTEXT.md              # Diff summary, ticket/spec excerpts, environment, scope
├── ROSTER.md               # Final agent list with affinity clusters
├── round1/
│   └── .md     # One file per agent — discovery findings
├── round2/
│   ├── ASSIGNMENT.md                 # Reviewer-to-author table from Round 1 triage
│   └── __on__.md   # One file per cross-review pair
├── round3/
│   └── .md     # One file per agent — final findings + implementations
├── SUMMARY.md              # Orchestrator digest, deduplicated, prioritized
└── TASKS.md                # Optional — task list for engineers (always in autonomous mode)

Resolution rules:

• ` — convention is .workspace/ at the repository root. Adjust to host convention (e.g., .agent-workspace/, .reviews/`).
• ` — current VCS branch name. Use the host's branch-detection command (git branch --show-current` or equivalent). If detached HEAD, use the short commit SHA.
• ` — local-time YYYYMMDD-HHmm`.
• `` — lowercase, hyphenated agent name.

Create the directory tree at the start of STEP 2.

---

Workflow

STEP 1 — Interview (interactive mode only)

Infer what you can first; ask only what you cannot.

Infer:

• Scope — current branch and full diff vs the main branch (or whatever base the user named). Use VCS to list changed files, count lines added/removed, and bucket by directory.
• Linked ticket / spec context — scan branch name, commit messages, and recent conversation for ticket references. Fetch any found.
• Environment — current worktree, what tooling is available (web research, DB query, integrations).

Then ask (batch into one prompt if the host supports multi-question forms; otherwise ask one at a time):

1. Scope — confirm what's being reviewed. (Branch, PR, commit range.)
2. Environment — anything non-standard agents should know? (Staging DB available? Local services running? Feature flags?)
3. Risk hotspots — areas the user wants extra eyes on.
4. Deprioritize — areas to skip or treat lightly.
5. Size — small (~5 agents), medium (~10–15), large (20+). Default from diff size.
6. Confidence policy — single choice:

• "Certain only" (drop weaker)
• "Allow Likely and Possible with explanation" (recommended for thorough reviews where missing a real issue is worse than surfacing a maybe)

In autonomous mode, the caller supplies all of these directly — skip the interview.

---

STEP 2 — Pre-fetch shared context

Create the artifact directory. Write CONTEXT.md with the small, shared inputs every agent needs. The diff itself does not go in here — each agent pulls its own slice from VCS. This file is metadata only.

Include:

• Branch name and base reference (so each agent can compute their own diff).
• Category breakdown of changed files: counts per area (e.g., backend / frontend / migrations / tests / config / docs / infra). Use whatever taxonomy fits the codebase.
• Excerpts (not full content) of any linked ticket, spec, or design doc.
• Special environment notes from the interview.
• Pointers (not contents) to any project-level rules, style guides, or architecture documents agents should respect (e.g., /RULES.md, /ARCHITECTURE.md).

Keep CONTEXT.md under one page. Detail belongs in the source files.

---

STEP 3 — Co-design the agent roster

Propose a roster tailored to the diff. There is no fixed catalog. Design fresh every time from file paths, commit messages, and ticket context.

Critical: keep scopes open-ended. The single biggest failure mode of this workflow is the orchestrator confining agents to narrow checklists and missing what the orchestrator didn't think of. Name AREAS worth a deep dive — do not pre-investigate them. A one-line scope naming an area and an angle is right. Five lines listing specific classes, files, methods, columns, or expected outcomes is wrong: it tells the agent what to find instead of letting them discover it.

You do not deeply read code in this step. Agents do.

Guidelines:

• Identify which areas the diff touches and which carry highest risk. Risk axes that apply to almost any modern stack:
• Correctness of domain logic — math, state machines, business invariants
• Concurrency and ordering — races, deadlocks, lost updates, event ordering, retry/idempotency
• State and persistence — schema evolution, migration safety, backfills, consistency
• External boundaries — API contracts, third-party integrations, version skew, failure modes
• Security — authn/authz, input validation, secret handling, injection, privilege escalation
• Multi-tenancy / scoping — leakage between users, tenants, organizations
• Performance and scale — hot paths, N+1, unbounded growth, memory pressure
• Observability and operability — logs, metrics, traces, alerts, rollback safety
• User-facing behavior — UX flows, accessibility, internationalization, error states
• Test coverage and quality — what the tests claim vs. what they actually verify
• Propose 3–4 agents per hot area from different angles, 1–2 on lighter areas.
• Each scope is one line, two at most: — . Examples:
• "Authentication and session handling — token lifecycle, replay, timing-sensitive checks"
• "Schema changes — backwards compatibility and backfill correctness"
• "Payment flow — idempotency under retry and partial failure"
• "Frontend state management — derived state, stale closures, race conditions across async updates"
• Do NOT enumerate specific classes, methods, files, columns, line numbers, or expected outcomes in the scope.
• No "general" agents. Every agent has a sharp focus.
• Flag each agent as high-impact (yes/no) — areas where false negatives are worse than false positives. High-impact agents may always keep Likely/Possible findings regardless of the global policy.

Present the roster (name, scope, high-impact flag, one-line rationale) and confirm with the user. Iterate. In autonomous mode, the caller may have specified hotspots — derive the roster directly and skip confirmation.

---

STEP 4 — Co-design review affinities

Cluster the roster into 3–5 affinity clusters (e.g., "Domain Logic", "Integrations", "Frontend & UX", "Reliability", "Security & Access Control").

Clusters are orchestrator-side hints for Round 2 reviewer selection only. They are never shown to agents. They are not hard partitions — Round 2 assignment is dynamic and may cross clusters when scope demands it.

Design clusters per review. A domain-heavy review may cluster differently from a security-heavy one.

Present clusters with one-line rationale. Confirm (interactive mode) or proceed (autonomous mode). Write the final roster + clusters to ROSTER.md.

---

STEP 5 — Round 1: Discovery (all agents in parallel)

Launch every agent in a single parallel dispatch. On hosts with parallel sub-agent execution, this is one batch call. On sequential hosts, launch them one after another using the same prompt.

Selecting agent type / capabilities: if your host offers specialized sub-agent variants (e.g., one tuned for backend, one for frontend, one for QA/E2E), pick the closest match per scope. If your host has only a generic code-reading agent, use that with full tool access. The prompt below is what matters — it is the same regardless of variant.

Each agent's prompt (template — substitute the bracketed parts):

You are an Ultra Review agent. Your scope is narrow and focused.

THIS IS NOT A NORMAL REVIEW
Not a line-by-line review. Not the validation pass you usually run alongside an
engineer. You run solo, in parallel with many peers, against a large diff. Find
REAL problems — bugs, defects, risks, design flaws, edge cases that break under
load, data corruption paths, security gaps, anything that should not ship.
Skip nitpicks, style, surface-level checks. Spend effort on the highest-risk
areas of your scope. Round 2 reviewers will try to disprove your findings —
commit only to claims you can defend with concrete evidence.

YOUR SCOPE

OTHER AGENTS WORKING IN PARALLEL

CONTEXT
Read in order before starting:
1. /CONTEXT.md
2. /ROSTER.md

SHARED ENVIRONMENT

YOUR JOB
Find real problems in your scope. Your scope names an AREA, not a checklist —
the orchestrator deliberately did NOT pre-list classes, files, edge cases, or
outcomes. You are the domain expert. Drive your own investigation.

Phase 1 — Discovery (first): read the code and build a mental model. Identify
entry points, boundaries, invariants, dependencies. Locate the risky subareas:
where the math is hard, where state is mutated under concurrency, where
failures cascade, where assumptions about external systems live, what is new
vs. an existing pattern. Commit to deep-diving the two or three highest-risk
subareas.

Phase 2 — Deep dive (~80% of your effort): investigate the chosen subareas
thoroughly. Read every relevant path. Run queries against any available data.
Reproduce edge cases mentally or in a sandbox if you can. Verify external
assumptions via web research or domain-integration tools when available.

Phase 3 — Sanity scan (~20%): sweep the rest of your area lightly. Note
anything that looks off but does not merit a full deep dive.

Tools: use everything your host platform makes available. Read code, search
(grep / ripgrep / equivalent), spawn helper search sub-agents for broad
investigation, run web research, query the database read-only, query domain
integrations. Do not save tokens or shortcut investigation.

Be specific. Cite : for every claim. Quote code, query results,
and external sources directly.

OUTPUT
Write findings to: /round1/.md using the te

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [alexrolls](https://github.com/alexrolls)
- **Source:** [alexrolls/ultra-review](https://github.com/alexrolls/ultra-review)
- **License:** MIT

Install and usage instructions live in the source repository linked above.