Agent Review Panel

Agent Review Panel v3.6.0

A multi-agent adversarial review system based on nine research foundations: ChatEval (ICLR 2024), AutoGen, Du et al. (ICML 2024), MachineSoM (ACL 2024), DebateLLM, DMAD (ICLR 2025), "Talk Isn't Always Cheap" (ICML 2025), CONSENSAGENT (ACL 2025), Trust or Escalate (ICLR 2025 Oral).

When NOT to Use This Skill

Do NOT trigger for these requests — they need single-agent handling or other skills:

• Single code review ("review this function for bugs")
• Quick sanity checks ("just a quick look before I push")
• Bug fixes ("fix the type errors", "fix the failing test")
• Peer review without multi-perspective signal ("peer review this doc")
• Code explanation ("what does this code do?")
• Deployment tasks ("deploy to staging")
• Addressing existing feedback ("address the PR comments")
• Skill improvement ("make this skill better") → use schliff
• Writing tests, READMEs, or documentation
• Asking for a single opinion ("what do you think?", "is this any good?")

The key signal is multiple independent perspectives — if the user wants one opinion, don't launch a panel.

Input

This skill takes as input one or more of: file paths to review, inline code/text in the conversation, a git diff or PR reference, or a plan/design document. It expects the user to specify (or let it auto-detect) what to review.

Dependencies

This skill depends on the Agent tool to launch parallel subagent reviewers and requires bash for context gathering (grep, file reads). All agents MUST use model: "opus". This includes VoltAgent specialist agents launched via subagent_type — always pass model: "opus" explicitly alongside subagent_type to override the agent's default model. Omitting it causes the launched agent to fall through to its own frontmatter-declared model (which may be sonnet or haiku), introducing cross-run reasoning variance. Knowledge mining reads from memory paths if they exist; if not available, it degrades gracefully — no hard dependency.

HTML report CDN dependencies (Phase 15.3 output file only): The generated review_panel_report.html loads Tailwind CSS, Chart.js, and — new in v2.15 — Prism.js from CDN for syntax highlighting in the Code Evidence sections of expandable issue cards. If the CDNs are unreachable, the HTML degrades gracefully: layout and text remain readable, charts show a placeholder, code blocks render as unstyled monospace.

Optional enhancement: When VoltAgent specialist agents are installed, the panel can use them instead of generic persona-prompted agents for stronger domain-specific reviews. See "VoltAgent Integration" section below.

This skill is scoped to multi-perspective adversarial review. For skill improvement requests, use schliff instead. For post-review plan updates, use plan-review-integrator. Supported versions: Claude Code v1.0+.

Examples

Example 1: Code review panel Input: "Do a review panel on src/auth/middleware.ts — I want multiple perspectives before merging" Output: Classifies as pure code → selects Correctness Hawk + Architecture Critic + Security Auditor + Devil's Advocate → gathers context → 4 parallel reviewers → 2 debate rounds → completeness audit → claim verification → supreme judge → writes review_panel_report.md

Example 2: Mixed content with deep research Input: "Deep review of our migration plan — it includes SQL and Terraform" Output: Classifies as mixed → adds Code Quality Auditor + Data Quality Auditor (SQL signal) + Reliability/SRE (infra signal) → runs web research for best practices → full panel → report with epistemic labels

Process Overview

Phase 1:    Setup                     → Identify work, pick personas, define criteria
Phase 2:    Data Flow Trace           → Trace critical path(s), document schemas [code only] (v2.14)
Phase 3:    Independent Review        → All reviewers evaluate in parallel (no cross-talk)
Phase 4:    Private Reflection        → Each reviewer re-reads source, rates own confidence
Phase 5:    Debate (rounds 1–3)       → Reviewers engage with each other + find new issues
Phase 6:    Round Summarization       → Distill resolved/unresolved points between rounds
Phase 7:    Blind Final               → Each reviewer gives final score independently
Phase 8:    Completeness Audit        → Dedicated agent scans for what the panel missed
Phase 9:    Verify Commands           → Run up to 5 reviewer verification commands (advisory)
Phase 10:   Claim Verification        → Verify all line-number citations against source
Phase 11:   Severity Verification     → Read actual code for every P0/P1, downgrade if overstated + web-verify external domain claims (v2.16.3)
Phase 12:   Verification Tier Assign  → Confidence draft (12a) + judge-advised refinement (12b)
Phase 13:   Targeted Verification     → Persona-matched agents dispatched per dispute point
Phase 14:   Supreme Judge             → Opus arbitrates everything including verification round
Phase 14.5: Post-Judge Verification   → Re-verify judge-introduced P0/P1 against ground truth (v3.2.0)
Phase 15:   Output Generation         → (parent) Three output files (all sequential: 15.1 → 15.2 → 15.3)
  Phase 15.1: Primary Markdown Report → Structured markdown summary (review_panel_report.md)
  Phase 15.2: Process History         → Full director's-cut log (review_panel_process.md)
  Phase 15.3: HTML Report             → Interactive dashboard (review_panel_report.html)

[Multi-Run mode (--runs N > 1): repeat Phases 2–15 with rotated personas, then:]
Phase 16:   Merge                     → Deduplicate, score stability, produce merged report (v2.14)

---

Phase 1: Setup

Identify the Work

The user provides: file paths, inline content, git diff/PR, or a plan/design doc. Collect full content, then run Context Gathering (below).

Classify content type (matters for persona selection):

• Pure code — only code files
• Pure plan/design — architecture docs, proposals, RFCs
• Mixed — plans with code snippets, SQL, or config
• Documentation — READMEs, guides, API docs

Review Mode Detection (v2.8)

Auto-detect review mode from content type. No user toggle.

| Content Type | Review Mode | Behavior | |---|---|---| | Pure code | Precise | Every finding MUST cite a specific file, line number, or code snippet. Findings without concrete evidence are demoted to [UNVERIFIED]. | | Pure plan/design | Exhaustive | Broader risk identification allowed. Findings may reference design sections or architectural patterns without line-number evidence. | | Mixed | Precise for code, Exhaustive for prose | Reviewers label each finding with its mode. Code findings without line citations are demoted. | | Documentation | Exhaustive | Same as plan/design. | | Subjective-quality deliverable (strategy report, marketing/creative copy, pitch deck, research synthesis) | Assessment | Goal is quality discrimination, not defect-finding. Score each axis 0–10 by subtract-points + veto, run the swap test, fact-check currency out-of-band, and control-validate the personas against a known-bad floor. See Assessment Mode below. |

The detected mode is injected into Phase 3 reviewer prompts and the judge prompt. Report header states the detected mode.

Detect Content Signals

Scan work for technology-specific signals (case-insensitive, 3+ keyword threshold). See references/signals-and-checklists.md for the full detection table and domain checklists. Signal detection only fires when auto-selecting personas.

Context Gathering

Run these steps before launching reviewers for file-path reviews. Skipping is the #1 cause of incorrect [CRITICAL] recommendations.

1. Sibling Directory Scan — From reviewed files' parent, scan for docs/,

README*, CLAUDE.md, config.py, package.json, etc. Read first 50 lines of each. If files are nested, scan both immediate parent and project root.

1. Reference Tracing — Scan for imports, config references, cross-file

references in comments, SQL table references, file path strings.

1. Safety Mechanism Discovery — Grep reviewed code + imports for: _valid,

_flag, _guard, _check, _mask, `/dev/null && \ WORKTREE=$(git rev-parse --show-toplevel) && \ BRANCH=$(git rev-parse --abbrev-ref HEAD)

# 2. Find the default branch (main or master) DEFAULTBRANCH=$(git symbolic-ref refs/remotes/origin/HEAD 2>/dev/null | sed 's@^refs/remotes/origin/@@') || DEFAULTBRANCH="main"

# 3. Find the branch point and count divergence MERGEBASE=$(git merge-base HEAD origin/$DEFAULTBRANCH 2>/dev/null) COMMITSBEHIND=$(git rev-list --count HEAD..origin/$DEFAULTBRANCH 2>/dev/null || echo "unknown")

# 4. List PRs/commits merged to main since branch point git log --oneline $MERGEBASE..origin/$DEFAULTBRANCH 2>/dev/null | head -20 ```

If commitsbehind > 0: Include a [STALE_BRANCH] warning in the context brief listing what was merged to main since the branch point. Inject into ALL reviewer prompts: "The code under review is {N} commits behind {defaultbranch}. These changes were merged since: {list}. Before claiming code or features are 'missing', check whether they exist on {default_branch} via git show {default_branch}:{filepath}."

If in a git worktree specifically (detected via git worktree list): add extra emphasis — worktrees are commonly used for isolated development and are especially prone to divergence from main.

Record in Context Brief: Add a "Codebase State" section with: branch name, commits behind main, key PRs merged since branch point, worktree status.

1. Knowledge Mining (tiered loading) — Mine local knowledge using a 3-tier

approach to minimize token waste while maximizing relevant context:

L0 — Index scan (~100 tokens each). Read only index lines and frontmatter description fields. Filter for relevance to the work under review.

• MEMORY.md — read index lines only (each ~150 chars). Match keywords from

the work's content type, domain, and technology signals.

• ~/.claude/skills/*/SKILL.md — read only the YAML description: field

(glob + grep for ^description:). Match against detected content signals.

• CLAUDE.md — always load (small, high-authority).

L1 — Summary scan (~500 tokens each). For L0-matched items only, read frontmatter + first paragraph to confirm relevance.

• Memory files (feedback_*.md, project_*.md) — read first 20 lines.

feedback_*.md files matching the review domain get automatic L2 promotion (past corrections are HIGHEST PRIORITY).

• Skill files — read the description: + ## When NOT to Use section.
• lessons.md — scan for lines matching review domain keywords.

L2 — Full content (no limit). Only for confirmed-relevant items from L1.

• Read the complete file for items that passed L1 relevance check.
• Typical yield: 3-8 files at L2 out of 50+ candidates at L0.

Deduplication: if the same insight appears in multiple sources, include only the most specific version (project > global > skill).

1. Web Research (deep research mode only) — Triggers when user requests

"deep review" or 5+ keywords from a signal group with no built-in checklist. Cap at 2 web searches. Tag findings with [WEB]. If the built-in domain checklist already covers the signal group, web research is skipped unless explicitly requested.

1. Context Brief — Compile into structured brief with sections: Codebase

State, System Documentation Found, Referenced Files, Safety Mechanisms, Knowledge Mining Results, Web Research Findings, Domain Checklist, Context Gaps.

1. User Confirmation — If significant context gaps exist or deep research

is available but not requested, ask before proceeding.

Select Personas with Agreement Intensity

If user specifies personas, use those. Otherwise select 4 from content-type sets:

For code/implementation:

1. Correctness Hawk (30%) — Bugs, logic errors, edge cases
2. Architecture Critic (50%) — Design patterns, coupling, extensibility
3. Security Auditor (30%) — Vulnerabilities, injection, auth gaps
4. Devil's Advocate (20%) — Challenges everything, proposes alternatives

For plans/designs (pure — no code):

1. Feasibility Analyst (60%) — Technical feasibility, timeline realism
2. Stakeholder Advocate (50%) — Business perspective, ROI
3. Risk Assessor (30%) — Failure modes, dependencies
4. Devil's Advocate (20%)

For mixed content (plans WITH code/SQL/config) — CRITICAL:

1. Feasibility Analyst (60%)
2. Code Quality Auditor (40%) — Line-by-line scrutiny of every snippet
3. Risk Assessor (30%)
4. Devil's Advocate (20%)

For documentation:

1. Clarity Editor (60%)
2. Technical Accuracy Reviewer (30%)
3. Completeness Checker (40%)
4. Devil's Advocate (20%)
5. Fresh-Reader Reviewer (50%) — reads the doc COLD as a newcomer with zero prior context: catches journey-narration ("now built", "corrected this round", dated probes, "was X — now Y", "superseded"), assumed prior knowledge / undefined jargon / internal nicknames, and feasibility-vs-built identity confusion — defects Clarity/Accuracy/Completeness routinely miss. Strongly recommended (replace Devil's Advocate) for any near-final delivery / feasibility / status / handoff document, especially one heavily iterated or mass-rewritten. Dispatch via the Fresh-Reader Delivery Reviewer agent.

For subjective-quality deliverables (Assessment mode — strategy / marketing / creative / research output): GENERATE 4 domain-tailored adversarial personas (do not reuse the generic sets — saturation comes from generic "is this good?" questions). Each must ask a question that exposes a difference a generic reviewer misses, and all four default to HIGH agreement-intensity (appreciative personas re-saturate):

1. Domain-Authenticity Critic (90%) — "does this ring true to a real practitioner/native of this space, or is it a plausible outsider's caricature?"
2. Executability / Operator (90%) — "could I act on this tomorrow? name the concrete next step, number, or channel — any vagueness = a castle in the air."
3. Decision-Maker / ROI (90%) — "would I fund or ship this? what's the measurable commitment, and is the claimed upside real or just baseline?"
4. Intent-Fidelity Critic (90%) — "does this serve the specific brief/brand/audience — or would it read identically for a competitor? (apply the swap test in Assessment Mode.)"

Tailor each archetype's wording to the deliverable's actual domain. Worked example — a China beauty-brand strategy report: 化妆品合规审查员 (regulatory, with veto) · 本土电商操盘手 (executability) · 比稿甲方 / ROI 审计官 (decision-maker) · 成分党品牌死忠 (intent fidelity). Reasoning strategies: adversarial simulation + first-principles.

After base selection, auto-add signal-detected personas (up to 6 total). Replace Devil's Advocate first if at cap (keep ≥1 DA if panel ≥4).

CRITICAL: If work contains ANY code/SQL/config snippets, always include Code Quality Auditor — the #1 cause of missed details in v1.

Reasoning Strategy Assignment (DMAD, ICLR 2025)

| Persona Type | Strategy | Injection | |---|---|---| | Correctness Hawk / Code Quality Auditor | Systematic enumeration | "Enumerate every code path, constant, edge case." | | Architecture Critic / Feasibility Analyst | Backward reasoning | "Start from desired outcome, trace backward." | | Security Auditor / Risk Assessor | Adversarial simulation | "Imagine you are an attacker. How would you break this?" | | Devil's Advocate | Analogical reasoning | "Compare to known failure patterns from similar projects." | | Stakeholder Advocate / Clarity Editor | First-principles | "Question every assumption from scratch." | | Fresh-Reader Reviewer | Cold first-read | "You have zero prior context. Read as

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: wan-huiyan
• Source: wan-huiyan/agent-review-panel
• License: MIT

Install and usage instructions live in the source repository linked above.