Gh Gemini Review Loop

Gemini Code Assist PR Review Loop

Overview

Use this skill to run the full GitHub PR loop: after PR creation, wait for gemini-code-assist to finish reviewing, fetch unresolved actionable review threads, acknowledge the requested fixes, implement clear fixes, verify them, commit and push to the PR branch, and ask Gemini Code Assist to re-review the latest revision.

Prefer thread-aware review data over flat PR comments. GitHub review threads preserve isResolved, isOutdated, file paths, line anchors, and diff hunks, which are necessary for reliable automation.

Thread States

Each Gemini review thread is in one of these states. The fetch script tags each thread accordingly, and the stop logic consumes the tag:

• RESOLVED — Gemini or the maintainer has explicitly resolved the thread. Skip.
• OUTDATED — The line anchor has moved out from under the thread (the code Gemini commented on no longer exists at that position). Auto-resolved by the script. Skip.
• ADDRESSED_BY_REPLY — Unresolved, but the current user (or another maintainer) has posted a substantive reply (≥30 chars, not a bot, not a token "ack"). Treated as a human decision to defer/wontfix; the loop does not try to fix this thread again. The script auto-resolves these on the next pass via GraphQL (see "GitHub Write Safety" below). Opt out with --no-resolve-addressed-by-reply.
• UNRESOLVED — Actionable. Drives the next fix attempt.

A thread can transition UNRESOLVED → ADDRESSED_BY_REPLY → RESOLVED (reply + auto-resolve), or UNRESOLVED → fixed in code → OUTDATED → RESOLVED (line moves out, auto-resolved).

Cycle Counting

A cycle is one Gemini re-review request posted by the agent after Gemini's initial review.

• Cycle 0: Gemini's initial automatic review at PR open. Free; does not count toward the cap.
• Cycles 1–N: Each subsequent re-review request the agent posts, where N is max_rereview_requests from ~/.config/gh-gemini-review-loop/preferences.json or the default 3. After cycle N, hard stop.
• Replies posted via repos/.../pulls/comments/{id}/replies do NOT count as a cycle.
• Pushes to the PR branch without a re-review request do NOT count as a cycle.
• Only re-reviews posted by the agent itself count. A human pinging @gemini-code-assist does not consume a cycle. The script auto-detects the agent's GitHub login via gh api user; override with --agent-login NAME or opt out with --no-agent-filter.

The cap blocks new re-review requests and new fix cycles unless the user raises the cap. The cap does not block stale-thread cleanup, addressed-by- reply cleanup, metrics updates, terminal classification, final reports, or recording terminal state. Even at the cap, the script may still resolve outdated threads, classify fixed-pending findings, and print or record the final summary.

Severity Ordering

Gemini prefixes inline review comments with a markdown image whose alt text is the severity (critical / high / medium / low). The script parses this and orders actionable threads critical → high → medium → low → unknown, so high-severity findings are reported and fixed first. The severity tag also appears in the per-thread markdown header, e.g. ## 1. src/auth.py:42 [high].

Pattern → Sweep → Converge

Gemini is an LLM reviewer: when it flags a code pattern, fixing only the flagged sites teaches it to flag more instances of the same pattern in other changed files next cycle. To collapse that expansion into one cycle, each cycle runs:

Finding → Pattern → Sweep → Verify → Re-review

1. Cluster. The cycle receipt's Patterns (N): section groups findings by a

deterministic pattern signature. Reason about patterns, not a flat finding list.

1. Sweep (report-then-go). For each multi-site pattern (count >= 2), grep the

PR's changed files for sibling instances of the same shape — including ones Gemini has not flagged yet — using the cluster's example sites as the template. Print a short sweep report (which extra sites, why), then fix the whole cluster plus the swept siblings in this cycle. Do not block on approval, but never edit unflagged code silently — the report must appear first.

1. Mark. Pass --swept-pattern (the sig: token from the Patterns

receipt) for each pattern you swept, alongside the usual --fixed-finding markers, so the convergence advisory can detect recurrence.

1. Verify (the repo profile) and re-review as usual.

The receipt's Convergence: line is advisory only. When a swept pattern reappears ("⚠ … RECURRED after sweep"), the sweep missed a variant or Gemini keeps re-flagging — decide whether to refine the sweep, stop, or continue. It never changes control flow; the re-review cap remains the only hard stop.

Sweep scope is changed files only — that is both safe (blast radius = the PR's own diff) and sufficient (Gemini only reviews changed files).

Loop Receipt

Pass --post-receipt to leave a one-comment audit trail on the PR after the loop runs: cycles used, threads resolved (outdated + addressed-by-reply), threads still pending, and severity breakdown of remaining actionable threads. Use --dry-run --post-receipt to preview the receipt without posting.

Sticky receipt (background visibility)

For richer visibility while the loop is in flight, use --sticky-receipt instead. It maintains one comment per PR that the script edits in place across loop invocations. State persists in ~/.config/gh-gemini-review-loop/state.json (override with GGRL_STATE_DIR env var).

• First invocation: posts a fresh comment with status RUNNING, stores its id locally.
• Subsequent invocations: PATCH the same comment in place. No new comments accrete on the PR.
• Status header is configurable via --receipt-status {running,done,stopped}. Default is RUNNING for sticky receipts.
• Tag the final invocation with --receipt-status done (or stopped) so the user sees the loop has finished.
• Discovery fallback: if the local state file is missing, the script searches PR comments for the embedded marker and re-attaches to the existing receipt.

When --sticky-receipt is appropriate: long-running interactive loops where the user is watching the PR tab, not the chat. Always paired with --receipt-status running at cycle start, done at clean exit, stopped at stop-condition.

When the one-shot --post-receipt is appropriate: scripted/batch contexts where each invocation is independent and you want a fresh audit comment per run.

Optional Judge Eval (--judge-mode)

The Gemini review loop supports an optional OpenAI-based judge eval. It classifies each Gemini finding as one of valid_actionable / false_positive / duplicate / already_addressed / explanation_only / needs_human, plus a severity_override and recommended_action. The judge is read-only — it never resolves threads, posts comments, or pushes.

Judge eval is off by default. Nothing is sent to OpenAI unless the user explicitly opts in.

Requires an OpenAI API key resolved by key_resolver.py (env var → dotfile → macOS Keychain → Linux Secret Service). No SDK install needed — the judge uses stdlib urllib. Missing key → judge gracefully skips with a structured skipped result + one stderr hint. The loop continues unchanged.

If the user asks how to set their key permanently, recommend the OS-keystore path:

python3 "$GGRL_PLUGIN_ROOT/skills/gh-gemini-review-loop/scripts/key_resolver.py" --set

This stores in macOS Keychain (Touch ID / password-protected), Linux Secret Service, or a chmod-600 dotfile fallback. No rc-file edits, no ps leakage, no shell-restart dance. For diagnostics, suggest key_resolver.py --print-source and judge_doctor.py.

The script (fetch_gemini_threads.py) is the single source of truth. It reads ~/.config/gh-gemini-review-loop/preferences.json on every invocation and combines the saved mode with the --judge-phase the agent supplies.

Phase is auto-inferred. You do not need to pass --judge-phase: a terminal --record-run invocation is treated as phase complete; every other fetch is phase cycle. So on_cycle runs on every cycle fetch and on_complete runs only at the terminal record-run, with no per-cycle flag to remember. An explicit --judge-phase still overrides the inference.

When judge_mode=on_cycle, show the deterministic judge block every cycle. If the judge was requested but skipped, relay the script-owned line exactly:

[loop] judge eval skipped: 

Do not hand-write a replacement judge table or skip explanation unless the script fails.

Discoverability

Do not prompt for judge eval during a normal loop run. Do not prompt at session start.

One-time tip — after fetch, before fixes. On the first cycle where actionable findings are present, if judge_tip_shown is not true in the prefs file, emit this tip immediately after the findings narration line, then call mark_tip_shown() to persist judge_tip_shown: true:

[loop] cycle 1/ — 4 actionable thread(s) (high: 1, medium: 3). Fixing.
[loop] Tip: judge eval can give a second opinion on these findings.
         Try: "run the Gemini loop with judge eval at completion"

The tip fires at the moment the user is looking at real findings — before any fixes are applied. It appears exactly once across all future sessions.

Use README examples, --help output, and marketplace description for broader discoverability.

When to prompt

Prompt with the current runtime's choice-prompt mechanism only when the user explicitly requests judge eval without specifying a mode. In Claude Code this may be AskUserQuestion; in Codex use the available user-input flow or ask one concise question directly.

> "enable judge eval" / "use judge eval" / "turn on eval"

Prompt text:

> Judge eval sends Gemini findings and related PR context to OpenAI. > > Choose eval mode: > 1. Every cycle > 2. At completion only > 3. Just this once > 4. Off

Persist via save_preferences(). Mapping:

• 1 → save_preferences("on_cycle")
• 2 → save_preferences("on_complete")
• 3 → do NOT save — pass --judge-mode once --judge-phase complete for this run only
• 4 → save_preferences("off")

Preference file

~/.config/gh-gemini-review-loop/preferences.json

{
  "schema_version": 2,
  "judge_mode": "off",
  "judge_tip_shown": true,
  "max_rereview_requests": 3
}

The file is created automatically on the first script invocation with safe defaults (judge_mode: off, max_rereview_requests: 3). No manual setup is required.

Key fields:

• judge_mode — controls when the OpenAI judge eval runs. Valid values: off, on_complete, on_cycle. Set via natural language ("enable judge eval") or save_preferences(). Default: off.
• max_rereview_requests — persistent loop cap. Overridable per-run with --max-rereview-requests N. Default: 3.
• judge_tip_shown — internal flag. true after the one-time "judge eval is available" tip has been shown in chat. Set automatically; do not edit manually.
• judge_model — OpenAI model used for eval. Default: gpt-4o-mini.

max_rereview_requests sets the persistent loop cap. The script reads it from ~/.config/gh-gemini-review-loop/preferences.json on every invocation. The CLI flag --max-rereview-requests N overrides it for a single invocation.

To configure the persistent cap, create or edit:

mkdir -p ~/.config/gh-gemini-review-loop
python3 - ` checks.
Otherwise, test directories discovered in the git tree (`tests`, `__tests__`,
`spec`, …) are each mapped to their nearest package marker and emitted with a
per-check `working_directory`. Otherwise, root single-stack detection applies.
A check may carry its own `working_directory` (relative to the repo root);
`run_profile` runs it there, falling back to the profile-level
`working_directory`.

1. Run `detect_profile.py `. It returns `{stack, confidence, reasons,
   candidate_checks, presets}`. `presets` is an explicit, ordered, code-built
   option list — do **not** hand-roll the menu or rely on the runtime prompt UI
   auto-adding an option.
2. If `stack == "unknown"` (empty `candidate_checks`, empty `presets`) → do
   **not** prompt or persist; use ad-hoc verification.
3. Reconcile against repo docs (`CLAUDE.md`, `CONTRIBUTING`, `README`). If docs
   pin a non-standard invocation, surface it as a note beside the menu — *"Repo
   docs pin `/opt/homebrew/bin/pytest`; pick **Customize manually** to use it."*
   Never auto-persist an absolute path from prose.
4. Prompt once with the current runtime's choice-prompt mechanism, using each
   `presets[i].label` verbatim as an option. Example menu for a multi-check Python repo:
   *"All detected — pytest + ruff check ." / "Tests only — pytest" / "Skip — use
   ad-hoc verification" / "Customize manually"*.
5. Persist the chosen preset via `judge.save_profile(...)`:
   - Has `customize == true` (**Customize manually**) → run the free-form NL
     customize path; persist the user's edited checks with `source="customized"`.
   - Otherwise persist `preset["checks"]` with `source=preset["source"]`
     (`confirmed` for All detected, `customized` for a narrower preset, `skipped`
     for Skip). Every persisted check is `required: true`.

### Subsequent runs

A profile (including a `skipped` one) exists → **no prompt**. If `source` is
`confirmed` or `customized`, first render and relay the deterministic profile
intro block:

```bash
python3 "$GGRL_PLUGIN_ROOT/skills/gh-gemini-review-loop/scripts/fetch_gemini_threads.py" \
  --profile-intro \
  --repo OWNER/REPO

Then run run_profile.py — it prints to_details() JSON and exits non-zero if a required check failed; feed its verification into --verification and the JSON into --verification-details. On skipped or unknown stack, relay the formatter's fallback profile-intro block and use today's ad-hoc "narrowest meaningful checks".

Customizing / un-skipping

source="skipped" suppresses automatic detection prompts only. Explicit user intent always overrides it:

• "add mypy to this repo's verification profile" / "change the checks to X" →

edit the profile via save_profile(..., source="customized").

• "set up a verification profile for this repo" → re-enter the detect → preset

menu → save flow and overwrite the profile, even if a skipped marker exists.

Gate semantics

The verify step fails iff any required check fails or times out. Non-required failures are recorded in --verification-details but do not flip --verification to failed. Feed ProfileRunResult.to_details() into --verification-details and its verification field into --verification.

Before running checks, render and relay the deterministic planned-verification block:

python3 "$GGRL_PLUGIN_ROOT/skills/gh-gemini-review-loop/scripts/fetch_gemini_threads.py" \
  --planned-verification \
  --repo OWNER/REPO

Then run the profile runner. If no runnable profile exists, the formatter says so; continue with ad-hoc verification.

Run Metrics

After a loop completes, the script can append one JSON record to a local append-only log and print a one-screen summary.

What is stored and where

Records are appended to ~/.config/gh-gemini-review-loop/runs.jsonl (override the directory with GGRL_STATE_DIR). The file is append-only JSONL — one record per completed loop run. It is never transmitted anywhere.

Each record holds counts only: findings fetched, fixed, needs-human, addressed-by-reply, cycles used, verification result, outcome, duration (seconds), finding areas/paths, and optionally a judge-derived breakdown (only when judge mode was on). The record also includes the repo and PR number so stats can be scoped per repo.

No identity is recorded — no git author, no GitHub login, no username. The data can

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: OrenAshkenazy
• Source: OrenAshkenazy/gh-gemini-review-loop
• License: MIT

Install and usage instructions live in the source repository linked above.