Scientific Writer

Scientific Writer Loop

The artifact is a piece of scientific writing (draft + its dataset + figures + optional code). Each iteration critiques → grades → revises: five specialist judges produce concrete findings, an independent peerreviewer turns the paper into a graded 0-100 score on the same axes, and a scientificwriter fixes the prose, figures, and code — running the user's ` to regenerate figures. The loop runs until the score clears ` or the budget is hit. All work happens on copies inside a sandbox; the user's originals are never touched.

The cast (all in this folder):

• roles/figures_judge.md, roles/scientific_judge.md, roles/style_judge.md,

roles/formatting_judge.md, roles/code_reviewer.md — the five critics; each emits the shared schemas/finding.schema.json.

• roles/peer_reviewer.md — the summative grader (its own honesty rules); emits

schemas/peer_review.schema.json and decides pass.

• roles/scientific_writer.md — the reviser; fixes code → regenerates figures → updates prose.
• schemas/finding.schema.json, schemas/peer_review.schema.json — the two validated outputs.

Spawn-or-degrade. On Claude Code, spawn the active judges as real Agent subagents in parallel, then one fresh peer_reviewer, then the scientific_writer; otherwise adopt each role inline. You are the orchestrator.

Why the grader is built this way (the honesty problem)

The peer_reviewer grades on the same axes the judges critique — which invites echoing, inflation under loop-termination pressure, and a writer that games the rubric. roles/peer_reviewer.md counters this: it (1) grades independently, re-deriving each axis from the paper + dataset before reading the critiques, (2) verifies a sample of numbers/citations itself rather than trusting "it's fixed", (3) must surface issues the judges missed, (4) holds a fixed, anchored, reproducible bar with no credit for effort or elapsed iterations, (5) applies hard gates (a confirmed block fails the paper regardless of the average), and (6) runs a substance check against surface compliance. The writer optimizes the judges' concrete findings; the grader judges holistically — so "address every finding" does not mechanically buy a pass.

When to use

Use when a written scientific draft exists and the user wants it pushed past a quality bar with multi-judge critique and an independent peer-review grade. Default: run the full critique→grade→revise loop below. Escape hatch: if the user only wants the critique (no rewriting), run one round of judges

• peer_reviewer and stop. Not for writing a paper from a blank page, and not for a standalone

literature survey.

Setup

Resolve bindings interactively. If loop.run.yaml exists in the working dir, load it, confirm the values in one line, and skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write loop.run.yaml (format: examples/run.example.yaml) and confirm every value — including which axes are active (4 vs 5) and the live/degraded literature tier — before creating any other files.

| binding | meaning | default | how to infer | |---|---|---|---| | ` | the draft to improve (markdown/text/tex) | — | scan the working dir for a likely draft | | | data file(s) the claims/figures derive from — used to verify numbers | — | scan for data files near the draft | | | directory of figure images the draft references | — | scan near the draft; may be null | | | source files that produce plots/results; **empty → code axis dropped** (n_axes=4, no codereviewer) | — | scan for plotting/analysis scripts | | | command that regenerates figures/results from the code; **null → figures/code are edit-only** (flagged "needs regeneration") | — | pyproject.toml/.venv/README; e.g. python3 code/makefigures.py | | | the single style the formatting_judge enforces (APA\|MLA) | APA | ask the user | | | optional venue whose style/length norms apply | — | ask the user | | | optional abstract/paper word-count targets | — | ask the user | | | the paper's core finding/contribution, 1-2 sentences — **frozen**; the writer may never change it | — | read the draft, extract it, confirm with the user | | | overall_score (0-100) the peer_reviewer must reach (and no hard gate) to stop | 85 | a solid paper without demanding perfection | | | max iterations | 6 | — | | | stop after this many consecutive no-improvement iterations | 2 | — | | | where working copies, critiques, reviews, and the ledger live | ./sandbox` | — |

Literature toolchain. Citation grounding (writer) and verification (scientificjudge, peerreviewer) go through the sibling literature-search skill — resolve ` (it installs as a sibling, e.g. ~/.claude/skills/literature-search/), = python3, and = /tools/litsearch.py; append --cache-dir /literature/.cache after a subcommand to reuse the cache. Confirm --help works at setup; if the skill is absent, tell the user and either install it (copy the repo's loops/literature-search folder into ~/.claude/skills/) or degrade all retrieval to WebSearch/WebFetch. The keyless **S2 + arXiv** core works with no setup; a free S2API_KEY makes snippet/cite reliable — run keys --init, have the user fill the printed keys.env themselves, and **never paste secrets into chat**. Record the tier (presence only) in loop.run.yaml`.

Environment. The writer regenerates figures by running the user's ` **in the user's own environment** — that code may need third-party deps (matplotlib, pandas, …), so the skill **ships none** and never installs them; it shells out to the user's command and reads the regenerated outputs. Helper code the skill writes stays stdlib-only. tools/lit_search.py` (in the sibling skill) is stdlib-only too.

Initialise the sandbox once bindings are confirmed (copy in the originals; never edit them in place):

/
├── loop.run.yaml        ← resolved bindings +  + literature_tiers
├── ledger.tsv           ← header only (see Ledger)
├── literature/.cache/   ← lit_search on-disk cache
└── iter1/
    ├── draft.md         ← COPY of 
    ├── figures/         ← COPY of 
    └── code/            ← COPY of    (omit if no code)

The loop

`` starts at 1; iteration 1 critiques and grades the unmodified draft (the baseline — no revision before it). Re-grade fresh every iteration: the score comes only from a new peer review of the revised paper, never carried over. Surface-only changes won't move it.

Copy this checklist and tick items off:

• [ ] Critique — spawn the active judges in parallel (spawn-or-degrade), each over the iter/ working copies + dataset; each writes iter/critiques/.json (validates against schemas/finding.schema.json). Skip code_reviewer when no code.
• [ ] Grade — spawn one fresh peer_reviewer (roles/peer_reviewer.md): it grades independently (own read first, then reconcile with the critiques; spot-checks numbers/citations; surfaces missed issues), writes iter/peer_review.json (validates against schemas/peer_review.schema.json): 1-5 per active axis, overall_score = 100 × Σscore / (5 × n_axes), hard gates → pass.
• [ ] Log — append one ledger.tsv row (see Ledger).
• [ ] Stop check — peer_review.pass == true, or N == , or overall_score flat for `` iterations → stop (see Stops).
• [ ] Revise — spawn scientific_writer (roles/scientific_writer.md) with the critiques, the peer review, `, , , and . It fixes block/gate items first, fixes code → regenerates figures (running the copied **inside the sandbox**) → updates prose, grounds new citations via , and writes iter/{draft.md,figures/,code/} + iter/revision_notes.md`.
• [ ] N = N + 1 and repeat.

A judge finding and a peer_review look like (abridged; full shapes in schemas/):

{"reviewer": "code_reviewer", "iteration": 1, "overall": "block",
 "summary": "Block: make_figures.py sorts the two columns independently, fabricating r=0.98 (true r≈0.62).",
 "findings": [{"urgency": "must_fix", "action_type": "replace", "area": "make_figures:broken-pairing",
   "finding": "xs=sorted(study); ys=sorted(score) destroys per-row pairing; r inflated to 0.98 (real ≈0.62).",
   "proposed_action": "Correlate the paired arrays; re-render Fig 1; update r everywhere.",
   "target_artifact": "iter1/code/make_figures.py", "evidence": "code/make_figures.py:~70; paired r=0.62"}]}

{"iteration": 1,
 "axes": {"figures": {"score": 1, "justification": "Fig 1 plots bug-sorted data; numbers != data.", "blocking_issues": ["fig r=0.98 != paired r=0.62"]},
          "scientific": {"score": 1, "justification": "Causal claim from one correlation; r unreproducible."},
          "style": {"score": 2, "justification": "Promotional, AI-flavored prose."},
          "formatting": {"score": 2, "justification": "Mixed APA/MLA; Results before Methods."},
          "code": {"score": 1, "justification": "Independent column sort fabricates r=0.98."}},
 "overall_score": 28.0, "pass": false,
 "gate_failures": ["make_figures.py sort bug makes r=0.98 an artifact (true 0.62)"],
 "issues_judges_missed": ["Methods omits the test used and n."],
 "spotchecks": [{"target": "paired Pearson r", "method": "recomputed from data", "result": "refuted"}],
 "substance_check": "Baseline iteration — nothing revised yet."}

Ledger

/ledger.tsv, tab-separated, never commas in free text:

iter	overall_score	pass	figures	scientific	style	formatting	code	top_fix	revision_summary
1	28.0	no	1	1	2	2	1	fix make_figures sort bug	baseline (no revision)
2	61.0	no	3	3	3	3	4	report honest r, de-causalize	fixed bug+claims; relabeled fig1
3	88.0	yes	4	5	4	4	5	-	unified APA; trimmed abstract; balanced sleep claim

Use - in the code column when the code axis is absent. The per-iteration critiques/*.json, peer_review.json, and revision_notes.md live in iter/. Report the best-scoring iteration when stopping on budget/plateau, not necessarily the last. Leave the sandbox untracked.

Constraints

• Never edit or run anything outside `` — read the originals once at setup, copy

them in, and work only on the copies; the `` is copied in and run from the sandbox.

• Never fabricate data, numbers, figures, or citations — new citations come from real `` /

WebFetch retrievals, quoted verbatim; on {"error","fallback"}, fall back to WebSearch/WebFetch.

• The grading bar is fixed and reproducible — the peer_reviewer never relaxes anchors to let the

loop finish; a confirmed hard gate fails the paper regardless of the average.

• Protect `` — the writer strengthens the same finding; it never changes the core result

or deletes a real finding to dodge a critique, because removing a result to raise a score is gaming.

• One coherent revision batch per iteration, blocks/gates first, so score moves are attributable.
• No installs — the skill ships no deps; `` runs in the user's env, helper code is

stdlib-only. Never print or commit API keys (keys.env stays gitignored).

Stops

The loop stops on the first of:

• Pass — peer_review.pass == true. Report the deliverable (iter/ artifacts), the score, and

the trajectory.

• Budget — N == . Report the best-scoring iteration as the deliverable.
• Plateau — overall_score flat for `` iterations. Report the best iteration + the

standing gate_failures/must_fix blockers.

Always end with the deliverable (iter/ path), its overall_score and pass/fail, the per-axis scores, the score trajectory from ledger.tsv, and — if it did not pass — the standing blockers (gate_failures + open must_fix) between the paper and the bar.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: gaasher
• Source: gaasher/Agent-Loop-Skills
• License: MIT

Install and usage instructions live in the source repository linked above.