Figure Comparison

/figure-comparison

Generate a single self-contained HTML report (.lightcone/comparison.html) that places paper reference artifacts from work/reference/ on the left and the project's reproduced artifacts from results// on the right, with red flags wherever a counterpart is missing. Images are embedded as base64 so the HTML is portable. The helper script and intermediate manifest also live under .lightcone/ so they don't pollute the baseline results.

Setup

1. Confirm project root. Read astra.yaml in the cwd. If missing, ask:

> "I do not see an astra.yaml here. Please cd to the ASTRA project > and re-invoke."

Stop until resolved.

1. Confirm results exist. Default universe is baseline, unless

comparison-report.yaml names reproduced files under another universe or the user supplied a universe explicitly. Check ls results//. If the directory is missing or empty, ask:

> "I cannot find populated results under results//. Build the > universe first (lc run --universe or equivalent), then > re-invoke."

Stop. Do NOT attempt to run the pipeline yourself -- this skill is read-only over the build artifacts.

1. Locate the paper reference substrate. The user may have passed a

path. Resolve it in this order:

1. If the argument is a directory containing metadata.json,

document.md, figures/, or tables/, use that directory as the paper reference root.

1. If the argument is an arXiv source directory containing .tex files,

use it as source_root, and use its parent work/reference/ as the paper reference root when that parent exists.

1. If no argument was supplied, prefer lc-from-paper's layout:

• work/reference/source/ when arXiv TeX source exists. Use the TeX

files there for labels/captions and the parsed artifacts under work/reference/{figures,tables,metadata.json} for renderable reference files.

• work/reference/document.md plus

work/reference/{figures,tables,metadata.json} when no TeX source exists. This is the PDF + Docling fallback from lc-from-paper.

1. Only after lc-from-paper paths fail, look for a legacy unzipped arXiv

dir in cwd: a directory containing both a *.tex file and figure files (*.pdf, *.png, *.eps). Common names: paper_source/, arxiv_source/, *_Original_Paper/.

If no usable reference substrate is found, ask:

> "Where is the paper reference directory? In a lc-from-paper project this > should usually be work/reference/, containing document.md, > metadata.json, and extracted figures/ / tables/."

If only work/reference/paper.pdf exists, ask the user to run the PARSE phase first so Docling or the TeX parser populates work/reference/. Do not compare directly against a whole PDF.

Phase 1 -- Understand the paper's main results

Read, in this order:

1. Scoped comparison artifacts, if present.

• If comparison-report.yaml exists, treat it as the highest-priority

scope because it records what lc-from-paper actually compared. Use its outputs: entries, including type, priority, paper_value, reproduced_value, reference_file, reproduced_file, match, and notes when present.

• Else if targets/targets.md exists, treat it as the scope ledger. Use

only the targets it names, including out-of-scope notes, priorities, reference paths, expected values/trends, and output/spec-home pointers.

• If neither file exists, use the default paper-driven flow below and

build a best-effort report from astra.yaml plus work/reference/.

1. astra.yaml -- specifically the top-level description, outputs:,

and findings: if present. Use it to map scoped targets to output IDs and to harvest declared findings. Do not assume ASTRA outputs have a dedicated filename-hint field; result paths come from the output ID and the result resolver in Phase 2.

1. The paper reference substrate, in this order:

• Read work/reference/metadata.json when present. It is the primary

index for paper figures and tables; its paths are relative to work/reference/ and usually point into figures/ or tables/.

• If work/reference/source/ exists, grep its TeX files for

\includegraphics, \label{fig:...}, \caption{...}, and \begin{table} to recover labels/captions that metadata may have missed.

• If only work/reference/document.md exists, use the markdown plus

metadata.json as the source of captions, table text, and in-text numerical claims. This is the Docling/Pandoc fallback; preserve its line numbers and do not pretend it is TeX.

• Grep the abstract, results, and discussion sections of the TeX or

markdown source for in-text numerical claims that look like primary results -- typically a quantity with value + uncertainty (e.g. $X = a \pm b$ unit). Prefer values that astra.yaml's findings: already names; do not try to extract every number in the paper.

Do NOT read the paper wholesale. For long papers (>500 lines), read only the abstract, results, and discussion sections.

If the paper is large or has many sections and neither comparison-report.yaml nor targets/targets.md exists, delegate the figure / table / value enumeration to a single subagent with subagent_type="general-purpose" -- pass it the paper path, the output schema below, and ask it to return only the inventory. One subagent is enough; do not fan out. Multiple subagents would have to re-read the same file.

Phase 2 -- Build the comparison manifest

Produce a manifest in memory (you'll write it as JSON in Phase 3) with three sections: figures, tables, values. Each entry pairs a paper-side artifact with a project-side artifact.

Build entries in this priority order:

1. From comparison-report.yaml if present. One manifest entry per

outputs. item. Use type to route it to figures, tables, or values. Use reference_file as the paper-side path and reproduced_file as the project-side path when present. Preserve the report's paper_value, reproduced_value, match, and notes in the manifest so the HTML reflects the completed COMPARE verdict.

1. Else from targets/targets.md if present. One manifest entry per

in-scope target. Use each target's reference path under targets/, its expected values/trends, and its output/spec-home pointer. If the ledger marks a target out of scope, omit it from the HTML unless the user asked for out-of-scope targets too.

1. Else use the default paper-driven inventory. Enumerate figures,

tables, and values from astra.yaml plus work/reference/, and fall back to filename-stem similarity only when no scoped ledger exists.

For project-side result paths, resolve every output ID with this order:

• Use an explicit reproduced_file from comparison-report.yaml or an

explicit reproduced path/glob from targets/targets.md, if present and the file exists.

• Search for flat files at results//. with the

first suitable type-specific extension: images (.png, .jpg, .jpeg, .pdf, .eps), tables (.csv, .parquet, .md, .txt), values (.json, .yaml, .yml, .txt, .md).

• If still unmatched and no scoped ledger exists, fall back to filename-stem

similarity within results//.

• If no match is found, use project_path: null and render a red

NOT PRODUCED panel. Do not include unrelated result files; the report is target-driven when target/report files exist, and paper-driven otherwise.

For tables: use work/reference/metadata.json and work/reference/tables/ when present. If TeX source exists, capture the raw LaTeX of the tabular block and any \caption{...}. If only work/reference/document.md exists, capture the Docling/Pandoc markdown table or the extracted table artifact under work/reference/tables/. The project side is whatever artifact carries the same content -- typically a CSV / parquet / markdown file at results//.. If astra.yaml declares no matching output, use project_path: null. If the paper contains no tables at all, leave the manifest's tables list empty; the helper must omit the entire Tables section from the HTML in that case (no header, no "no tables" placeholder).

For values: each entry is {name, paper_value, paper_uncertainty?, project_value?, project_value_source?, paper_quote}. Pull paper_value from the in-text claim or astra.yaml's findings.*.paper_value. Pull project_value from astra.yaml's findings.*.replicated_value if present, otherwise from a scoped comparison-report.yaml entry or a flat result summary file at results//. that you can read statically. Never compute or re-derive values yourself. If no project value can be located statically, leave it null and flag in the HTML.

When comparison-report.yaml or targets/targets.md exists, the values list is scoped to that file. Otherwise, be exhaustive about values, not selective. A common failure mode is the values section ending up with only 1--3 entries, which makes the report feel thin. Aim for every numerical claim that the paper asserts and the project tracks. Concretely, harvest from:

• Every entry under findings: in astra.yaml -- one manifest entry

per finding, even when several findings share a parent quantity.

• The paper's abstract: every ± it reports.
• The paper's results and discussion sections: every fitted parameter,

every feature location ("dip near x = X₁", "peak at x = X₂"), every reported sample size after a specific cut, every bin width or step used as a result-defining choice, every reported accuracy / score / metric.

• Any explicit reproduction targets in astra.yaml's findings:.

It is fine to repeat one quantity in multiple manifest entries when the paper reports it under different conditions (preliminary vs. final, per-subset, per-bin median, per-method variant). Each condition is its own row. Feature locations are values too: encode "feature located at domain coordinate X" as {name: "", paper_value: "", paper_unit: ""}. Target ≥6 value entries on a typical paper. If you end up with fewer than 4, you are filtering too aggressively -- re-read astra.yaml's findings: and the paper's results section.

Phase 3 -- Generate the HTML

Use a small Python helper rather than embedding base64 inline through your tool calls -- multi-MB image base64 strings would balloon your context.

Use the existing .lightcone/ directory in the project root. Do not create directories in this skill. All three files this skill writes -- manifest, helper, and final HTML -- live there.

1. Write the manifest as JSON to

.lightcone/comparison_manifest.json. Schema:

``json { "project_name": "...", "paper_path": "work/reference/document.md", "scope_source": "comparison-report.yaml", "universe": "baseline", "results_path": "results/baseline", "figures": [ { "paper_label": "fig:main_result", "paper_caption": "...", "paper_path": "targets/main_result.pdf", "project_output_id": "primary_metric_plot", "project_path": "results/baseline/primary_metric_plot.png" } ], "tables": [ { "paper_label": "tab:summary", "paper_caption": "...", "paper_latex": "\\begin{tabular}{...}\\end{tabular}", "project_output_id": "...", "project_path": "results/baseline/summary_table.csv" } ], "values": [ { "name": "primary_metric", "paper_value": "12.5", "paper_uncertainty": "0.4", "paper_unit": "", "paper_quote": "we find $\\mathrm{metric} = 12.5 \\pm 0.4$ ", "project_value": "12.47", "project_uncertainty": "0.41", "project_value_source": "results/baseline/metric.json" } ] } ``

figures, tables, and values may each be []. Empty lists mean the helper skips that section entirely. There is no unmatched_baseline field -- baseline files the paper does not reference are not in scope for this report.

Use null for any missing field. Paths are relative to the project root.

1. Write the helper script to .lightcone/build_comparison.py.

The helper must:

• Read the manifest JSON.
• For each figure entry: emit one `` per figure,

with the structure described in "Required HTML structure" below -- a single ` containing a and one row-level status badge, followed by a of two s (paper, project). One badge per row, in flow inside .row-head. **Never emit per-cell absolutely-positioned badges.** Read paperpath and projectpath as bytes, base64-encode, and embed each image inside its cell. **PDFs must be converted to PNG before base64-encoding -- never embed PDFs as PDF data URIs.** Use ` uniformly for every figure cell. Conversion order to try, falling back if a tool is unavailable:

1. pdf2image (Python) -- convert_from_path(path, dpi=150)[0]
2. pypdfium2 -- render page 1 at 150 DPI to a PIL image
3. shell out to pdftoppm -png -r 150 -f 1 -l 1

and read the resulting PNG

1. shell out to magick [0] -density 150 (ImageMagick)

If none are available, the helper renders a small ⚠️ panel that says PDF preview unavailable -- install pdf2image or pdftoppm and links to the .pdf file path. Do not fall back to embedding the PDF binary. PNG / JPG inputs skip conversion and are base64-encoded directly. For any non-image type, embed as a UTF-8 text block. Missing path → render a red panel saying ❌ NOT PRODUCED with the expected output ID. Captions live as `` inside each cell, never as a row-spanning element.

• For each table entry: paper side renders the captured LaTeX inside

` plus the caption; project side renders the project file (CSV/parquet → first ~20 rows as an HTML table; markdown → render as `; missing → red ❌ panel). Same row structure as figures.

• For each value entry: emit one ``

per value -- same card layout as figures, not a `.** The row has a .row-head (value name + single status badge), a .row-grid of two .cells (paper | project), and a trailing .value-note with the σ delta. The paper cell shows the value (with uncertainty and unit) and the paperquote as a . The project cell shows the value and the projectvaluesource as a small line. Compute a simple status -- ✅ if both values exist and the project value lies within ±1 paper-uncertainty of the paper value; ⚠️ if both exist but disagree by more than that; ❌ if either is missing. If paperuncertainty is null, fall back to a 5%-tolerance comparison: ✅ if |prj − paper| ≤ max(0.05·|paper|, 0.05). Do NOT do anything more sophisticated; you cannot run code. **Do not render values as a single HTML ` -- the report's whole point is side-by-side cards.

• Emit a single self-contained HTML file with inline CSS in the

Vellum aesthetic (see below): the ` carries the parchment background and grain, and **all content lives inside a single that is the lighter --surface cream card with soft drop shadows.** This is non-negotiable -- the cream page card on top of the parchment body is the headline visual. Two content columns (paper | project) per row, the project name in the `, and a top-of-page summary line counting found / missing for each non-empty section. Skip any section whose manifest list is empty -- omit its header and content entirely; do not emit a "no tables found" placehold

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: LightconeResearch
• Source: LightconeResearch/lightcone-cli
• License: BSD-3-Clause
• Homepage: https://docs.lightconeresearch.org/

Install and usage instructions live in the source repository linked above.