# Paper Parser > Ingest an arXiv-style physics PDF and produce a structured paper record with extracted equations (LaTeX + sympy), figures (images + captions + inline claim quotes), and numerical claims with provenance. Uses Opus 4.7 vision for dense scientific diagrams. Run first in the DAG before physics-interpreter. - **Type:** Skill - **Install:** `agentstack add skill-leventilo-mobius-paper-parser` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [leventilo](https://agentstack.voostack.com/s/leventilo) - **Installs:** 0 - **Category:** [Content & Media](https://agentstack.voostack.com/c/content-and-media) - **Latest version:** 0.1.0 - **License:** MIT - **Upstream author:** [leventilo](https://github.com/leventilo) - **Source:** https://github.com/leventilo/mobius/tree/main/skills/paper-parser ## Install ```sh agentstack add skill-leventilo-mobius-paper-parser ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # paper-parser ## 1. Purpose and scope `paper-parser` is the entry node of the Mobius DAG. It reads a single PDF (arXiv preprint, journal article, or technical report) and produces the `paper` sub-tree of the canonical SimSpec — equations as LaTeX and sympy, figures as PNGs with captions and sub-panel structure, and numerical claims with verbatim source quotes and UCUM-normalized units. The skill does NOT: - reason about physics (that is `physics-interpreter`'s job — governing equations get tagged here, classified there) - compose simulation specifications (that is `simspec-author`) - verify claims against simulation output (that is `science-integrity` / `paper-diff`) - generate visualizations (that is `viz-mapper`) - emit Python primitives or simulator code (that is `primitive-generator`) Everything downstream of paper-parser assumes that the JSON returned here is correct, lossless within the limits documented below, and reproducible across re-runs given the same PDF bytes. ## 2. Input Required: - `pdf_path: str` — absolute path to a readable PDF file. Multi-column arXiv layout is the assumed common case. Optional: - `page_range: tuple[int, int] | None` — inclusive (start, end) page hint. Default scans the entire document. Useful when the orchestrator wants to skip references or supplementary material on a known-structured paper. - `focus_sections: list[str] | None` — section title hints (e.g. `["Methods", "Results"]`). Treated as a soft prior on which pages matter for claim extraction; never used to drop pages outright. The PDF is opened read-only. Nothing is written next to the input. All artefacts go to `/tmp/mobius_figures/` and `/tmp/mobius_parser_cache/`. ## 3. Output A JSON object matching the `paper` sub-tree of the SimSpec schema: ```json { "doi": "10.xxxx/...", "arxivId": "2501.12345", "title": "...", "authors": ["..."], "abstract": "...", "equations": [ { "id": "eq-1", "latex": "\\nabla \\cdot \\vec{E} = \\rho/\\epsilon_0", "sympy_repr": "Eq(Derivative(E, x) + ..., rho/epsilon_0)", "variables": ["E", "rho", "epsilon_0"], "page": 3, "bbox": [120.5, 410.0, 480.2, 442.7], "surrounding_context": "...prose ±200 chars around the equation...", "confidence": "high" } ], "figures": [ { "id": "fig-1", "png_path": "/tmp/mobius_figures//fig-1.png", "caption": "Verbatim caption text...", "subpanels": [ {"label": "a", "bbox_in_figure": [0.0, 0.0, 0.5, 1.0], "png_path": "..."} ], "inline_numerical_claims": [ {"value": 532.0, "unit": "nm", "location_hint": "panel (a) wavelength"} ], "page": 5, "confidence": 0.86 } ], "numerical_claims": [ { "id": "claim-1", "value": 1.4e-3, "unit_ucum": "kg.m-3", "value_uncertainty": 0.05e-3, "section": "Results", "source_quote": "the measured density was 1.4 x 10^-3 kg/m^3 +/- 0.05 x 10^-3", "claim_type": "measurement", "confidence": 0.92 } ] } ``` Field contracts: - `equations[*].latex` is the canonical form. `sympy_repr` is `str(sympy_expr)` and is null when parsing fails. - `equations[*].variables` is the list of free symbols recovered from sympy. When sympy parsing fails it falls back to a regex over `[A-Za-z_][A-Za-z0-9_]*` excluding LaTeX command names. - `figures[*].png_path` is always an absolute path. The figure plus each sub-panel are saved as separate PNGs. - `numerical_claims[*].unit_ucum` is a UCUM 1.9 string. When the original is unrecognised, the field carries the raw string and `confidence` is reduced. - `confidence` on equations is `"high" | "medium" | "low"`. On claims and figures it is a float in `[0, 1]`. ## 3.bis Output format (mandatory) You MUST emit your final output as a single fenced ```json block at the END of your reply, with NO prose after the closing fence. The orchestrator parses that block by regex (`/```(?:json)?\s*\n([\s\S]*?)\n\s*```/`) and ignores everything else in your text content. Code-execution `bash`/`text_editor` tool-result blocks are fine to interleave during work, but the LAST text block in the response MUST end with the canonical artifact fence. The canonical artifact for `paper-parser` (consumed downstream as `FiguresJson` in `server/src/types.ts`): ```json { "figures": [ { "id": "fig1", "png_path": "/mnt/user-data/outputs/fig1.png", "caption": "verbatim caption text", "page": 3 } ] } ``` The four fields (`id`, `png_path`, `caption`, `page`) are required for every figure. Additional fields documented in §3 (subpanels, inline_numerical_claims, confidence, bbox, equations, numerical_claims, doi, arxivId, title, authors, abstract) MAY be included alongside as enrichment — downstream skills tolerate them — but the four required keys per figure are non-negotiable. PNGs MUST be written to `/mnt/user-data/outputs/` so downstream phases can read them. If you cannot produce a complete artifact (encrypted PDF, no parsable pages, vision unavailable for a scanned scan), emit a fenced JSON block carrying a single `error` field: ```json { "error": "encrypted PDF; fitz.open raised PasswordError on page 1" } ``` DO NOT emit narration, summaries, or follow-up questions after the closing fence — they break the orchestrator's downstream consumption and get silently dropped. ## 4. Three-phase extraction The skill bundles three Python scripts. The orchestrator calls them in sequence; intermediate results are passed by value, no shared filesystem state required. ### Phase A — `scripts/extract_equations.py` PyMuPDF block-level pass: 1. Iterate pages in the requested range. 2. For each page, run `page.get_text("dict")` to obtain blocks with font metadata. 3. Hunt LaTeX delimiters (`$...$`, `\(...\)`, `\[...\]`, `\begin{equation}...\end{equation}`, `\begin{align}...\end{align}`) over reconstructed line text. 4. For each candidate, attempt `sympy.parsing.latex.parse_latex(latex, backend="antlr")`. On success, capture `expr.free_symbols`. On failure, keep the latex with `confidence: "low"` and continue. 5. For pages where step 3 finds nothing but font analysis hints at math (italic-heavy spans, glyphs from `Cambria Math`, `STIX`, `LMRoman` italic, or a span size more than 1.5x the body median), rasterize the page region and submit to Opus 4.7 vision for equation extraction. Vision fallback prompt (also a constant in the script): > "This is a region from a physics paper page that may contain governing equations. Identify every governing equation visible. For each, output {latex, location_description}. Skip inline variable definitions and footnote references. Output strict JSON array. No prose outside JSON." Deduplication: equations are canonicalized through sympy when possible (`sympify(expr)` or `srepr` equality); duplicates that appear in both prose and a numbered display block are merged, keeping the higher-confidence record and the larger bbox. ### Phase B — `scripts/extract_figures.py` 1. For each page in range, render at 200 DPI via `page.get_pixmap(dpi=200)`. 2. Submit the rasterized page to Opus 4.7 vision with a single tightly scoped prompt that asks for figure boxes, captions verbatim, sub-panel labels, and numerical annotations. The system prompt is a module constant; see `scripts/extract_figures.py`. 3. Crop each figure bbox from the high-DPI rendering and save as `/tmp/mobius_figures//fig-{n}.png`. 4. For figures with sub-panels, crop and save each sub-panel as `fig-{n}-{label}.png`. 5. Caption text is stored verbatim. Inline numerical claims found in the caption are stored on the figure record AND propagated into `numerical_claims` with `claim_type: "measurement"` and a back-reference to the figure id. ### Phase C — `scripts/extract_claims.py` 1. Concatenate body text (skipping References / Bibliography / Acknowledgments), all figure captions, and the surrounding context of every equation. 2. Chunk into ~4000-token segments with 500-token overlap. 3. For each chunk, call Opus 4.7 with a tight extraction prompt (see `scripts/extract_claims.py`). The prompt forbids extraction of pure formula symbols, page numbers, and years. 4. Merge chunk results, deduplicate by `(value, unit_ucum, source_quote[:50])`. 5. Normalize units via Pint loaded with UCUM-compatible definitions. When a unit cannot be parsed, keep the raw string and lower the confidence. ## 5. Vision invocation protocol All three phases use the same model and the same beta/no-beta surface: ```python from anthropic import Anthropic import base64 client = Anthropic() response = client.messages.create( model="claude-opus-4-7", max_tokens=4096, system=PHASE_SYSTEM_PROMPT, messages=[{ "role": "user", "content": [ { "type": "image", "source": { "type": "base64", "media_type": "image/png", "data": base64.b64encode(png_bytes).decode("utf-8"), }, }, {"type": "text", "text": PHASE_USER_PROMPT}, ], }], ) ``` Model identifier is locked to `claude-opus-4-7`. Adaptive thinking is implicit on Opus 4.7; do not pass `budget_tokens` (deprecated, returns 400). Temperature defaults are tuned per phase: 0 for equation OCR, 0 for figure detection, 0.2 for claim extraction. Phase-specific user prompts are stored verbatim as module-level constants in each script: - `EQUATION_VISION_PROMPT` in `extract_equations.py` - `FIGURE_VISION_SYSTEM` and `FIGURE_VISION_USER` in `extract_figures.py` - `CLAIM_SYSTEM_PROMPT` in `extract_claims.py` When the orchestrator updates a prompt, it bumps the cache namespace (see section 7) so stale results never come back. ## 6. Failure modes Scanned PDFs with poor OCR. PyMuPDF returns degenerate text blocks (no spans, glyph soup). Detection: a page where `len(page.get_text()) / equations.v1.json figures.v1.json claims.v1.json ``` Cache invalidation is manual: bump `PROMPT_VERSION` at the top of each script. A second invocation on the same PDF reads each phase's JSON in O(file-read) time. Vision cost on a re-run is exactly zero unless prompts changed. The figure PNGs sit alongside in `/tmp/mobius_figures//` — the JSON paths point there directly, so cache-served runs return paths that are still valid on disk. ## 8. Hand-off to physics-interpreter The output is the `paper` field of the SimSpec root. `physics-interpreter` reads it under that path: ``` simspec.paper.equations[*].latex simspec.paper.equations[*].sympy_repr simspec.paper.equations[*].variables simspec.paper.figures[*].png_path simspec.paper.figures[*].caption simspec.paper.numerical_claims[*] ``` The interpreter uses `equations[*]` to classify the PDE family, `figures[*].caption` to identify boundary conditions verbally described, and `numerical_claims[*]` to seed parameter values for the simulation. Nothing in the schema is optional from the interpreter's perspective; `paper-parser` is responsible for emitting at least empty arrays when a paper has no figures or no numerical claims. ## 9. Performance expectations Reference paper: typical arXiv physics preprint, 30 pages, 5 figures, 15 displayed equations, 50 numerical claims. - Phase A: ~10 s (pure PyMuPDF) + 1-2 vision calls x 8 s ~ 25 s total - Phase B: 5 figure-bearing pages x 8 s vision ~ 40 s - Phase C: 10 chunks x 6 s claim extraction ~ 60 s Total: under 2 minutes on a Managed Agents container. Token budget: roughly 40 k Opus tokens per paper, dominated by Phase B image input. Cache hit on a re-run: ~0.5 s end-to-end. When a paper exceeds these by more than 3x, the orchestrator should treat it as an outlier and route to a chunked re-parse with more aggressive page filtering. ## 10. Adversarial note — multi-panel figures on dense 2-column papers Counterexample known to break v1: figures with 3x3 sub-panel grids where the per-panel label sits in the upper-left corner in 6-pt sans-serif and the caption is split across the two columns of text below. On a single full-page rasterization at 200 DPI, the label glyphs are around 9 pixels tall — Opus 4.7 vision identifies the figure and extracts the joint caption, but assigns inconsistent panel labels (sometimes `(a)..(i)`, sometimes `(a)..(c)` repeated three times across rows). Confidence: this fails on roughly 1 in 5 dense biophysics / condensed matter papers in ad-hoc testing. v2 strategy (already wired into Phase B as a partial fallback): crop the figure bbox at 2x zoom and re-submit with a prompt that explicitly enumerates expected panel labels. If the label set still does not stabilise, fall back to per-panel cropping driven by horizontal/vertical white-space detection in the cropped figure (`scipy.ndimage.label` on a thresholded grayscale), then re-run vision per sub-crop. This is implemented for a single zoom step in v1 — full white-space-driven cropping is a v2 deliverable. The orchestrator should treat any figure record whose sub-panel labels are `?` or whose `confidence < 0.5` as a candidate for the v2 path, log it, and continue. This is the single largest source of low-confidence output on real arXiv corpora and is called out explicitly so downstream skills do not silently propagate ambiguous panel labels into the simulator's UI. ## Source & license This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [leventilo](https://github.com/leventilo) - **Source:** [leventilo/mobius](https://github.com/leventilo/mobius) - **License:** MIT Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/skill-leventilo-mobius-paper-parser - Seller: https://agentstack.voostack.com/s/leventilo - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.