Pdf Extract

PDF-to-Markdown Extraction

Convert any PDF document into structured Markdown with extracted diagram/figure images (PNG) and a YAML manifest linking everything together.

Prerequisites

Required CLI tools (check before starting):

pdftoppm -v    # from poppler-utils
convert -version  # from imagemagick

If missing, tell the user to install: sudo apt install poppler-utils imagemagick

The Critical Lesson: Page Mapping First

The single biggest source of errors in PDF extraction is assuming page numbers match content structure. PDFs have cover pages, table-of-contents pages, multi-section pages, and appendices that shift everything.

You MUST build a page map before extracting anything. Read every page of the PDF to understand what content is on each page. This takes a few minutes but prevents hours of debugging wrong extractions.

Workflow

Step 1: Understand the Document

Determine from the user's request or from $ARGUMENTS:

• Source PDF path: Where the PDF is located
• Output directory: Where to write the extracted files
• Scope: The whole document, or specific pages/sections?
• Document type: Report, article, manual, form, presentation, etc.

If the user hasn't specified an output directory, create one next to the PDF (e.g., document_name/ alongside document_name.pdf).

Step 2: Build the Page Map

Read the PDF page by page using the Read tool (it handles PDFs natively). For large PDFs, read in batches of 15-20 pages.

Read the PDF: pages "1-15"

Verify the actual page count. Don't trust content like slide footers ("1 of 8") or table-of-contents page ranges — these can be wrong. Read the last few pages to confirm where the document actually ends. If pdftoppm fails on a page, the PDF doesn't have that page.

For each page, record:

• Page number (1-based)
• What section/chapter it belongs to
• What content is on it (headings, body text, figures, tables, appendices)
• Whether it contains images/diagrams that need extraction

Write the page map down before proceeding. Example:

Page 1: Title page
Page 2: Table of Contents
Page 3: Section 1 introduction (text only)
Page 4: Section 1 continued (FIGURE: bar chart of revenue)
Page 5: Section 2 (text + TABLE: quarterly results)
Page 6: Section 2 continued (FIGURE: process flow diagram)
Page 7: Appendix A (text only)

Step 3: Create Directory Structure

output_dir/
├── content.md           # Full document content in Markdown
├── manifest.yaml        # Structured metadata
├── extract_images.sh    # Reproducible extraction script
└── img/                 # Extracted figure PNGs

For longer documents with clear sections, you may split into multiple markdown files (e.g., section_1.md, section_2.md) — use your judgment based on document length and structure.

Step 4: Extract Text Content

Read each page and convert to Markdown. General rules:

• Headings: Map document headings to markdown headings (#, ##, ###)
• Lists: Use markdown lists (bulleted or numbered as appropriate)
• Tables: Use markdown tables. For complex tables, use HTML if needed.
• Math/formulas: Use LaTeX notation: $inline$ and $$display$$
• Emphasis: Preserve bold, italic, underline where meaningful
• Page breaks: Use --- between major sections if helpful
• Footnotes: Convert to markdown footnotes [^1] or inline notes
• Cross-references: Preserve as markdown links where possible

Preserve the document's logical structure. Don't just dump raw text — organize it so a reader (human or AI) can navigate the content.

Math-heavy documents (academic papers, textbooks):

• Use LaTeX notation consistently: $inline$ for inline, $$display$$ for

display equations

• Number equations with \tag{N} to match the source document's numbering
• Preserve theorem/definition/proof structure with bold labels
• Special functions (Gamma, Beta, hypergeometric) should use standard LaTeX

commands: \Gamma, \Beta, {}_2F_1, etc.

• If a document has no figures at all (common for pure-math papers), skip Steps

5-6 entirely and note images: [] in the manifest

Step 5: Extract Images and Figures

Only extract pages that contain actual figures, diagrams, charts, or images. Skip decorative elements, logos, and backgrounds unless the user specifically wants them.

The extraction process:

1. Extract full page as PNG at 300 dpi:

``bash pdftoppm -png -r 300 -f {PAGE} -l {PAGE} "$PDF" temp{PAGE} ``

Use a unique prefix per page (e.g., temp2, temp3, temp6) instead of a shared temp prefix. This prevents filename collisions if you extract multiple pages before cleaning up, and makes the ls + convert commands unambiguous.

pdftoppm page padding: The output filename uses zero-padded page numbers, but the padding width depends on the total page count of the PDF:

• PDFs with 1-9 pages: temp-1.png, temp-2.png (no padding)
• PDFs with 10-99 pages: temp-02.png, temp-14.png (2-digit)
• PDFs with 100-999 pages: temp-002.png, temp-014.png (3-digit)

Always check with ls temp*.png after extraction to see the actual filename before writing the convert command. This is a common source of bugs — don't guess the padding, verify it.

1. Determine crop coordinates. Read the extracted full-page PNG to see

exactly what's on it, then crop to the figure area: ``bash convert temp-{PAGE_PADDED}.png -crop {W}x{H}+{X}+{Y} +repage output.png ``

At 300 dpi, common page sizes are:

• A4: ~2480x3508 pixels (portrait)
• US Letter: ~2550x3300 pixels (portrait)
• Widescreen 16:9 slides: ~3996x2250 pixels (landscape)
• Standard 4:3 slides: ~3300x2475 pixels (landscape)

Use this to estimate where the figure sits on the page. Always verify by reading the full-page PNG first.

1. Verify the crop. Read the cropped image to confirm:

• The figure is fully visible (not cut off)
• All labels, legends, and annotations are readable
• No extraneous text from surrounding content is included
• If the crop is wrong, adjust and re-extract

1. If the crop cuts off content, make it LARGER, not smaller. Start generous

(e.g., full width, generous height) and tighten later. A too-large crop with some whitespace is far better than a too-tight crop missing labels or legends.

Naming convention for extracted images:

• Use descriptive names: fig1_revenue_chart.png, diagram_process_flow.png
• Include the figure number if the document has numbered figures
• Keep names lowercase with underscores

Step 6: Write Image References in Markdown

Use this three-part format to make images useful to both humans and AI:

> **Figure {N}** | Type: {chart/diagram/photo/illustration} | Source: PDF p. {page}
>
> **Content:** {What the figure shows — one or two sentences}
>
> **Key elements:** {List the important visual elements}
>
> **Labels/annotations:** {Any text, numbers, or labels visible in the figure}

The blockquote captures everything an AI agent needs to understand the image without seeing the pixels. Be thorough — list every labeled element and important visual detail.

Step 7: Write the Manifest

Create manifest.yaml with structured metadata:

document:
  title: "Document Title"
  type: report  # report, article, manual, form, presentation, etc.
  pages: 7
  language: en  # ISO 639-1 code

source:
  pdf: "path/to/source.pdf"
  pages: "1-7"
  page_map:
    - page: 1
      content: "Title page"
    - page: 2
      content: "Table of Contents"
    # ... one entry per page

files:
  content: content.md
  extraction_script: extract_images.sh

images:
  - id: fig1_revenue_chart
    file: img/fig1_revenue_chart.png
    page: 4
    type: chart  # chart, diagram, photo, illustration, map, screenshot, etc.

    content:
      summary: "Bar chart showing quarterly revenue growth from Q1-Q4 2024"
      elements:
        - id: "bars"
          type: data_series
          details: "Four grouped bars, one per quarter"
        - id: "y_axis"
          type: axis
          details: "Revenue in millions USD, range 0-50M"
      labels:
        - "Q1: $32M"
        - "Q2: $38M"
        - "Q3: $41M"
        - "Q4: $47M"

    extraction:
      source_page: 4
      resolution_dpi: 300
      crop: "2200x1200+150+1800"
      tool: "pdftoppm + imagemagick"

Step 8: Write the Extraction Script

Create extract_images.sh that reproduces the extraction. It should:

• Be idempotent (safe to run multiple times)
• Use variables for paths
• Clean up temporary files
• Print what it extracted

#!/bin/bash
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
IMG_DIR="$SCRIPT_DIR/img"
PDF="$SCRIPT_DIR/{relative_path_to_pdf}"
mkdir -p "$IMG_DIR"
cd "$IMG_DIR"

# Page {N} — {description of what's being extracted}
pdftoppm -png -r 300 -f {N} -l {N} "$PDF" temp{N}
ls temp{N}*.png  # verify filename before proceeding
convert temp{N}-{PADDED}.png -crop {geometry} +repage {output}.png
rm temp{N}-{PADDED}.png

echo "Extracted images:"
ls -la "$IMG_DIR"

Step 9: Verify Everything

1. Read back the generated markdown and check for formatting errors
2. Read the extracted images to verify they show the correct content
3. Validate the manifest references point to files that exist
4. Run the extraction script to confirm it's reproducible

Adapting to Document Types

Different documents need different treatment:

Reports and Articles

• Preserve section hierarchy faithfully
• Extract all figures and charts
• Keep tables in markdown format
• Preserve citations and references

Manuals and Documentation

• Pay special attention to numbered steps and procedures
• Extract screenshots and UI diagrams
• Preserve code blocks and command examples
• Keep cross-references intact

Forms

• Represent form fields as blank lines or placeholders
• Note field types (text, checkbox, dropdown)
• Preserve field labels and instructions

Presentations (slide decks exported to PDF)

• Each page = one slide; use --- between slides
• Slide diagrams often fill 60-80% of the slide area — start with generous

crops (e.g., 3800x1700 for a widescreen slide) and tighten if needed

• Tables on slides should be converted to markdown tables, not extracted as

images (tables render better as text for downstream use)

• Capture speaker notes if present
• Title slides and text-only slides don't need image extraction — focus on

slides with diagrams, charts, and infographics

Quality Checklist

Before declaring an extraction complete:

• [ ] Page map is written in the manifest
• [ ] Document structure is faithfully represented in markdown
• [ ] All figures/diagrams are extracted (not text-only pages)
• [ ] Each image is verified to show the correct content
• [ ] Crops include all labels and annotations with margin
• [ ] Image descriptions list all important visual elements
• [ ] Manifest includes extraction provenance for each image
• [ ] Extraction script is reproducible
• [ ] No content is missing or duplicated

Common Pitfalls

These are real issues encountered during testing — not hypothetical:

1. pdftoppm filename padding: The padding changes based on total PDF page

count. A 15-page PDF uses 2-digit padding (temp-04.png), but a 5-page PDF uses no padding (temp-4.png). Always ls temp*.png to check.

1. Crop coordinates need iteration: Your first crop estimate will often be

wrong. Read the full-page PNG, estimate, crop, verify, adjust. Budget 2-4 attempts per figure. Start generous and tighten.

1. Page numbers vs content structure: A "Figure 3" might be on PDF page 7.

The page map prevents this confusion.

1. Multi-figure pages: Some pages have multiple figures. Extract each one

separately with different crop coordinates.

1. Figures spanning page breaks: Occasionally a figure spans two pages.

Extract both pages and note this in the manifest.

1. Fragile relative paths in extraction scripts: If the output directory is

far from the PDF, deeply nested relative paths (e.g., ../../../../../...) are fragile. Prefer absolute paths or keep the output directory close to the source PDF.

1. Slide decks vs documents: Presentations have different dimensions (16:9

widescreen is common at 3996x2250 at 300 dpi). Each page = one slide. Diagrams often occupy most of the slide area, so crops tend to be larger relative to the page.

1. Trusting content for page counts: Slide footers ("1 of 8"), headers, and

table-of-contents entries can be wrong. A PDF that says "page 1 of 8" might only have 7 actual pages. Always verify by reading the actual last pages.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: pengguanya
• Source: pengguanya/claude-toolkit
• License: MIT

Install and usage instructions live in the source repository linked above.