Ocr And Documents

PDF & Document Extraction

For DOCX: use python-docx (parses actual document structure, far better than OCR). For PPTX: see the powerpoint skill (uses python-pptx with full slide/notes support). This skill covers PDFs and scanned documents.

Dependency Rule

Codex does not guarantee pymupdf, pymupdf4llm, marker-pdf, MinerU, python-docx, python-pptx, Docker, or curl are installed. Before using a local extractor, verify the command or import works in the current environment. If a dependency is missing, install it only when appropriate for the project/session, use another available extractor, or explain the degraded path.

---

Step 1: Check if Remote URL Available

If the document has a URL, try the current agent's available web, browser, or HTTP-fetch capability first:

https://arxiv.org/pdf/2402.03300
https://example.com/report.pdf

This handles PDF-to-markdown conversion with no local dependencies when the tool is available.

Only use local extraction when: the file is local, web extraction is unavailable/fails, or you need batch processing.

✅ CHECKPOINT 1: Did web extraction succeed?

• YES → You're done. Return the extracted content.
• NO (failed or file is local) → Proceed to Step 2.

---

Step 2: Choose Local Extractor

Decide between pymupdf and marker-pdf based on document characteristics:

| Feature | pymupdf (~25MB) | marker-pdf (~3-5GB) | |---------|-----------------|---------------------| | Text-based PDF | ✅ | ✅ | | Scanned PDF (OCR) | ❌ | ✅ (90+ languages) | | Tables | ✅ (basic) | ✅ (high accuracy) | | Equations / LaTeX | ❌ | ✅ | | Code blocks | ❌ | ✅ | | Forms | ❌ | ✅ | | Headers/footers removal | ❌ | ✅ | | Reading order detection | ❌ | ✅ | | Images extraction | ✅ (embedded) | ✅ (with context) | | Images → text (OCR) | ❌ | ✅ | | EPUB | ✅ | ✅ | | Markdown output | ✅ (via pymupdf4llm) | ✅ (native, higher quality) | | Install size | ~25MB | ~3-5GB (PyTorch + models) | | Speed | Instant | ~1-14s/page (CPU), ~0.2s/page (GPU) |

Decision rule: Use pymupdf unless you need OCR, equations, forms, or complex layout analysis. If you need OCR/complex layout, use MinerU only when the mineru CLI is installed and verified; otherwise use marker-pdf if installed/approved.

If the user needs marker capabilities but the system lacks ~5GB free disk: > "This document needs OCR/advanced extraction. MinerU or marker-pdf can handle it if installed; marker-pdf may require ~3-5GB. I can verify the available tools first."

✅ CHECKPOINT 2: Which extractor is appropriate?

• Text-based PDF, no OCR/equations needed → Use pymupdf (Step 3a).
• Scanned PDF, OCR, equations, or complex layout → Use verified MinerU (Step 3c) or marker-pdf (Step 3b).
• DOCX/PPTX/XLSX → Prefer dedicated document libraries/skills when available; use verified MinerU only if the local CLI supports the file type.
• Insufficient disk for marker-pdf? → Try verified MinerU or another installed document tool; otherwise explain the limitation.

---

Step 3a: Extract with pymupdf (lightweight)

pip install pymupdf pymupdf4llm

Via helper script:

python scripts/extract_pymupdf.py document.pdf              # Plain text
python scripts/extract_pymupdf.py document.pdf --markdown    # Markdown
python scripts/extract_pymupdf.py document.pdf --tables      # Tables
python scripts/extract_pymupdf.py document.pdf --images out/ # Extract images
python scripts/extract_pymupdf.py document.pdf --metadata    # Title, author, pages
python scripts/extract_pymupdf.py document.pdf --pages 0-4   # Specific pages

Inline:

python -c "
import pymupdf
doc = pymupdf.open('document.pdf')
for page in doc:
    print(page.get_text())
"

---

Step 3b: Extract with marker-pdf (high-quality OCR)

# Check disk space first
python scripts/extract_marker.py --check

pip install marker-pdf

Via helper script:

python scripts/extract_marker.py document.pdf                # Markdown
python scripts/extract_marker.py document.pdf --json         # JSON with metadata
python scripts/extract_marker.py document.pdf --output_dir out/  # Save images
python scripts/extract_marker.py scanned.pdf                 # Scanned PDF (OCR)
python scripts/extract_marker.py document.pdf --use_llm      # LLM-boosted accuracy

CLI (installed with marker-pdf):

marker_single document.pdf --output_dir ./output
marker /path/to/folder --workers 4    # Batch

---

Step 3c: Extract with MinerU (optional external CLI)

MinerU is an external CLI/Python environment, not a Codex built-in. Use it only after verifying it exists:

mineru --version

When installed, MinerU can handle PDF/images and may support DOCX/PPTX/XLSX depending on the installed version, producing structured Markdown with layout information.

基本用法：

# 自动模式（推荐）— 自动判断文本型还是扫描型
mineru -p document.pdf -o output_dir

# OCR 模式 — 扫描版 PDF 或图片
mineru -p scanned.pdf -o output_dir -m ocr -l ch

# 指定页范围
mineru -p document.pdf -o output_dir -s 0 -e 9   # 第 1-10 页

后端选择：

# pipeline 后端 — CPU 友好，通用（默认 auto 方法）
mineru -p document.pdf -o output_dir -b pipeline

# hybrid-engine — 高精度，需要本地算力（默认后端）
mineru -p document.pdf -o output_dir -b hybrid-engine

支持的文件格式：

• PDF（文本型 + 扫描型）
• 图片（PNG/JPG）
• DOCX、PPTX、XLSX

输出：output_dir/ 下生成 Markdown 文件 + 图片资源

适用场景：

• 扫描版 PDF（pymupdf 无法处理）
• 含公式/表格的学术论文
• 图片中的文字提取
• DOCX/PPTX 转 Markdown

注意：

• MinerU 需要单独安装和维护；不要假设 Codex、Claude Code 或任意 agent runtime 自带
• 首次使用 hybrid-engine 后端时可能需要下载模型
• 如无 GPU，建议用 -b pipeline 后端（CPU 可用）

✅ CHECKPOINT 3: Verify extraction output quality

• Output looks correct → Return the extracted content to the user.
• Output is garbled or missing text → If using pymupdf and document is scanned → Switch to marker-pdf (Step 3b). If marker-pdf produced poor output → Try --use_llm flag for higher accuracy.
• Tables/equations not preserved → If using pymupdf → Switch to marker-pdf which has better table and equation support.

---

Alternative: Local PaddleOCR

> ⚠️ 如果 MinerU 已安装且输出质量更好，优先用 MinerU（Step 3c）。 > 以下 PaddleOCR 配置仅作备用参考。

For on-premises OCR, deploy PaddleOCR as a Docker service. Docker and the image are external dependencies:

# docker-compose.yml
services:
  paddleocr-noavx:
    image: c403/paddleocr:2.2.2-noavx
    ports:
      - "5000:5000"

API:

curl -X POST http://localhost:5000/ocr -F "img=@image.png"

Note: Field name must be img, not file or image. Output is plain text only — no layout/formatting preservation.

---

Step 4: Arxiv Papers

For arxiv papers specifically, use these URL patterns:

# Abstract only (fast; use the agent's available web/browser/search capability)
https://arxiv.org/abs/2402.03300

# Full paper
https://arxiv.org/pdf/2402.03300

# Search
Search the web for: arxiv GRPO reinforcement learning 2026

---

Step 5: Split, Merge & Search

pymupdf handles these natively after pymupdf is installed — use the current agent's code execution capability or inline Python:

# Split: extract pages 1-5 to a new PDF
import pymupdf
doc = pymupdf.open("report.pdf")
new = pymupdf.open()
for i in range(5):
    new.insert_pdf(doc, from_page=i, to_page=i)
new.save("pages_1-5.pdf")

# Merge multiple PDFs
import pymupdf
result = pymupdf.open()
for path in ["a.pdf", "b.pdf", "c.pdf"]:
    result.insert_pdf(pymupdf.open(path))
result.save("merged.pdf")

# Search for text across all pages
import pymupdf
doc = pymupdf.open("report.pdf")
for i, page in enumerate(doc):
    results = page.search_for("revenue")
    if results:
        print(f"Page {i+1}: {len(results)} match(es)")
        print(page.get_text("text"))

No additional PDF library is needed beyond pymupdf for these operations.

---

Anti-Patterns

DO NOT do the following:

1. Don't use marker-pdf for simple text-based PDFs — It wastes ~3-5GB of resources and runs slower for no benefit. Always try pymupdf first unless you explicitly need OCR/equations/forms.

1. Don't use system python — Always use a virtual environment (e.g. python -m venv) for running extraction scripts. System python may lack dependencies or cause conflicts.

1. Don't skip web extraction for URLs — If a document has a public URL, try the current agent's available web/browser/HTTP capability first. It's faster, requires no local Python dependencies, and handles many PDFs well when available.

1. Don't assume all PDFs are text-based — If text extraction returns empty or garbled output, the document is likely scanned. Switch to marker-pdf or PaddleOCR.

1. Don't install marker-pdf without checking disk space — It requires ~5GB. Run python scripts/extract_marker.py --check before installing.

1. Don't use PaddleOCR for layout preservation — PaddleOCR returns plain text only with no formatting. Use verified MinerU or marker-pdf if layout/structure matters.

1. Don't use python-docx or python-pptx for PDFs — These are for Word and PowerPoint files only. For PDFs, use pymupdf or marker-pdf.

---

Notes

• Web extraction is the first choice for URLs when the current agent provides it
• pymupdf is the safe default when installed — fast, no model downloads, works for text-based PDFs
• MinerU is a strong OCR/complex document option only when installed and verified
• marker-pdf is an alternative OCR/layout extractor; install only when needed and after checking disk space
• Both helper scripts accept --help for full usage
• marker-pdf downloads ~2.5GB of models to ~/.cache/huggingface/ on first use
• MinerU CLI, when installed: mineru -p file -o output_dir -m auto -l ch (use -b pipeline for CPU-only)

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: montewaltrip188-hash
• Source: montewaltrip188-hash/claudecode-wiki-skills
• License: MIT

Install and usage instructions live in the source repository linked above.