Document Extraction

Document Extraction (ADE)

Overview

LandingAI's Agentic Document Extraction (ADE) is a document processing SaaS that parses, extracts, and classifies documents without requiring templates or training. The landingai-ade Python library is the recommended approach for most use cases. It wraps the REST API and handles authentication and response parsing for you.

ADE provides these core API functions:

| API | Python Method | What It Does | |-----|--------------|--------------| | Parse | client.parse() | Converts documents into structured Markdown, chunks, and metadata. Always the first step. | | Extract | client.extract() | Pulls specific fields from Markdown using a JSON schema. | | Build Extract Schema | client.extract_build_schema() | Generates or refines a JSON extraction schema from Markdown using AI. | | Split | client.split() | Classifies and separates multi-document batches by document type. | | Classify | client.classify() | Classifies each page in a document by type. Use to route pages before parsing. (Preview) | | Section | client.section() | Generates a hierarchical table of contents from parsed Markdown. (Preview) | | Parse Jobs (Create) | client.parse_jobs.create() | Creates an async parse job for large files (up to 6,000 pages). | | Parse Jobs (Get) | client.parse_jobs.get() | Retrieves the status and results of an async parse job. | | Parse Jobs (List) | client.parse_jobs.list() | Lists all async parse jobs with optional status filtering. | | Extract Jobs (Create) | REST API (no SDK method) | Creates an async extract job for long documents or large, complex schemas. | | Extract Jobs (Get) | REST API (no SDK method) | Retrieves the status and results of an async extract job. | | Extract Jobs (List) | REST API (no SDK method) | Lists all async extract jobs with optional status filtering. |

Key Benefits:

• No ML training or templates required
• Layout-agnostic parsing (works with any document structure)
• Supports 20+ file formats (PDF, images, spreadsheets, presentations)
• Precise visual grounding (bounding boxes, page numbers)
• Multiple models optimized for different document types

Quick Start

1. Installation

Never install packages globally without user approval. Always check for a local Python environment first.

1. .venv/bin/python       : uv-managed (this project)
2. venv/bin/python        : standard Python venv
3. uv run python          : if pyproject.toml exists
4. poetry run python      : if poetry.lock exists
5. python3                : system fallback; warn the user

Use the local environment to install: landingai-ade, python-dotenv

2. API Key Setup

The user may have already setup a .env file in the same directory as the document-extraction skill with the API key. You MUST check this path first (ls -la .*/skills/document-extraction/.env). Also try checking on the same directory as this SKILL.md file.

If not, provide instructions to create one. The script below will search for .env in common locations and load it.

.venv/bin/python - `

**EU Endpoint:** If using the EU endpoint, set `environment="eu"` when initializing the client.

### 3. Basic Parse Example

```python
from dotenv import load_dotenv
load_dotenv()  # Load API key from .env

from landingai_ade import LandingAIADE
from pathlib import Path

client = LandingAIADE()

# Parse a document
response = client.parse(
    document=Path("document.pdf"),
    model="dpt-2-latest"
)

# Access results
print(f"Pages: {response.metadata.page_count}")
print(f"Chunks: {len(response.chunks)}")
print("\nMarkdown output:")
print(response.markdown[:500])  # First 500 chars

# Save Markdown for extraction
with open("output.md", "w", encoding="utf-8") as f:
    f.write(response.markdown)

4. Basic Extract Example

from dotenv import load_dotenv
load_dotenv()

from landingai_ade import LandingAIADE
from landingai_ade.lib import pydantic_to_json_schema
from pydantic import BaseModel, Field
from pathlib import Path

# Define extraction schema using Pydantic
class Invoice(BaseModel):
    invoice_number: str = Field(description="Invoice number")
    invoice_date: str = Field(description="Invoice date")
    total_amount: float = Field(description="Total amount in USD")
    vendor_name: str = Field(description="Vendor name")

# Convert to JSON schema
schema = pydantic_to_json_schema(Invoice)

client = LandingAIADE()

# Extract from parsed markdown
response = client.extract(
    schema=schema,
    markdown=Path("output.md"),  # From parse step
    model="extract-latest"
)

# Access extracted data
print(response.extraction)
# Output: {'invoice_number': 'INV-12345', 'invoice_date': '2024-01-15', ...}

# Check extraction metadata (traceability)
print(response.extraction_metadata)

Document Parsing

Parse Local Files

from dotenv import load_dotenv
load_dotenv()

from landingai_ade import LandingAIADE
from pathlib import Path

client = LandingAIADE()

response = client.parse(
    document=Path("/path/to/document.pdf"),
    model="dpt-2-latest"
)

# Work with chunks
for chunk in response.chunks:
    print(f"Type: {chunk.type}, Page: {chunk.grounding.page}")
    print(f"Content: {chunk.markdown[:100]}...")

Parse Remote URLs

response = client.parse(
    document_url="https://example.com/document.pdf",
    model="dpt-2-latest"
)

Parse Spreadsheets

Spreadsheets (CSV, XLSX) return a different response type than documents. Key differences:

| Field | Documents (ParseResponse) | Spreadsheets (SpreadsheetParseResponse) | |---|---|---| | metadata.page_count | ✓ | ✗ (uses sheet_count, total_rows, total_cells, total_chunks, total_images) | | splits[].pages | ✓ | ✗ (uses sheets: array of sheet indices) | | grounding (top-level) | ✓ | ✗ (not present for spreadsheets) | | Chunk grounding | Always present | Optional (null for table chunks, present for embedded image chunks) |

response = client.parse(
    document=Path("data.xlsx"),
    model="dpt-2-latest"
)

# Spreadsheet metadata
print(f"Sheets: {response.metadata.sheet_count}")
print(f"Total rows: {response.metadata.total_rows}")
print(f"Total cells: {response.metadata.total_cells}")

# Splits use 'sheets' instead of 'pages'
for split in response.splits:
    print(f"Sheet indices: {split.sheets}")
    print(f"Markdown: {split.markdown[:200]}...")

Model Selection

• dpt-2-latest: Complex documents with logos, signatures, ID cards
• dpt-2-mini: Simple, digitally-native documents (faster, cheaper)
• dpt-1: ❌ Deprecated; migrate to dpt-2

Parse Large Files (Async)

For files up to 1 GB or 6,000 pages, use Parse Jobs:

import time
from dotenv import load_dotenv
load_dotenv()

from landingai_ade import LandingAIADE
from pathlib import Path

client = LandingAIADE()

# Step 1: Create parse job
job = client.parse_jobs.create(
    document=Path("large_document.pdf"),
    model="dpt-2-latest"
)

job_id = job.job_id
print(f"Job {job_id} created")

# Step 2: Poll for completion
while True:
    response = client.parse_jobs.get(job_id)
    if response.status == "completed":
        print(f"Job {job_id} completed")
        break
    print(f"Progress: {response.progress * 100:.0f}%")
    time.sleep(5)

# Step 3: Access results
# Results are in response.data (or response.output_url for large results)
if response.data:
    print(f"Chunks: {len(response.data.chunks)}")
    with open("output.md", "w", encoding="utf-8") as f:
        f.write(response.data.markdown)
elif response.output_url:
    # Results > 1MB are returned as a presigned URL
    print(f"Download results from: {response.output_url}")

Job Status Response Fields:

• job_id, status (pending, processing, completed, failed, cancelled), progress (0-1)
• data: The ParseResponse (or SpreadsheetParseResponse) when complete and result 1MB or when output_save_url was used. Expires after 1 hour; a new URL is generated on each GET.
• metadata: Same as sync parse (filename, page_count, duration_ms, etc.)
• failure_reason: Error message if job failed

Zero Data Retention (ZDR)

If ZDR is enabled for your organization, you must provide an output_save_url where parsed results will be saved. The results will not be returned in the API response. ZDR is not enabled by default. Typically output_save_url is a presigned url with write permissions to your S3 bucket, but you can also use other storage solutions that support file uploads via HTTP PUT requests.

job = client.parse_jobs.create(
    document=Path("sensitive_document.pdf"),
    model="dpt-2-latest",
    output_save_url="https://your-bucket.s3.amazonaws.com/output.json"
)

List Parse Jobs

List all async parse jobs with optional pagination and status filtering:

# List recent jobs
jobs_response = client.parse_jobs.list(page=0, page_size=10)
for job in jobs_response.jobs:
    print(f"{job.job_id}: {job.status} ({job.progress:.0%})")

# Filter by status
completed = client.parse_jobs.list(status="completed", page_size=5)
print(f"Completed jobs: {len(completed.jobs)}, more: {completed.has_more}")

Available status filters: pending, processing, completed, failed, cancelled

Understanding Parse Outputs

Parse returns a ParseResponse with:

• markdown: Complete document in Markdown with HTML anchor tags
• chunks: Array of extracted elements (each with unique ID, type, content, and per-chunk grounding)
• grounding: Dictionary mapping element IDs to detailed location data (page, bounding box, grounding type, and table cell position). See [JSON Response](#json-response) for structure.
• metadata: Processing info: filename, org_id, page_count, duration_ms, credit_usage (float), job_id, version, failed_pages
• splits: Array of split objects grouping chunks. Always present (contains a single "full" split by default, or per-page splits if split="page" was used). Note: Parse splits use a class field (values: "full" or "page"), which is different from the Split API's classification field.

Common chunk types: text, table, figure, logo, card, attestation, scan_code, marginalia

For detailed chunk type reference, see [references/chunk-types.md](references/chunk-types.md)

> Anchor tag prefix in chunk.markdown: Every chunk's markdown field > is prefixed with an HTML anchor tag embedding the chunk UUID: > \n\nActual content…. This is how the full document > markdown links back to individual chunks. Strip it before string matching, > display, or RAG indexing: > > ``python > import re > _ANCHOR_RE = re.compile(r"]*>\s*", re.IGNORECASE) > > def chunk_text(ch) -> str: > """Return clean chunk markdown without the anchor prefix.""" > return _ANCHOR_RE.sub("", ch.markdown or "").strip() > > # Example: fingerprint match against a section of the full markdown > intro_chunks = [ch for ch in response.chunks > if chunk_text(ch)[:80] in intro_markdown] > ``

Saving Parse Responses

The SDK provides a built-in save_to parameter on parse(), extract(), and split() (both sync and async clients) that saves the full JSON response to disk after the API call. It accepts two modes.

Directory mode (auto-generated filename):

from pathlib import Path

response = client.parse(
    document=Path("document.pdf"),
    model="dpt-2-latest",
    save_to="output/",  # Creates output/document_parse_output.json
)
print(response.markdown[:200])  # response is still returned for immediate use

The filename pattern is {input_filename}_{method}_output.json. If save_to can't derive a filename (for example, when markdown= is passed as a raw string), the file is named {method}_output.json.

Full-path mode (exact filename):

response = client.parse(
    document=Path("document.pdf"),
    save_to="output/parsed.json",  # Saves exactly to this path
)

If save_to ends in .json, the SDK writes to that path and creates any missing parent directories.

save_to is a client-side convenience: it does not change the API response, just writes it to disk.

> SDK version note: Directory mode is available from landingai-ade v1.4.0. The full-path mode and async save_to (on AsyncLandingAIADE.parse/extract/split) require v1.13.0 or later.

Saving just the Markdown field. save_to only writes the full JSON response. To save the Markdown string on its own (for example, to run Extract on it later), write it directly:

with open("document_parsed.md", "w", encoding="utf-8") as f:
    f.write(response.markdown)

For any other custom serialization, dump the full response with response.model_dump() rather than hand-picking fields, so you don't drop important data like the splits array or complete grounding information.

Parse Parameters

import json

response = client.parse(
    document=Path("document.pdf"),
    model="dpt-2-latest",
    split="page",                                          # Optional: organize chunks by page
    password="secret",                                     # Optional: decrypt protected files (ZDR only)
    custom_prompts=json.dumps({"figure": "YOUR_PROMPT"}),  # Optional: customize figure captions (DPT-2 only)
    save_to="output/",                                     # Optional: auto-save response JSON
)

Parse Password-Protected Files

Organizations with Zero Data Retention (ZDR) enabled can parse password-protected files by passing the password parameter. Supported formats: PDF, DOC, DOCX, ODT, PPT, PPTX, XLSX.

# Sync parse
response = client.parse(
    document=Path("encrypted.pdf"),
    password="document_password",
    model="dpt-2-latest"
)

# Async parse jobs
job = client.parse_jobs.create(
    document=Path("encrypted.pdf"),
    password="document_password",
    model="dpt-2-latest"
)

> Note: Without ZDR the API returns HTTP 422. If the password is wrong the API > returns HTTP 422 with a decryption error. The parameter is ignored for unencrypted documents.

Custom Prompts for Figure Descriptions

Use the optional custom_prompts parameter to control how ADE describes figures during parsing. Useful for domain-specific charts, standardized formats, or specific languages.

import json

response = client.parse(
    document=Path("document.pdf"),
    model="dpt-2-latest",
    custom_prompts=json.dumps({"figure": "Describe axis labels in detail."}),
)

Constraints: DPT-2 only (DPT-2 mini returns HTTP 422). Only figure key is supported. Max 512 characters. Must be passed as a JSON string via json.dumps (not a plain dict).

Structured Data Extraction

Schema Definition

Define what to extract using JSON Schema or Pydantic models.

Pydantic approach (recommended for Python):

from pydantic import BaseModel, Field
from landingai_ade.lib import pydantic_to_json_schema

class BankStatement(BaseModel):
    account_holder: str = Field(description="Account holder name")
    account_number: str = Field(description="Account number")
    beginning_balance: float = Field(description="Beginning balance in USD")
    ending_balance: float = Field(description="Ending balance in USD")

schema = pydantic_to_json_schema(BankStatement)

JSON Schema approach:

import json

schema = json.dumps({
    "type": "object",
    "properties": {
        "account_holder": {
            "type": "string",
            "description": "Account holder name"
        },
        "account_number": {
            "type": "string",
            "description": "Account number"
        }
    }
})

client.extract() requires schema to be a JSON string. Use json.dumps() when defining a schema as a Python dict. `pydantic_

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: landing-ai
• Source: landing-ai/ade-document-processing-skills
• License: MIT

Install and usage instructions live in the source repository linked above.