Extrait

extrait

High-level LLM text generation and structured JSON extraction with validation, repair, and streaming.

Features

• Multi-candidate JSON extraction from LLM responses
• Automatic repair with jsonrepair
• Zod schema validation and coercion
• Optional self-healing for validation failures
• Streaming support
• MCP tools
• Vector embeddings (OpenAI-compatible + Voyage AI)

Installation

Install extrait with your preferred package manager.

bun add extrait
# or
npm install extrait
# or
deno add npm:extrait

Quick Start

openai-compatible targets the Responses API. Use openai-compatible-legacy for servers that only expose Chat Completions.

import { createLLM, prompt, s } from "extrait";
import { z } from "zod";

const llm = createLLM({
  provider: "openai-compatible",
  model: "mistralai/ministral-3-3b",
  transport: {
    baseURL: "http://localhost:1234/v1",
    apiKey: process.env.LLM_API_KEY ?? "local-demo-key",
  },
});

const text = "Beat two eggs, add flour and milk, then cook thin pancakes in a hot pan.";

const RecipeSchema = s.schema(
  "Recipe",
  z.object({
    title: s.string().min(1).describe("Short recipe title"),
    ingredients: s.array(s.string()).min(1).describe("Ingredient list"),
  })
);

const result = await llm.structured(
  RecipeSchema,
  prompt`Extract a simple recipe from this text: """${text}"""`
);

console.log(result.data);

Examples at a Glance

These examples cover the most common usage patterns in the repository.

• [examples/simple.ts](examples/simple.ts) - Basic structured output with streaming
• [examples/generate.ts](examples/generate.ts) - High-level text generation
• [examples/logprobs.ts](examples/logprobs.ts) - Token log probabilities and alternatives
• [examples/streaming.ts](examples/streaming.ts) - Real-time partial output and snapshot updates
• [examples/calculator-tool.ts](examples/calculator-tool.ts) - Structured extraction with MCP tools
• [examples/streaming-turns-with-tools.ts](examples/streaming-turns-with-tools.ts) - Streaming MCP turns, transitions, and reasoning blocks
• [examples/conversation.ts](examples/conversation.ts) - Multi-turn prompts and multimodal content
• [examples/image-analysis.ts](examples/image-analysis.ts) - Vision input with structured output
• [examples/embeddings.ts](examples/embeddings.ts) - Embeddings and similarity workflows

bun run dev simple "Bun.js runtime"
bun run dev generate "Bun.js runtime"
bun run dev streaming
bun run dev calculator-tool

API Reference

The sections below cover the main building blocks of the library.

Create an LLM Client

Use createLLM() to configure the provider, model, transport, and client defaults.

const llm = createLLM({
  provider: "openai-compatible" | "openai-compatible-legacy" | "anthropic-compatible",
  model: "gpt-5-nano",
  baseURL: "https://api.openai.com",       // optional alias for transport.baseURL
  apiKey: process.env.LLM_API_KEY,         // optional alias for transport.apiKey
  transport: {
    baseURL: "https://api.openai.com",     // optional
    apiKey: process.env.LLM_API_KEY,       // optional
    path: "/v1/responses",                 // optional provider endpoint override
    embeddingPath: "/v1/embeddings",       // optional embedding endpoint (openai-compatible only)
    headers: { "x-trace-id": "docs-demo" }, // optional extra headers
    defaultBody: { user: "docs-demo" },    // optional provider body defaults
    version: "2023-06-01",                 // anthropic-compatible only
    defaultMaxTokens: 4096,                // anthropic-compatible only (default: 1024)
    defaultMaxToolRounds: 10,              // optional cap on MCP tool rounds (default: 100)
    fetcher: fetch,                        // optional custom fetch implementation
  },
  defaults: {
    mode: "loose" | "strict",             // loose allows repair
    selfHeal: 1,                          // optional retry attempts
    debug: false,                         // optional structured debug output
    // or:
    // debug: { enabled: true, verbose: true },
    systemPrompt: "You are a helpful assistant.",
    timeout: {
      request: 30_000,
      tool: 10_000,
    },
  },
});

baseURL and apiKey at the top level are shorthand aliases for transport.baseURL and transport.apiKey. For request-specific options such as stream, request, schemaInstruction, and parse tuning, see the sections below.

Common setup patterns:

// OpenAI-compatible gateway or local endpoint with top-level aliases
const llm = createLLM({
  provider: "openai-compatible",
  model: "gpt-4o-mini",
  baseURL: process.env.LLM_BASE_URL ?? "http://localhost:1234/v1",
  apiKey: process.env.LLM_API_KEY ?? "local-demo-key",
});

// Chat Completions-only gateway
const legacy = createLLM({
  provider: "openai-compatible-legacy",
  model: "local-model",
  transport: {
    baseURL: "http://localhost:1234",
    apiKey: "local-demo-key",
  },
});

// Anthropic-compatible endpoint with explicit API version
const anthropic = createLLM({
  provider: "anthropic-compatible",
  model: "claude-3-5-sonnet-latest",
  transport: {
    baseURL: "https://api.anthropic.com",
    apiKey: process.env.LLM_API_KEY,
    version: "2023-06-01",
  },
});

Defining Schemas

Use the s wrapper around Zod for schema names, descriptions, and a more ergonomic authoring flow.

import { s } from "extrait";
import { z } from "zod";

const Schema = s.schema(
  "SchemaName",
  z.object({
    // String fields
    text: s.string().min(1).describe("Field description"),
    optional: s.string().optional(),
    withDefault: s.string().default("value"),

    // Numbers
    count: s.number().int().min(0).max(100),
    score: s.number().min(0).max(1),

    // Arrays
    items: s.array(s.string()).min(1).max(10),

    // Nested objects
    nested: z.object({
      field: s.string(),
    }),

    // Enums (use native Zod)
    category: z.enum(["a", "b", "c"]),

    // Booleans
    flag: s.boolean(),
  })
);

Making Structured Calls

structured() accepts a schema plus either a tagged prompt, a fluent prompt builder, or a raw message payload.

// Simple prompt
const result = await llm.structured(
  Schema,
  prompt`Your prompt with ${variables}`
);

// Multi-part prompt
const result = await llm.structured(
  Schema,
  prompt()
    .system`You are an expert assistant.`
    .user`Analyze: """${input}"""`
);

// Multi-turn conversation
const conversationResult = await llm.structured(
  Schema,
  prompt()
    .system`You are an expert assistant.`
    .user`Hello`
    .assistant`Hi, how can I help?`
    .user`Analyze: """${input}"""`
);

// With options
const result = await llm.structured(
  Schema,
  prompt`Your prompt`,
  {
    mode: "loose",
    selfHeal: 1,
    debug: true,
    systemPrompt: "You are a helpful assistant.",
    stream: {
      to: "stdout",
      onData: (event) => {
        if (event.delta.text) {
          console.log("New visible text:", event.delta.text);
        }
        if (event.delta.reasoning) {
          console.log("New reasoning text:", event.delta.reasoning);
        }

        console.log("Current visible text:", event.snapshot.text);
        console.log("Current reasoning:", event.snapshot.reasoning);
        console.log("Current structured snapshot:", event.snapshot.data);

        if (event.done) {
          console.log("Streaming done.");
        }
      },
    },
    request: {
      signal: AbortSignal.timeout(30_000),  // optional AbortSignal
      reasoningEffort: "medium",            // optional reasoning effort hint
    },
    timeout: {
      request: 30_000,  // ms per LLM HTTP request
      tool: 10_000,     // ms per MCP tool call
    },
  }
);

prompt() builds an ordered messages payload. Use `prompt... for a single string prompt, or the fluent builder for multi-turn conversations. The LLMMessage` type is exported if you need to type your own message arrays.

In stream.onData, the event is split into two layers:

• event.delta.text is only the newly received visible text since the previous event.
• event.delta.reasoning is only the newly received reasoning text since the previous event.
• event.snapshot.text is the full visible text accumulated so far.
• event.snapshot.reasoning is the full normalized reasoning accumulated so far.
• event.snapshot.data is the best structured JSON snapshot that can be parsed from the stream so far. It may stay unchanged while event.delta.text continues to grow.

Typical usage is:

• render event.delta.text directly to a terminal or chat UI
• optionally render event.delta.reasoning in a separate reasoning panel
• use event.snapshot.data to drive partial structured UI state
• use event.snapshot.text / event.snapshot.reasoning when you need the full accumulated state instead of only the latest increment

You can also pass provider request options through request:

const result = await llm.structured(
  Schema,
  prompt`Summarize this document: """${text}"""`,
  {
    request: {
      temperature: 0,
      maxTokens: 800,
      body: { user: "demo-user" },
    },
  }
);

Making Text Calls

generate() is the high-level API for non-structured generation. It accepts the same prompt shapes as structured(), but does not inject any schema or parse the output.

// Simple prompt
const result = await llm.generate(
  prompt`Write a short summary of ${topic}.`
);

// Multi-message prompt
const result = await llm.generate(
  prompt()
    .system`You are a concise assistant.`
    .user`Summarize: """${text}"""`
);

// Raw messages payload
const result = await llm.generate({
  prompt: {
    messages: [
      { role: "user", content: "Say hello in one sentence." },
    ],
  },
});

Streaming mirrors structured(), except the snapshot only contains text and reasoning:

const result = await llm.generate(
  prompt`Explain ${topic} in one short paragraph.`,
  {
    stream: {
      enabled: true,
      onData: (event) => {
        process.stdout.write(event.delta.text);

        console.log("Full text so far:", event.snapshot.text);
        console.log("Full reasoning so far:", event.snapshot.reasoning);

        if (event.done) {
          console.log("Streaming done.");
        }
      },
    },
  }
);

Provider request options and MCP tools still go through request:

const result = await llm.generate(
  prompt`Use tools if needed and answer the user clearly.`,
  {
    request: {
      temperature: 0,
      maxTokens: 800,
      reasoningEffort: "medium",
      mcpClients: [calculatorMCP],
      maxToolRounds: 10,
    },
  }
);

On openai-compatible, this is sent as reasoning: { effort }; on openai-compatible-legacy, as reasoning_effort. In both cases max is mapped to xhigh. On anthropic-compatible, this is sent as output_config.effort and auto-enables thinking: { type: "adaptive" }; the thinking content comes back in result.reasoning.

For existing history or multi-turn conversations, pass messages directly:

const messages = conversation("You are a helpful assistant.", [
  { role: "user", text: "What is the speed of light?" },
  { role: "assistant", text: "Approximately 299,792 km/s in a vacuum." },
  { role: "user", text: "How long does light take to reach Earth from the Sun?" },
]);

const result = await llm.generate({ prompt: { messages } });

Use llm.adapter.complete(...) or llm.adapter.stream(...) only when you need the raw low-level provider interface.

Token Logprobs

Set request.topLogprobs (0-20) to get token-level probabilities on openai-compatible and openai-compatible-legacy. The result exposes logprobs.content — one entry per generated token, each with its chosen token, logprob, and the requested number of alternatives.

const result = await llm.generate(prompt`Answer with one word: yes or no?`, {
  request: { topLogprobs: 3 },
});

for (const token of result.logprobs?.content ?? []) {
  const probability = Math.exp(token.logprob);
  console.log(token.token, probability, token.top_logprobs);
}

On openai-compatible (Responses API) this sends top_logprobs and automatically adds the message.output_text.logprobs include; on openai-compatible-legacy it sends logprobs: true + top_logprobs. Streaming accumulates logprobs across chunks. Anthropic has no logprobs API, so anthropic-compatible never returns them.

Some Responses-dialect servers (e.g. llama.cpp) also require the Chat Completions-style flag; pass it through transport.defaultBody: { logprobs: true }. See [examples/logprobs.ts](examples/logprobs.ts).

Images (multimodal)

Use images() to build base64 image content blocks for vision-capable models.

import { images, prompt } from "extrait";
import { readFileSync } from "fs";

const base64 = readFileSync("photo.png").toString("base64");
const img = { base64, mimeType: "image/png" };

// With prompt() builder — pass LLMMessageContent array to .user() or .assistant()
const result = await llm.structured(Schema,
  prompt()
    .system`You are a vision assistant.`
    .user([{ type: "text", text: "Describe this image." }, ...images(img)])
);

// With raw messages array
const result = await llm.structured(Schema, {
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Describe this image." },
        ...images(img),
      ],
    },
  ],
});

// Multiple images
const content = [
  { type: "text", text: "Compare these two images." },
  ...images([
    { base64: base64A, mimeType: "image/png" },
    { base64: base64B, mimeType: "image/jpeg" },
  ]),
];

images() accepts a single { base64, mimeType } object or an array, and always returns an LLMImageContent[] that spreads directly into a content array.

Conversations (multi-turn history)

Use conversation() to build a LLMMessage[] from an existing conversation history. This is the idiomatic way to pass prior turns to the LLM.

import { conversation } from "extrait";

const messages = conversation("You are a helpful assistant.", [
  { role: "user",      text: "What is the speed of light?" },
  { role: "assistant", text: "Approximately 299,792 km/s in a vacuum." },
  { role: "user",      text: "How long does light take to reach Earth from the Sun?" },
]);

// High-level text generation
const response = await llm.generate({ prompt: { messages } });

// Or to structured extraction
const result = await llm.structured(Schema, { messages });

Entries with images produce multimodal content automatically:

const messages = conversation("You are a vision assistant.", [
  {
    role: "user",
    text: "What is in this image?",
    images: [{ base64, mimeType: "image/png" }],
  },
]);

Result Object

Successful generate() calls return normalized text/reasoning plus request metadata:

{
  text: string,
  reasoning: string,
  attempts: GenerateAttempt[],
  usage?: {
    inputTokens?: number,
    outputTokens?: number,
    totalTokens?: number,
    cost?: number,
  },
  finishReason?: string,
  logprobs?: LLMLogprobs,          // when request.topLogprobs is set
  reasoningBlocks?: ReasoningBlock[], // per-turn reasoning for multi-round MCP streams
}

Each attempts entry includes:

{
  attempt: number,
  via: "complete" | "stream",
  text: string,
  reasoning: string,
  usage?: LLMUsage,
  finishReason?: string,
}

Successful structured() calls return validated data plus normalized text/reasoning and trace metadata.

{
  data: T,                      // Validated data matching schema
  text: string,                 // Visible model text, without inline  blocks
  reasoning: string,            // Normalized re

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [tterrasson](https://github.com/tterrasson)
- **Source:** [tterrasson/extrait](https://github.com/tterrasson/extrait)
- **License:** MIT
- **Homepage:** https://www.npmjs.com/package/extrait

Install and usage instructions live in the source repository linked above.