Llmtrim

llmtrim

llmtrim is a local proxy that compresses your LLM API requests so you pay less, with no change to the answers. It sits between your AI tools and the provider, strips the wasted tokens out of every request, and forwards it on. You get the same answers for a smaller bill.

−31% input and −74% output tokens, measured live across 112 A/B cases, with no change in answer quality.

One static binary. ~5 ms per call, no model to load.

Use it as a proxy, a CLI, an MCP server, or a library (Python · Ruby · Swift · Kotlin · JS · TS · WASM).

What it does • In action • Get started • CLI & library • Works with • Numbers • How it compares

---

What it actually does

You run Claude Code, Codex, Cursor, or your own app. Every time it talks to an LLM, it sends a big blob of text: your system prompt, the tool definitions, the whole conversation history, and the raw output of every command it ran. You pay for every one of those tokens, on every single turn.

A lot of that text is waste. A 200-line build log where only 2 lines are errors. A tool schema resent identically 50 times. A JSON array with 500 near-identical rows. The model doesn't need the bulk of it to answer well, but you're billed for all of it.

llmtrim removes the waste before it's sent. It installs as a local proxy that sits between your tool and the LLM provider. Requests pass through it, get compressed, and continue to the provider. The reply comes back unchanged. Your tool doesn't know it's there; you just get a smaller bill.

  before:  your tool ───── full request ─────▶  OpenAI / Anthropic / …
                    ◀──────── reply ──────────

  after:   your tool ──▶ llmtrim ──smaller──▶  OpenAI / Anthropic / …
                            (on your machine)
                    ◀──────── reply ──────────  (same answer)

> [!IMPORTANT] > It can never make your bill bigger or break a request. Every compression step is re-measured with the provider's real tokenizer; if a step doesn't actually save tokens, it's reverted. If the provider rejects the compressed request, the original is resent verbatim. Worst case is zero savings, never a worse outcome.

Everything runs locally. Nothing is ever sent to us.

See it on real output

Here's one real thing llmtrim does, end to end. An AI agent ran a build, and the bash tool returned a 58-line log. Only two lines matter (the errors), but all 58 get sent to the model and billed.

Before, what the model would receive (58 lines, 4,662 chars):

[2026-06-13T10:02:00Z] INFO  compiling module core::worker::task_0 (incremental)
[2026-06-13T10:02:01Z] INFO  compiling module core::worker::task_1 (incremental)
[2026-06-13T10:02:02Z] INFO  compiling module core::worker::task_2 (incremental)
... 27 more near-identical INFO lines ...
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
... 25 more INFO lines ...
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

After, what llmtrim sends instead (5 lines, 978 chars, −79%):

[{}] INFO compiling module core::worker::task_{} (incremental) [×30: (10:02:00Z..10:02:29Z step 1s; 0..29)]
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
[{}] INFO compiling module core::net::conn_{} (incremental) [×25: 10:02:32Z..10:02:56Z; 0..24]
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

Both errors and the summary survive verbatim. The repetitive INFO lines fold into a template plus their values, losslessly, because the range is regular (task_0..task_29). The model still sees exactly what happened; it just costs a fifth as much.

> If that's useful to you, a ⭐ helps other people find it.

Try it yourself on any request body:

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai

Log-folding is just one of ten compressors. A different one re-encodes bulky JSON arrays into a compact table, with the same data in a third of the tokens:

before:  [{"id":1,"city":"Paris","ok":true},{"id":2,"city":"Lyon","ok":false}, … 200 rows]
after:   [200]{id,city,ok}: 1,Paris,true; 2,Lyon,false; …          (TOON encoding, lossless)

Each compressor fires only where it pays:

| Where the waste is | What llmtrim does | |---|---| | Tool output (build logs, diffs, grep dumps, big JSON) | Keep the signal (errors, changes, matches), fold the noise | | Long context (pasted docs, history) | Rank and keep the chunks relevant to the question; drop the rest | | Source code | Keep the bodies of relevant functions, reduce the rest to signatures | | Tool schemas (resent every turn) | Trim descriptions, drop unused tools, keep the cache prefix stable | | JSON / record arrays | Re-encode to a compact table format, sample huge arrays | | The model's reply | Ask for terser output where it won't hurt the answer |

Full stage reference (all 10 compressors)

Stages run in savings order. Nothing under a cache_control marker is ever rewritten.

| Stage | What it does | When it runs | |---|---|---| | tool-output | Lossless template fold first, then window logs · diffs · grep · dumps down to errors / changes / matches | tool results | | cache discipline | Mark + stabilize the invariant prefix (sort tools/schema · OpenAI prompt_cache_key) so it stays cached | tools | | lexical retrieval | BM25+ ranking with RM3 feedback · TextTiling topic cuts · budgeted non-redundant selection; question protected | long context | | skeletonization | tree-sitter keeps relevant function bodies, drops the rest to signatures (14 languages) | code | | serialize + hygiene | Minify JSON, encode record arrays to TOON or CSV, Unicode-normalize | always · lossless | | json sample | Down-sample huge record arrays: first/last + outliers + a query-biased diverse sample | big JSON | | dedup | Collapse duplicate + near-duplicate lines (prose only) | always | | output control | Terse instruction · Chain-of-Draft · token budget · native JSON schema | auto | | tool layer | Static tool selection + description trimming | tools | | multimodal | Downscale images to the provider's resolution cap | images |

Default auto switches each stage on only where it pays. safe runs the lossless stages only. [Full config →](#configuration)

Get started

> [!NOTE] > Works with any tool that routes through HTTPS_PROXY: Claude Code, Codex, Cursor, Aider, your own app. GitHub Copilot pins its certificates and can't be intercepted ([full list](#works-with)).

# 1. Install (any OS, prebuilt binary, no Rust needed)
npm install -g @llmtrim/cli@latest && llmtrim setup

# 2. Open a new shell. Your AI tools now route through llmtrim automatically.

# 3. Watch the savings add up as you work
llmtrim status

No Node? Use an installer instead:

# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh

# Windows (PowerShell)
irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex

Or your own package manager, same binary everywhere: brew install fkiene/tap/llmtrim · cargo binstall llmtrim · scoop install llmtrim · docker run ghcr.io/fkiene/llmtrim. Full options in [INSTALL.md](INSTALL.md).

Is this safe to install?

setup is a local HTTPS proxy, the same technique as mitmproxy, scoped to LLM APIs. It changes exactly three things (a CA certificate in ~/.llmtrim/, a proxy setting in your shell profile, a login service), and llmtrim uninstall reverses all three. No API keys are stored (it forwards your tool's own auth), and your prompts never touch disk; only an anonymous count of tokens saved is kept. Full threat model: [SECURITY.md](SECURITY.md).

What gets installed, and how to verify the cert is harmless

1. A private certificate in ~/.llmtrim/, cryptographically constrained to LLM API domains. It cannot read your bank, email, or any other traffic, even if the key were stolen. Check that yourself:

``bash llmtrim ca # prints the cert path openssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints" # those domains are the only ones it can ever touch ``

1. A proxy setting in your shell profile (HTTPS_PROXY + NODE_EXTRA_CA_CERTS).
2. A background service that starts at login.

If the service stops, your tools fail fast with a connection error rather than silently bypassing it.

Day-to-day commands

llmtrim status      # health + savings dashboard (aliases: monitor, gain)
llmtrim doctor      # something off? end-to-end diagnosis; each check names its fix
llmtrim start       # start the background proxy
llmtrim stop        # stop it
llmtrim serve       # run in the foreground instead (Ctrl-C to quit)
llmtrim wrap claude # run an agent, guaranteeing this session routes through llmtrim (fails fast if it can't)
llmtrim update      # update to the latest release + restart
llmtrim uninstall   # exact inverse of setup: removes all three changes

llmtrim status --daily (or --weekly / --monthly) gives a time-series report; add --json or --csv to export.

Use it as a CLI, MCP, or library

The same compression runs with no proxy and no setup, as a one-shot CLI, an embeddable Rust crate, native bindings for Python, Ruby, Swift and Kotlin, or a WebAssembly module for JavaScript (browser, Node, Cloudflare Worker). No extra model calls, no network: the deterministic engine runs in your process.

| Language | Install | |---|---| | Rust | cargo add llmtrim-core | | Python | pip install llmtrim | | Ruby | gem install llmtrim | | Kotlin | implementation("io.github.fkiene:llmtrim:0.4.0") (Maven Central) | | Swift | .package(url: "https://github.com/fkiene/llmtrim-swift", from: "0.1.8") (SwiftPM) |

CLI. Pipe a request in, get a compressed one out:

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai > out.json
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim send     --provider openai   # compress, call, print

Rust. The engine is the llmtrim-core crate (no tokio, no network in its dependency tree):

use llmtrim_core::{compress, compress_with_config, config::DenseConfig, ir::ProviderKind};

// None auto-detects the provider from the request shape.
let out = compress(request_json, Some(ProviderKind::OpenAi))?;
println!("{} -> {} input tokens", out.input_tokens_before, out.input_tokens_after);

// …or pass an explicit preset/config:
let out = compress_with_config(request_json, Some(ProviderKind::OpenAi), &DenseConfig::preset("agent").unwrap())?;

Python / Ruby / Swift / Kotlin. One flat compress(input, provider, preset) call, generated from the same Rust engine via UniFFI. The compiled engine is bundled in each package, so there's no Rust toolchain to install:

import llmtrim

out = llmtrim.compress(request_json, llmtrim.Provider.OPEN_AI, "aggressive")
print(out.input_tokens_before, "->", out.input_tokens_after)

> [!NOTE] > Every binding returns the compressed request_json plus the before/after token counts, and maps errors to native exceptions. Per-language install and usage live in [crates/llmtrim-uniffi](crates/llmtrim-uniffi).

JavaScript / TypeScript (WebAssembly). The same compress(input, provider, preset) call, compiled to WebAssembly, runs in the browser, Node, Bun, Deno, or a Cloudflare Worker with no network or filesystem access. The output type is fully typed in TypeScript:

import { compress } from "@llmtrim/js"; // @llmtrim/wasm is an alias for the same package

const out = compress(requestJson, "openai", "aggressive");
console.log(out.input_tokens_before, "->", out.input_tokens_after);

> [!NOTE] > To stay small, the WASM build uses the estimate tokenizer: the absolute token counts are approximate, but the savings percentage is unaffected. Build and usage live in [crates/llmtrim-wasm](crates/llmtrim-wasm).

MCP server. llmtrim mcp speaks the Model Context Protocol over stdin/stdout, so any MCP client can compress payloads and read your savings without the proxy. It exposes three tools: llmtrim_compress (compress a full request body, honoring your ~/.llmtrim config like the proxy), llmtrim_compress_text (shrink one text blob, lossless), and llmtrim_stats (your savings ledger). Every call records to the same ledger, so MCP traffic shows up in llmtrim status.

llmtrim mcp install          # register with Claude Code (one command)
llmtrim mcp install --print  # or print the config to paste into any other client

The printed block is the standard MCP config; for a client you edit by hand it looks like:

{
  "mcpServers": {
    "llmtrim": { "command": "llmtrim", "args": ["mcp"] }
  }
}

Works with

Any tool that honors HTTPS_PROXY and an env-provided CA. That covers all 18 agents below plus any HTTPS_PROXY-aware CLI or SDK you build yourself:

| Tool | Works | Notes | |---|:---:|---| | Claude Code | ✅ | Prompt-cache discount stays intact | | Codex CLI | ✅ | | | Gemini CLI | ✅ | | | Cursor (IDE), Cline, Roo, Kilo Code | ✅ | VS Code extensions; set NODE_EXTRA_CA_CERTS for the Node host process | | Goose, OpenCode, Crush, Mux, Forge, OpenClaw, Pi/OMP | ✅ | CLI agents on standard provider hosts | | Qwen Code, Grok CLI, Kimi Code, Mistral Vibe | ✅ | Provider hosts ship in the llm_providers registry, intercepted out of the box | | Aider, any other HTTPS_PROXY-aware CLI | ✅ | | | Hermes, Droid (BYOK mode) | ✅ | Interceptable only when a direct provider key is configured; see [guide](HERMES.md) for Hermes | | Your own app / SDK | ✅ | Or call the [CLI / library](#use-it-as-a-cli-mcp-or-library) directly | | GitHub Copilot | ❌ | Certificate pinning blocks interception | | Warp, Devin | ❌ | Provider call is server-side; a local proxy never sees it | | Cursor Agent, Kiro | ❌ | Routes through a vendor gateway, not a standard provider host |

Prefer no proxy? Any MCP client (Claude Code, Cursor, custom agents) can call llmtrim directly as tools instead: run llmtrim mcp install, or see [CLI, MCP, or library](#use-it-as-a-cli-mcp-or-library).

Providers come from the llm_providers registry (OpenAI, Anthropic, Google, DeepSeek, Mistral, xAI, Moonshot, Zhipu, Qwen, OpenRouter, …) and update with it. Every non-LLM connection passes through untouched.

Configuration

Zero config needed. The default (auto) inspects each request and picks the right compressors for its shape: tool-heavy → agent, code → code, long context with a question → rag, otherwise → aggressive.

Three tiers cover almost everyone. To force one, set LLMTRIM_PRESET= or preset = "" in $XDG_CONFIG_HOME/llmtrim/config.toml:

| preset | for | | --- | --- | | auto (default) | routes each request to the right compressors; right for almost everyone | | safe | lossless input only: byte-faithful round-trip, no lossy stages | | aggressive | squeeze everything, accept lossy cuts (quality-gated) |

Advanced presets

auto composes these per request shape, so most users never set them directly. Pick one when you know your traffic and want to skip shape detection:

| preset | for | | --- | --- | | agent | tool-calling loops: prunes the tool block first-turn-only so the prompt cache stays warm | | code | coding turns: skeletonize and minify code, compress pasted l

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: fkiene
• Source: fkiene/llmtrim
• License: MPL-2.0
• Homepage: https://www.npmjs.com/package/@llmtrim/cli

Install and usage instructions live in the source repository linked above.