# Ctxprofile > Offline CLI that costs an LLM request per context component and flags tool definitions you ship but never call. - **Type:** MCP server - **Install:** `agentstack add mcp-evge14n-ctxprofile` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [Evge14n](https://agentstack.voostack.com/s/evge14n) - **Installs:** 0 - **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml) - **Latest version:** 0.1.0 - **License:** MIT - **Upstream author:** [Evge14n](https://github.com/Evge14n) - **Source:** https://github.com/Evge14n/ctxprofile ## Install ```sh agentstack add mcp-evge14n-ctxprofile ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # ctxprofile [](https://github.com/Evge14n/ctxprofile/actions/workflows/ci.yml) **An offline CLI that costs a single LLM request per context component — and flags the tool definitions you ship on every request but never call.** Your usage block gives you one input-token number. It does not tell you that four MCP servers put 46 tool schemas in front of every request, that they cost about $0.04 a call, or that the model never touched any of them. ``` pip install git+https://github.com/Evge14n/ctxprofile ctxprofile demo ``` That image is the output of that command, rendered from a real run by [a committed script](tools/render_terminal_svg.py) so it cannot drift from what the tool actually prints. Zero runtime dependencies, no API key, no proxy — it reads a captured request and does arithmetic. --- `ctxprofile` tells you *which part of the context* is spending the money: each individual tool schema, the system prompt, each history turn, each tool result — in tokens **and dollars** — from one captured request, with no proxy and no network. ``` model: claude-opus-4-8 input tokens: 900 exact $ (reconciled to usage) component kind tokens % $ cold $ cached write_file tool_def 238 26.4% 0.00119 0.00012 [UNUSED] web_search tool_def 224 24.9% 0.00112 0.00011 [UNUSED] read_file tool_def 183 20.3% 0.00092 0.00009 system system 85 9.4% 0.00043 0.00004 user[0] history 51 5.7% 0.00026 0.00003 assistant[1] history 41 4.6% 0.00020 0.00002 user[3] current_user 41 4.6% 0.00020 0.00002 tool_result[2] tool_result 37 4.1% 0.00018 0.00002 total $ (cold, uncached upper bound): 0.00450 2 tools not called in this capture ($0.00231 at the cold rate if they are never called): web_search, write_file ``` Half the input cost of that request is two tool definitions the model never used. You pay for them on every single call. **On real servers it gets worse.** Install four popular MCP servers — filesystem, memory, sequential-thinking, playwright — and before you type anything your request already carries **46 tool definitions, ~8,300 tokens, about $0.04 per call**. Tool schemas were 99.8% of that request's input, and a single tool (`sequentialthinking`) was the most expensive definition of all 46 — 6.6× the average filesystem tool. Measured, reproducible, and written up in **[docs/case-study-mcp.md](docs/case-study-mcp.md)**. **Across 19 published servers it is 186 tool definitions, ~52,000 tokens and $0.26 on every request** — a quarter of a 200K context window, spent before the user types. And two thirds of what those definitions ship is not language at all but the JSON structure around it, which is a measurement in characters that needs no tokenizer to believe. One server was 39% of that bill. The corpus and the method are in **[docs/mcp-tool-anatomy.md](docs/mcp-tool-anatomy.md)**. ## Install ``` pip install git+https://github.com/Evge14n/ctxprofile ctxprofile demo # runs on a bundled sample of four real MCP servers ``` Python 3.11+, zero runtime dependencies. A PyPI release is pending; after it, `pip install ctxprofile` will work directly. ## Capture real requests Point your Claude client at the built-in proxy and it tees every non-streaming `/v1/messages` request and response to a file — the input the rest of the commands read. Only the bodies are stored, so your API key never lands on disk. ``` ctxprofile capture --port 8787 --out captures/ # then, in your client: ANTHROPIC_BASE_URL=http://127.0.0.1:8787 claude -p "refactor auth.py" ``` Stdlib only, no certificate interception (plaintext localhost in, TLS out). Streaming (SSE) responses are reassembled into a normal message, so the merged `usage` is available for `analyze`. Captures hold plaintext prompts and output — treat the directory as secret. ## Use Capture one request/response pair as JSON (the raw Anthropic Messages API request, optionally with the response so costs reconcile to the real `usage`), then: ``` ctxprofile analyze capture.json ctxprofile analyze capture.json --json # machine-readable ctxprofile analyze request-only.json --model claude-sonnet-5 ctxprofile compare before.json after.json # $ delta of a prompt/tool change ``` The input is either a bare request, or `{ "request": {...}, "response": {...} }`. `compare` shows exactly what a change costs. Drop a dead tool and see it: ``` compare (A -> B) model: claude-opus-4-8 total $ (cold): 0.00450 -> 0.00350 (-0.00100, -22%) component Δ tokens Δ $ cold note web_search -224 -0.00112 removed ... dead tools: ['web_search', 'write_file'] -> ['write_file'] ``` ## Measure instead of estimating (optional) The offline split is a ~4 chars/token heuristic. `--online` replaces it with a real count for the two parts the API serializes independently of everything else — the system prompt and each individual tool definition — by measuring each one as a marginal against the `count_tokens` endpoint. That is the number the dead-tool figure is built on, so it is the one worth measuring. ``` pip install "ctxprofile[online]" export ANTHROPIC_API_KEY=... ctxprofile analyze capture.json --online ``` ``` model: claude-opus-4-8 input tokens: 900 exact $ (reconciled to usage) component kind tokens % $ cold $ cached write_file tool_def 234* 26.0% 0.00117 0.00012 [UNUSED] web_search tool_def 221* 24.6% 0.00111 0.00011 [UNUSED] read_file tool_def 180* 20.0% 0.00090 0.00009 system system 85* 9.4% 0.00043 0.00004 user[0] history 52 5.8% 0.00026 0.00003 assistant[1] history 42 4.7% 0.00021 0.00002 user[3] current_user 41 4.6% 0.00020 0.00002 tool_result[2] tool_result 38 4.2% 0.00019 0.00002 (tool block overhead) tool_def 7* 0.8% 0.00003 0.00000 * tokens measured with count_tokens (6 calls); unmarked rows are the ~4 chars/token estimate (tool block overhead) is what having any tools at all costs; removing one tool does not recover it on the rows it could check, the estimate ran 70% below the measured value — the unmarked rows carry that kind of error total $ (cold, uncached upper bound): 0.00450 2 tools not called in this capture ($0.00228 at the cold rate if they are never called): web_search, write_file ``` That is a real run of the command above on `examples/capture.json`, with the endpoint answered by the local stand-in from [tests/test_online_http.py](tests/test_online_http.py) — the token values are that stub's, not Anthropic's. With a key they come from the real tokenizer; the shape and the labelling are what you get either way. Each `*` tool row is that tool's **leave-one-out marginal** — what you stop paying if you delete it. The shared tool-block overhead is a separate row rather than being spread across the tools, because deleting a tool does not recover it; folding it in would overstate every "remove this and save" figure. Measuring part of a request also tells you something about the rest of it, so `--online` reports how far the heuristic was from the measured value on the rows it could check. That turns the estimate on the remaining rows from an unqualified guess into one whose error you know the size and direction of. Counting tokens is free but rate-limited, and this costs roughly one call per tool. `--online-max-calls` (default 64) refuses the run before spending anything if a request would exceed it. Message and history rows stay estimated — differencing them out would cost a call per turn for a number that changes every request anyway. The core CLI is unaffected: no key, no network, no dependencies. ## Find out what keeps breaking your cache A cache hit costs 0.1× and a rebuild costs 1.25×, so a prompt edit that lands high in the prefix is expensive in a way the bill never spells out. The usage block tells you a rebuild happened; it cannot tell you what caused it. Two consecutive captures can, because the cause is the first place their prefixes differ. ``` ctxprofile churn captures/*.json ``` ``` churn model: claude-opus-4-8 4 captures 1 rebuilds ttl assumed 5m when cause rebuilt excess $ 2026-08-09T11:43:32 system (system) 1840 0.01058 total excess over cache reads: 0.01058 ``` Appending a message is not a divergence — a conversation that only grew keeps its cached prefix, so the common case is correctly silent. When nothing changed, this refuses to invent a culprit: an unchanged prefix after longer than the TTL is reported as `ttl expiry`, and one inside the TTL as `unexplained`. Switching model gets its own label, because caches are model-scoped. The excess column is the honest number: what the rebuild cost *above* what a hit would have cost, not the whole write. Needs captures that carry `captured_at` and a `usage` block — which is what `ctxprofile capture` writes. ## Lock the context floor and gate it in CI Treat your static context — the system prompt and the tool set you ship on every request — like a lockfile. Commit a baseline, and let CI fail any PR that raises it or adds a tool without a deliberate re-lock. ``` ctxprofile lock --from captures/agent-turn.json # writes .ctxprofile.lock ctxprofile ci --lock .ctxprofile.lock captures/*.json ``` ``` FAIL captures/agent-turn.json - static floor +410 tokens over lock (allowed +0) - new tool(s) not in lock: web_search ``` Accepting a rise is a deliberate one-line commit: re-run `ctxprofile lock` and commit the new `.ctxprofile.lock`. You can also gate absolute cost and dead tools with a budget: `ctxbudget.toml`: ```toml max_input_tokens = 20000 max_total_usd_cold = 0.05 max_dead_tools = 0 ``` Drop either into GitHub Actions; it exits `1` on any breach, so the step blocks the PR: ```yaml - run: ctxprofile ci --budget ctxbudget.toml --lock .ctxprofile.lock captures/*.json ``` Add `--format github` to print a markdown summary you can post as a PR comment. Honest scope: the lock covers the **static floor** — the system prompt, the tool definitions, and the set of tools you ship — which lives in your repo. It does not lock per-request cost (RAG chunks, history, tool results); that is not in the repo and moves on every call. The gate fires on the delta of a stable estimator, so the estimator's bias cancels. ## Audit MCP-server tool bloat across a corpus Single-request dead-tool detection is noisy — one request calls a couple of tools and everything else looks idle. The signal is a **corpus**: point `mcp-audit` at your tool definitions plus a set of `claude -p` / Agent SDK JSONL traces, and it groups the wasted tokens by MCP server. ``` ctxprofile mcp-audit --defs request.json --traces runs/*.jsonl ``` ``` mcp-audit model: claude-opus-4-8 window: 3 model calls server tools tok/req calls $/req cold mcp__ruflo 2 130 0 0.00065 mcp__files 1 49 2 0.00024 (local) 1 40 1 0.00020 mcp__ruflo: 130 tok/req, 0 calls in 3 model calls — $0.00065 shipped every request [window: 3 model calls — a rarely used tool can look unused in a short window] ``` `mcp-audit` takes `--online` too, which measures each tool schema instead of estimating it — the same flag and the same `--online-max-calls` guard as `analyze`. `--defs` is a raw request (it carries the tool schemas, so the per-tool token estimate is grounded in the real schema text); `--traces` are SDK or `claude -p --output-format stream-json` transcripts (they carry the call counts). It reports a call **rate over a stated window**, never a boolean "dead" — a tool used once a week looks unused in a short trace. ## Use it from an agent (MCP server) ctxprofile ships an optional MCP server, so a Claude agent can profile its own requests. Install the extra and register it: ``` pip install "ctxprofile[mcp]" ``` Then add it to your MCP client (e.g. Claude Code): ```json { "mcpServers": { "ctxprofile": { "command": "ctxprofile-mcp" } } } ``` It exposes two tools — `analyze_request` (cost a captured request per component) and `audit_mcp_servers` (dead-tool cost by server). The core CLI stays dependency-free; only this optional server needs `mcp`. ### Measure your own MCP servers The same extra ships a collector that connects to the servers in your MCP client config, reads their real tool schemas, and writes a request you can analyze: ``` python tools/collect_mcp_tools.py your-mcp-config.json -o mine.json ctxprofile analyze mine.json ``` That is how the numbers in the [case study](docs/case-study-mcp.md) and the [anatomy survey](docs/mcp-tool-anatomy.md) were produced. To see what your own definitions are *made of* rather than what they cost: ``` python tools/tool_anatomy.py mine.json ``` ## What it does - **Per-component dollars.** Splits the request into `system`, one row **per named tool**, `history`, `tool_result`, and `current_user`, and prices each with the standard Claude rates. Existing tools count tokens; the cost is what you actually pay. - **Dead-tool detection.** Any tool defined in `tools[]` that never appears in an assistant `tool_use` block is flagged `[UNUSED]`, with the dollars it wastes on every request. Removing an unused tool is often the single cheapest large saving in an agent setup. - **Cold vs cached.** Shows each component at the cold input rate and at the 0.1× prompt-cache read rate, so you can see what caching is (or isn't) buying you. - **Exact when it can be.** With a `usage` block present, the total and the dollars are exact; the per-component split is reconciled to that billed total. Without one, everything is a labelled estimate. ## How the split is computed Tokens per component come from a stable offline heuristic (~4 chars/token). That estimate is only used for the **proportional** split; when the capture includes a real `usage` block, `ctxprofile` scales the split so the components sum to the exact billed input and prices the exact total. So the headline dollars are real; the per-component division is a grounded estimate, marked as such. With `--online`, the system prompt and every tool definition are measured against `count_tokens` instead of estimated, and only the remaining components share out what the billed total has left. Measured rows are marked `*` in the table and carry `"exact": true` in `--json`. ## What it can and can't tell you Owning the boundary is the point. Numbers are labelled by how much you can trust them. | Signal | Confidence | | --- | --- | | Total input $ (capture has `usage`) | Exact — from the billed usage block | | Blended cache $ (usage has a cache split) | Exact — from the cache buckets | | Which tools are defined vs. actually called | Exact — structural, over this capture's request **and** its response | | Dead-tool token cost | Estimated tokens (~4 chars/token, reconciled to the billed total); **measured** with `--online`; list-price rate | | Per-component token split | Estimate — a stable ~4 chars/token heuristic, reconciled to the exact total | | System and per-tool tokens with `--online` | Measured — a `count_tokens` marginal, not a heuristic | | Message / history split with `--online` | Still an estimate — but `--online` reports th … ## Source & license This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [Evge14n](https://github.com/Evge14n) - **Source:** [Evge14n/ctxprofile](https://github.com/Evge14n/ctxprofile) - **License:** MIT Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/mcp-evge14n-ctxprofile - Seller: https://agentstack.voostack.com/s/evge14n - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.