Hermes Context Optimization

Hermes Context Optimization

Use this when the user asks about Hermes prompt/context size, “hello” startup cost, compression behavior, memory/profile bloat, tool-schema overhead, skill loading, session-store/search-index storage, or multimodal/visual-context approaches such as Snapcompact.

For large state.db search-index maintenance, resumable optimize-storage --no-vacuum runs, foreground timeout handling, and post-run verification, follow references/session-store-optimization.md.

When bounded workers repeatedly fail with session_persistence_failed because a live profile DB is lock-contended, preserve the worktree handoff and use the non-disruptive isolated-HERMES_HOME workflow in references/isolated-worker-state-store.md. Diagnose the profile-specific DB and lock owners first; never terminate a user's live TUI or optimize its store merely to unblock a coding worker.

Operating style for the user

• Be concise first. If he asks for the savings, give the numbers, not a lecture.
• Separate hard blockers from engineering choices.
• Prefer token/accounting estimates grounded in actual provider/runtime data when available; use ratios only when clearly labeled.
• Do not imply that all startup prompt text is interchangeable. Bucket it by runtime role.

Long standing-goal payloads

When a long /goal prompt is displayed as a compact token such as [[ … [77 lines] … ]], treat that representation as potentially lossy rendered text, not as the authoritative goal body. Hermes' goal state stores the string it receives and does not dereference wiki-style links or recover omitted lines. Before allowing a goal worker to edit anything, require it to read the complete authoritative specification from a file and verify that the first turn contains the expected task scope. Use a short file-backed goal such as: Read and execute the complete goal specification at /absolute/path/goal.md; treat that file as authoritative; do not act on truncated display text. If the worker loads Obsidian or searches for a note matching the compact label, pause/clear the goal: it received the display placeholder, not the task. Do not accept a “goal achieved” result caused only by missing input; require artifact or test evidence.

Reference: references/goal-long-prompt-preservation.md.

Native first-turn inspection

When the question is what Hermes loads or sends on the first message of a fresh session, start with the native surfaces rather than reconstructing the prompt manually:

• Preflight fixed payload: hermes prompt-size --platform ; add --json for machine-readable output. It runs offline and reports system-prompt tiers, skills, memory/profile, and tool-schema bytes/count using platform-resolved tools.
• Live request after the first turn: /usage; it separates system prompt, built-in tools, rules, skills, MCP schemas, subagent definitions, memory, and conversation.
• Treat the displayed token categories as estimates (chars / 4), not exact provider-tokenizer counts. Use provider-reported input tokens for the exact total. Do not present proportionally scaled category estimates as exact tokenizer measurements; label them as approximations if used.
• Start from the fixed-payload basics: memory block, user-profile block, skills index, core system prompt, built-in tool schemas, delegation schema, MCP schemas, then conversation. If the user asks specifically how large user.md or memory.md is under Mnemosyne, report the live generated prompt blocks even when no literal Markdown files exist.
• Do not blame an intentionally isolated/test Tool Router merely because the default profile exposes a broad tool surface. First establish the normal baseline composition. Inspect router policy or logs only when the question is about routing, an unexpected regression, or whether schemas should have been narrowed.
• For finer offline categories, construct the same inspection agent and call agent.context_breakdown.compute_session_context_breakdown(agent, messages=[]). This separates core system prompt, built-in tool definitions, skills, MCP, delegation schema, and combined memory/profile. Remember that this command also uses the rough chars / 4 estimator.
• In Docker, run the same command through docker exec -it hermes prompt-size ... or docker compose exec hermes prompt-size ....
• Distinguish a composition report from a raw wire dump: prompt-size does not print the complete prompt contents, and live MCP registration is best verified through /usage.

Full command matrix, Docker examples, and interpretation pitfalls: references/native-context-inspection.md.

Explain schema growth correctly

Tool definitions do not grow merely because a conversation gets older. They are a fixed schema surface for that session/request. The baseline grows when Hermes updates, enabled toolsets, plugins, MCP servers, Office integrations, or dynamic provider tools register additional callable schemas. Prompt caching may reduce repeat billing and latency, but it does not return the occupied context window. State these distinctions before recommending pruning, profile specialization, consolidation, or dynamic routing.

Initial payload triage

When analyzing Hermes startup cost, split the payload into these buckets:

| Bucket | Can be compressed/lazy-loaded? | Notes | |---|---:|---| | Core system/developer rules | Partly, but keep authoritative text | Identity, safety, tool policy, authority hierarchy, injection handling, platform rules need reliable system-channel text. | | Tool schemas | No, unless tools are pruned | Provider needs machine-readable JSON schemas to expose function calls. A screenshot of schemas is not a callable tool registry. | | Memory/user profile | Yes | Good candidate for tiering, retrieval, or visual appendix. | | Skill directory/index | Yes | Prefer tiny router/index at startup; load skills on demand. | | Environment/project notes | Yes | Reference context; retrieve or attach only when relevant. | | Prior sessions/compressed history | Yes | Good candidate for summary + artifact fallback. |

Preferred optimization order

1. Measure the payload composition if tools/logs are available: system text, memory/profile, skill directory, tool schemas, platform metadata.
2. Prune/gate tool schemas first. Tool schemas often dominate startup cost and cannot be replaced by images.
3. Shrink the authoritative bootloader to core identity/rules only.
4. Tier memory/profile into always-loaded core facts vs domain/project facts retrieved on demand.
5. Gate skill directory behind a tiny class-level index and explicit skill_view loading.
6. Use visual context/Snapcompact only for reference material, not for authority or tool registration.

Live latency attribution

For gateway or voice latency, measure one clean tool-free turn first and inspect logs from a separate controller afterward. Never have the measured agent call log/file tools inside its own latency probe; those calls add model round trips and invalidate the sample. Break the result into model time, actual tool execution, framework overhead, client-to-gateway transport, STT endpointing, and TTS synthesis/playback. Label boundaries that the current build does not timestamp as unmeasured rather than estimating them.

Tool-heavy turns commonly spend most of their time in the repeated model calls around tools, not in tool execution. Compare single-call turns at several input-token sizes to prove context-pressure effects. Tool Router can reduce schemas and unnecessary tool selection, but it does not shrink accumulated conversation history; pair it with an explicit compression/context policy when history is the bottleneck.

Full calculation method and voice-specific acceptance shape: references/gateway-voice-latency-attribution.md.

Snapcompact / visual-context guidance

Snapcompact-style image context can reduce reference-text tokens by roughly the article’s observed ratio:

10,000 text tokens -> 3,279 image tokens ≈ 67.2% reduction

Quick estimate:

image_tokens ≈ text_tokens * 0.3279
saved_tokens ≈ text_tokens * 0.6721

For Hermes startup, report realistic savings only for the imageable portion:

• 4k imageable reference tokens -> saves ~2.7k
• 8k imageable reference tokens -> saves ~5.4k
• 12k imageable reference tokens -> saves ~8.1k

Do not claim the entire initial payload can become an image unless tool schemas and authoritative rules are separately handled.

Safe architecture

Recommended shape:

minimal authoritative text bootloader
+ typed user prompt
+ minimal/gated tool schemas
+ optional visual reference appendix
+ retrieval/lazy loading for memory, skills, and project/domain context

For local-first setups, consider:

snapcompact image -> local vision model extracts relevant section -> main model receives extracted text

This avoids forcing the primary model to OCR every appendix on every turn, but adds latency and OCR risk.

Memory compaction pattern (for reducing initial prompt)

When shrinking always-loaded memory/user_profile:

• Remove entries that duplicate existing skills, AGENTS.md, or vault docs.
• Merge overlapping preferences into single compact bullets.
• Drop stale/session-specific notes and auto-captured noise.
• Keep only durable behavioral facts that materially affect how the agent acts on every turn.

Operational pitfalls

• Vague profile labels cause downstream hallucination. A label like "20 years HK experience" is too broad — a model can't tell if it means finance, hospitality, teaching, or import/export, so it guesses. When the user corrects a mistaken extrapolation, trace back to the SOURCE label and make it specific (e.g., "20 years Hong Kong housekeeping/hospitality experience"). Don't just correct the session output — fix the underlying profile data. See references/profile-data-accuracy.md.
• Memory tool loop trap: calling the memory tool without specifying a valid operation causes repeated identical failures and hard-stops further calls to it in that turn. Always include an explicit action field with concrete old_text/new content; if unsure, use terminal or file inspection first instead of guessing entries. Never ping it as “read current state”; rely on last known list or external checks.
• Repeated-call guardrail: the same tool call (identical arguments) failing 3 times triggers repeatedexactfailure_block and blocks that tool until you change strategy. When a tool fails:
• Do NOT retry with identical arguments hoping it will work next time.
• Read the error message; adjust parameters, shape, or approach before calling again.
• If the tool does not support what you’re trying (e.g., “read” when only add/replace/remove exist), stop using it for that purpose and switch to an alternative method.
• Do not over-explain when the user asks for a numerical answer; compute or estimate and answer directly.

Memory compaction (proven batch workflow)

When shrinking memory or user_profile:

1) Inspect current entries via terminal if possible:

• python3 -c "import yaml; c=yaml.safe_load(open('~/.hermes/config.yaml')); print(c.get('mnemosyne',{}).get('memory',[]))"
• If that is empty but you see injected MEMORY/USER PROFILE at startup, those are in mnemosyne's DB (not config.yaml). You must work from the text already injected into your context — there is no read-only memory() call.

2) Use a single operations batch instead of many small calls:

• Build one operations list with add/replace/remove entries only for facts you can match as substrings in what's currently injected.
• Batch rules:
• Remove: domain notes already covered by skills (business, work, community, browser, content-style).
• Replace: verbose environment/model/provider notes → compact bullets.
• Add: only durable behavioral facts that affect every turn.
• Watch for all-or-nothing behavior: if one replace/remove fails due to a mismatched old_text, the entire batch is rejected — ensure exact substring matches against what you see injected.

3) Avoid tool-loop traps:

• memory() has no read-only mode; calling it with action=None or identical arguments repeatedly will trigger repeatedexactfailure_block and halt further calls.
• If blocked, stop retrying unchanged; switch to terminal inspection or file reads instead.

4) Expected savings:

• A well-maintained system memory can be 1–2K chars (around 300–600 tokens). Anything above ~3K is usually hoarding stale task notes that belong in skills, vault files, or session history.

Pitfalls

• Do not put mandatory behavior rules only inside an image; image text is reference content, not a reliable system-channel control surface.
• Do not put tool schemas only inside an image; providers require structured schemas for tool calling.
• Do not attach large visual appendices to trivial greetings unless the provider's image-token accounting and latency justify it.
• Compression routing pitfall: auxiliary.compression overrides compression.summary_model. If compression seems stuck on an unexpected provider (e.g., DeepSeek), check auxiliary.compression first — it is usually the actual source of truth.

Displayed output vs model-visible context

When the user asks whether a long tool result shown on screen is also fully reread by the model, distinguish the presentation layer from the active prompt. The UI/gateway may display the complete tool output, while Hermes later compacts the model-visible transcript. During context cleanup, the compressor receives the full transcript including tool messages, then performs a cheap pre-pass before any LLM summary: older tool results are replaced with informative one-line records (tool name, operation, exit status or size), identical results are deduplicated while the newest full copy is retained, and old tool-call arguments are truncated. Recent tool results remain verbatim. This is context pruning, not merely a JSON transport separator; JSON is used for message/tool structure and structured summaries, while the token savings come from replacing old content. The original full session remains available in the session store.

Use this concise explanation when asked: “The screen and model context are separate. Hermes can show the full result, but older tool output is later replaced internally with a compact record; recent output stays full.” Do not incorrectly imply that every long result is immediately omitted or that JSON encoding itself causes the reduction.

Compression model requirements (from official behavior)

• Hermes sends the entire selected middle section in one call_llm(task="compression") request; it does not map-reduce or chunk the middle automatically.
• Conservative default: give the compressor at least the parent model's effective compression trigger/window. This avoids auto-lowering, truncation, and hidden instruction/summary overhead.
• Measured narrow exception: a smaller compressor can be valid when the actual middle is calculated and live-proven. Approximate the first-cycle payload as:

usable_prompt = parent_context - parent_max_tokens
trigger = usable_prompt × compression.threshold
middle ≈ trigger × (1 - compression.target_ratio)

Then add headroom for the structured-summary instructions, prior summaries, tool metadata, and token-estimation variance. Never describe the raw middle estimate as the whole request size.

• Shared-compressor rule: size the compressor for the largest active parent plane it serves. A Gemma instance that compresses both an 80K local worker and a 184K cloud orchestrator must satisfy the cloud parent's trigger, not merely the worker's smaller requirement.
• Keep providers..models..context_length, auxiliary.compression.context_length, and the backend's live loaded context truthful and aligned. Do not advertise a larger context merely to bypass a conservative guard; either load th

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: AtlasOmnia
• Source: AtlasOmnia/donna-starter
• License: MIT

Install and usage instructions live in the source repository linked above.