Context Optimization

Context Optimization Techniques

Context optimization extends effective capacity through strategic compression, masking, caching, and partitioning. The goal is making better use of available capacity — not magically increasing windows. Effective optimization can double or triple effective context without larger models.

When to Use / Not Use

Use when:

• Context limits constrain task complexity
• Optimizing for cost reduction (fewer tokens = lower costs)
• Reducing latency for long conversations
• Implementing long-running agent systems
• Building production systems at scale

Do NOT use when:

• Compressing or summarizing context -> use context-compression
• Diagnosing context failures or degradation -> use context-degradation
• Learning context basics -> use context-fundamentals
• File-based context patterns or scratch pads -> use filesystem-context

Decision Tree

What optimization problem are you solving?
├── Tool outputs dominate token usage (>80%)
│   └── Observation Masking -> Replace verbose outputs with references
├── Context approaching limits (>70% utilization)
│   ├── Message history dominates -> Compaction + Summarization
│   ├── Retrieved docs dominate -> Summarization or Partitioning
│   └── Multiple components -> Combine strategies
├── Repeated requests with common prefixes
│   └── KV-Cache Optimization -> Stable prefix ordering
├── Single context too large for one agent
│   └── Context Partitioning -> Sub-agent isolation
├── Need to measure if optimization is working
│   └── See §Performance Targets
└── Not about extending capacity? -> See related skills

Four Primary Strategies

| Strategy | Mechanism | Best For | Expected Savings | |----------|-----------|----------|-----------------| | Compaction | Summarize context near limits, reinitialize with summary | Message history dominating | 50-70% token reduction | | Observation Masking | Replace verbose tool outputs with compact references | Tool output dominance (80%+ of tokens) | 60-80% reduction in masked obs | | KV-Cache Optimization | Reuse cached KV computations across shared prefixes | Repeated requests with stable prefixes | 70%+ cache hit rate | | Context Partitioning | Split work across sub-agents with isolated contexts | Single context too large | Isolation + clean focus |

Compaction

Summarize context sections when approaching limits, then reinitialize with the summary. Priority order for compression:

1. Tool outputs — Replace with summaries of key findings, metrics, conclusions
2. Old turns — Summarize early conversation, preserve decisions and context shifts
3. Retrieved docs — Summarize if recent versions exist
4. Never compress — System prompt

Target: 50-70% token reduction with less than 5% quality degradation.

Observation Masking

Replace verbose tool outputs with compact references once the output has served its purpose.

| Category | Action | Examples | |----------|--------|---------| | Never mask | Critical to current task, most recent turn, active reasoning | Current file being edited, error being debugged | | Consider masking | 3+ turns old, verbose with extractable key points, purpose served | Large file reads, search results already acted on | | Always mask | Repeated outputs, boilerplate headers/footers, already summarized | Help text, version banners, duplicate reads |

if len(observation) > max_length:
    ref_id = store_observation(observation)
    return f"[Obs:{ref_id} elided. Key: {extract_key(observation)}]"

KV-Cache Optimization

KV-cache stores Key and Value tensors from inference, growing linearly with sequence length. Prefix caching reuses KV blocks across requests with identical prefixes via hash-based block matching.

Cache-friendly ordering principle: Stable elements first, unique elements last.

context = [system_prompt, tool_definitions]  # Cacheable (stable)
context += [reused_templates]                 # Reusable (semi-stable)
context += [unique_content]                   # Unique (never cached)

Cache stability design:

• Avoid dynamic content (timestamps, random IDs) in prefixes
• Use consistent formatting across sessions
• Keep structure stable even when content changes

Context Partitioning

Most aggressive optimization: partition work across sub-agents with isolated contexts. Each sub-agent operates in a clean context focused on its subtask.

When to partition:

• Single context would exceed limits
• Independent subtasks with minimal dependencies
• Need separation of concerns (search context vs. synthesis context)

Result aggregation: Validate all partitions completed → merge compatible results → summarize if still too large.

Context Budget Management

| Budget Category | Typical Allocation | Notes | |----------------|-------------------|-------| | System prompt | 5-10% | Never compress | | Tool definitions | 10-15% | Semi-stable, cacheable | | Retrieved documents | 15-30% | Dynamic, filterable | | Message history | 20-40% | Growing, compactable | | Tool outputs | 30-50% | Dominant, maskable | | Reserved buffer | 10% | Headroom for responses |

Trigger signals for optimization:

• Token utilization above 80%
• Degradation indicators (quality dropping)
• Performance drops (latency increasing)
• Cost thresholds exceeded

Anti-Patterns

| Anti-Pattern | Problem | Solution | |---|---|---| | Optimizing without measuring | Cannot distinguish effective optimization from harmful compression | Measure before and after: token counts, quality metrics, cache hit rates | | Masking too aggressively | Masking recent or critical observations causes agent to lose context mid-task | Follow masking categories: never mask current-turn or active-reasoning outputs | | Ignoring cache stability | Dynamic content in prefixes (timestamps, random IDs) breaks cache hits | Place stable content first; push dynamic content to end of context | | Partitioning too early | Isolation overhead exceeds savings for moderately-sized contexts | Partition only when single context would exceed limits or subtasks are genuinely independent | | Compressing system prompt | System prompt establishes identity; compression loses behavioral constraints | Never compress system prompt; always include in cacheable prefix | | No compaction triggers | Context fills silently until severe degradation | Implement triggers at 70-80% utilization; monitor usage against budget | | Applying single strategy | Different context components need different optimization techniques | Combine strategies based on context composition (see Decision Tree) | | Assuming optimization is free | Each strategy has quality cost; aggressive targets degrade output | Target 50-70% compaction with <5% quality loss; measure, don't assume |

Guidelines

1. Measure before optimizing — know your current state
2. Apply compaction before masking when possible
3. Design for cache stability with consistent prompts
4. Partition before context becomes problematic
5. Monitor optimization effectiveness over time
6. Balance token savings against quality preservation
7. Test optimization at production scale
8. Implement graceful degradation for edge cases

Integration

• context-fundamentals - Prerequisite: basic context concepts
• context-degradation - Understanding when optimization is needed
• context-compression - Compression as one optimization technique
• multi-agent-patterns - Partitioning as isolation
• evaluation - Measuring optimization effectiveness
• memory-systems - Offloading context to memory

References

Internal reference:

• [Optimization Techniques Reference](./references/optimization_techniques.md) - Detailed technical reference

External resources:

• Research on context window limitations and KV-cache optimization
• Production engineering guides from leading AI labs

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: viktorbezdek
• Source: viktorbezdek/skillstack
• License: MIT

Install and usage instructions live in the source repository linked above.