Caching

Caching

There are only two hard things in computer science: cache invalidation and naming things. This skill covers the first one.

When to Cache

Cache when ALL of these are true:

1. Read-heavy: Data is read much more often than written
2. Expensive to compute/fetch: Database query, API call, heavy computation
3. Tolerant of staleness: It's OK if users see data that's a few seconds/minutes old
4. Stable enough: Data doesn't change every request

Don't cache when:

• Data must be real-time (account balance, inventory count at checkout)
• Data changes on every request
• The source is already fast enough
• You can't reason about when the cache is wrong

Cache Placement

User → CDN Cache → Reverse Proxy Cache → App Cache → Database Cache → Database

| Layer | What to Cache | TTL | Example | |-------|--------------|-----|---------| | Browser | Static assets, API responses | Minutes to forever | Cache-Control: max-age=31536000 for hashed assets | | CDN | Static files, public API responses | Minutes to hours | CloudFront, Fastly | | Reverse Proxy | Full page/response cache | Seconds to minutes | Nginx, Varnish | | Application | Computed results, API responses | Seconds to minutes | Redis, Memcached, in-memory | | Database | Query results, materialized views | Varies | Query cache, materialized views |

Rule: Cache as close to the user as possible. Each layer closer to the user avoids more work.

Cache Strategies

Cache-Aside (Lazy Loading)

READ:
  1. Check cache
  2. Cache hit → return cached data
  3. Cache miss → fetch from source → write to cache → return

WRITE:
  1. Write to source
  2. Invalidate cache (delete, don't update)

• Pro: Only caches data that's actually requested
• Con: First request always slow (cache miss)
• Best for: Most cases. Default choice.

Write-Through

WRITE:
  1. Write to cache AND source simultaneously

READ:
  1. Always read from cache (guaranteed fresh)

• Pro: Cache always consistent
• Con: Write latency higher, caches data that may never be read
• Best for: Data that's written and immediately re-read

Write-Behind (Write-Back)

WRITE:
  1. Write to cache
  2. Asynchronously flush to source (batched)

READ:
  1. Read from cache

• Pro: Very fast writes
• Con: Risk of data loss if cache fails before flush
• Best for: High write throughput, analytics, counters

Invalidation Patterns

TTL (Time-To-Live)

cache.set(key, value, ttl=300)  # Expire after 5 minutes

• Simplest approach. Data is eventually consistent.
• Choose TTL based on: how stale can users tolerate? How expensive is a miss?

Event-Based Invalidation

# On data change, explicitly delete the cache entry
def update_user(user_id, data):
    db.update(user_id, data)
    cache.delete(f"user:{user_id}")
    cache.delete(f"user_list:page:*")  # Related caches too

• More complex but fresher data.
• Must track ALL caches affected by each data change.

Versioned Keys

cache_key = f"user:{user_id}:v{user.updated_at}"

• Old versions naturally expire. No explicit invalidation needed.
• Cache size grows until old versions TTL out.

Cache Stampede Prevention

When a popular cache key expires, hundreds of requests simultaneously hit the source. Solutions:

Lock / Mutex

# Only one request rebuilds the cache; others wait
if cache.miss(key):
    if cache.acquire_lock(key):
        value = expensive_query()
        cache.set(key, value)
        cache.release_lock(key)
    else:
        wait_for_key(key)  # or return stale

Stale-While-Revalidate

# Serve stale data while one request refreshes in the background
cache.set(key, value, ttl=300, stale_ttl=600)
# Serves stale data for 5 extra minutes while refreshing

Jittered TTL

# Spread expirations to avoid thundering herd
ttl = base_ttl + random(0, jitter_seconds)

Common Anti-Patterns

| Anti-Pattern | Problem | Fix | |-------------|---------|-----| | Cache everything | Memory bloat, stale data everywhere | Cache only what's expensive and read-heavy | | No TTL | Stale data lives forever | Always set a TTL, even if long | | Cache & update (not invalidate) | Race conditions between concurrent updates | Invalidate (delete) on write, not update | | Caching errors | Error response served to all users | Only cache successful responses | | Unbounded cache | Memory grows until OOM | Set max size with eviction policy (LRU) | | Cache key without version | Deploy new code, old cache format causes errors | Include version/schema in cache keys |

Cache Key Design

{service}:{entity}:{id}:{version}

# Examples
api:user:123:v2
api:user_list:page:1:per:20
api:search:q=shoes:page:1

• Deterministic: same input → same key
• Namespaced: no collisions between services
• Inspectable: you can read the key and know what it caches

Review Checklist

• [ ] Cache has a TTL (no unbounded caching)
• [ ] Cache is invalidated on write (not just TTL-dependent)
• [ ] Error responses are not cached
• [ ] Cache keys are deterministic and namespaced
• [ ] Cache stampede handled for hot keys
• [ ] Cache size bounded with eviction policy
• [ ] Metrics: hit rate, miss rate, eviction rate
• [ ] Fallback works when cache is down (graceful degradation)

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: pvnarp
• Source: pvnarp/agent-skills
• License: MIT

Install and usage instructions live in the source repository linked above.