Caching Strategies

Caching Strategies

Overview

Poorly designed caches cause data staleness, thundering herds on cold starts, unbounded memory growth, and invalidation bugs that silently serve wrong data for hours. Use this guide when designing a new cache layer, reviewing caching code, or debugging performance regressions tied to cache misuse.

When to use: Adding caching to a service or API; reviewing code that reads from Redis, Memcached, or an in-process store; evaluating CDN configuration; auditing memory usage of long-running processes; troubleshooting stale-data incidents.

Quick Reference

| Pattern | Core Idea | Primary Red Flag | |---------|-----------|-----------------| | Cache-Aside | App checks cache, falls back to DB, populates cache | No TTL set; stale data served indefinitely | | Read-Through | Cache layer fetches from DB automatically on miss | Miss storms on cold start; no stampede protection | | Write-Through | Write hits cache and DB synchronously | Write latency doubles; inconsistency if either fails | | Write-Behind | Write hits cache, DB updated asynchronously | Data loss if cache crashes before flush | | Stampede / Thundering Herd | Many callers race on a single cold key | All callers hit DB simultaneously on expiry | | Singleflight / Mutex | One caller fetches; others wait for that result | Lock contention if fetch is slow | | CDN / Cache-Control | Edge caches static and semi-static responses | Missing stale-while-revalidate; over-aggressive purging | | LRU / LFU / TTL Eviction | Remove entries by recency, frequency, or age | Unbounded cache size; wrong eviction policy for access pattern | | Multi-Layer Cache | L1 in-process, L2 distributed, L3 CDN | L1/L2 inconsistency; missing invalidation at each layer | | Event-Based Invalidation | Domain events trigger targeted cache evictions | Missed event; stale data blindness after writes | | Versioned Keys | Key includes version or content hash | Old keys accumulate; no expiry on versioned entries |

---

Patterns in Detail

1. Cache-Aside (Lazy Loading)

The application owns the cache interaction: check cache first, fetch from the source of truth on a miss, then populate the cache.

Red Flags:

• No TTL on the key — stale data served indefinitely after the underlying record changes
• Cache population inside a transaction — cache and DB can diverge if the transaction rolls back
• Cache key not namespaced — collisions across environments or tenants
• No negative caching — every miss for a non-existent key hammers the DB

TypeScript:

async function getUser(id: string): Promise {
  const cacheKey = `user:v1:${id}`;
  const cached = await redis.get(cacheKey);
  if (cached !== null) return JSON.parse(cached) as User;

  const user = await db.users.findById(id);
  if (user) {
    await redis.set(cacheKey, JSON.stringify(user), 'EX', 300); // 5-min TTL
  } else {
    await redis.set(cacheKey, 'null', 'EX', 30); // negative cache: 30s
  }
  return user;
}

Go:

func GetUser(ctx context.Context, id string) (*User, error) {
    key := fmt.Sprintf("user:v1:%s", id)
    val, err := rdb.Get(ctx, key).Result()
    if err == nil {
        var u User
        if err := json.Unmarshal([]byte(val), &u); err == nil {
            return &u, nil
        }
    }
    user, err := db.FindUserByID(ctx, id)
    if err != nil {
        return nil, fmt.Errorf("GetUser(%s): %w", id, err)
    }
    data, _ := json.Marshal(user)
    rdb.Set(ctx, key, data, 5*time.Minute)
    return user, nil
}

Redis CLI — verify TTL is set:

TTL user:v1:abc123   # should return > 0; -1 means no expiry (bug)

---

2. Read-Through and Write-Through Patterns

Read-Through delegates the cache-miss fetch to the cache layer itself (e.g., a library or proxy). The application always reads from the cache.

Write-Through writes to the cache and the backing store in the same operation, keeping them in sync at write time.

Red Flags — Read-Through:

• No stampede protection on cold start — all callers fire the backing-store fetch simultaneously
• Read-through layer lacks circuit-breaker — DB failure bypasses cache entirely

Red Flags — Write-Through:

• Write latency spikes because both cache and DB must acknowledge before returning
• Partial failure: cache write succeeds but DB write fails (or vice versa) — use transactions or sagas to recover
• Write-through on infrequently read data — wasteful to cache data that may never be read

Python — write-through with Redis and Postgres:

def update_user(user_id: str, data: dict) -> User:
    with db.transaction():
        user = db.users.update(user_id, data)   # persist first
        cache_key = f"user:v1:{user_id}"
        redis.set(cache_key, json.dumps(user.to_dict()), ex=300)
    return user  # both succeed or neither does (via transaction rollback)

TypeScript — read-through wrapper:

class ReadThroughCache {
  constructor(
    private readonly fetch: (key: string) => Promise,
    private readonly ttlSeconds: number,
  ) {}

  async get(key: string): Promise {
    const cached = await redis.get(key);
    if (cached !== null) return JSON.parse(cached) as T;
    const value = await this.fetch(key);           // delegates to backing store
    await redis.set(key, JSON.stringify(value), 'EX', this.ttlSeconds);
    return value;
  }
}

---

3. Write-Behind (Write-Back)

Writes land in the cache immediately and are flushed to the backing store asynchronously. Reduces write latency at the cost of durability.

Red Flags:

• Cache node crashes before flush — data loss with no recovery path
• Flush queue grows unboundedly under write bursts — OOM risk
• Write-behind used for financial or audit records — wrong durability trade-off
• No ordered flush — later write overwritten by an earlier queued write

Go — write-behind with buffered flush:

type WriteBehindCache struct {
    mu      sync.Mutex
    pending map[string]User
    flush   func(users []User) error
}

func (c *WriteBehindCache) Set(u User) {
    c.mu.Lock()
    c.pending[u.ID] = u   // latest write wins per key
    c.mu.Unlock()
}

func (c *WriteBehindCache) FlushLoop(ctx context.Context, interval time.Duration) {
    ticker := time.NewTicker(interval)
    defer ticker.Stop()
    for {
        select {
        case  0 {
                if err := c.flush(batch); err != nil {
                    // re-queue or alert — never silently drop
                }
            }
        case >();

async function getWithLock(key: string, fetch: () => Promise, ttl: number): Promise {
  const cached = await redis.get(key);
  if (cached !== null) return JSON.parse(cached) as T;

  if (!inflightRequests.has(key)) {
    const promise = fetch().then(async (val) => {
      await redis.set(key, JSON.stringify(val), 'EX', ttl);
      inflightRequests.delete(key);
      return val;
    }).catch((err) => { inflightRequests.delete(key); throw err; });
    inflightRequests.set(key, promise);
  }
  return inflightRequests.get(key) as Promise;
}

Jitter to stagger expiry:

import random

def set_with_jitter(redis_client, key: str, value: str, base_ttl: int, jitter: int = 30):
    """Add random jitter to prevent synchronized mass expiry."""
    ttl = base_ttl + random.randint(0, jitter)
    redis_client.set(key, value, ex=ttl)

Redis CLI — probabilistic early refresh (XFetch algorithm):

# Read value plus stored expiry time; refresh if within probabilistic window
# Implemented in application layer — not a native Redis command

---

5. CDN Caching and Cache-Control Headers

HTTP caching at the CDN edge reduces origin load and latency. Correct Cache-Control directives are critical for correctness and performance.

Red Flags:

• Cache-Control: no-store on publicly cacheable assets — unnecessary origin hits
• No stale-while-revalidate — users wait for full round-trip on every revalidation
• Cache-Control: max-age=0 paired with no ETag/Last-Modified — every request validates but headers are wrong
• Missing Vary: Accept-Encoding on compressed responses — CDN serves wrong encoding to some clients
• Caching authenticated API responses without Cache-Control: private — data leakage between users

Common directives:

| Directive | Meaning | |-----------|---------| | max-age=N | Cache for N seconds (client + CDN) | | s-maxage=N | CDN-only max age (overrides max-age for shared caches) | | stale-while-revalidate=N | Serve stale content for N seconds while fetching fresh in background | | stale-if-error=N | Serve stale for N seconds if origin returns 5xx | | no-cache | Must revalidate with origin before serving (not "don't cache") | | no-store | Never store the response in any cache | | private | Browser may cache; CDN must not | | immutable | Resource will never change; skip revalidation for max-age duration |

TypeScript — Express headers for a product API:

app.get('/products/:id', async (req, res) => {
  const product = await getProduct(req.params.id);
  res
    .set('Cache-Control', 'public, s-maxage=60, stale-while-revalidate=30, stale-if-error=300')
    .set('ETag', hashProduct(product))
    .json(product);
});

// Static assets — long TTL with immutable (content-addressed filename)
app.use('/static', express.static('dist', {
  maxAge: '1y',
  setHeaders: (res) => res.set('Cache-Control', 'public, max-age=31536000, immutable'),
}));

CDN purge on update — Python:

import httpx

def purge_cdn_key(path: str) -> None:
    """Purge a single path from the CDN after a write."""
    resp = httpx.post(
        f"https://api.cdn-provider.com/purge",
        json={"paths": [path]},
        headers={"Authorization": f"Bearer {CDN_API_TOKEN}"},
        timeout=5.0,
    )
    resp.raise_for_status()

---

6. Eviction Policies

When the cache reaches capacity, an eviction policy decides which entries to remove. Choosing the wrong policy wastes memory and degrades hit rates.

Red Flags:

• Default Redis noeviction policy — writes fail under memory pressure instead of evicting entries
• LRU applied to a scan-heavy workload — recently scanned cold data evicts hot frequently accessed data
• No TTL on any key — cache grows unboundedly until OOM
• Eviction policy set globally but not tuned per key namespace

Policy Comparison:

| Policy | Evicts | Best For | |--------|--------|----------| | LRU (Least Recently Used) | Oldest last-accessed entry | General-purpose caches with recency-biased access | | LFU (Least Frequently Used) | Least-accessed entry over time | Workloads with clear hot/cold data split | | TTL-based | Oldest by remaining TTL | Content caches where freshness is the primary concern | | Random | Random entry | Very high throughput where approximation is acceptable | | Adaptive (ARC, W-TinyLFU) | Balances recency and frequency | Mixed workloads; used by Caffeine (Java), Ristretto (Go) |

Redis — configure eviction policy:

# Set in redis.conf or at runtime:
CONFIG SET maxmemory 512mb
CONFIG SET maxmemory-policy allkeys-lru   # evict any key by LRU when full
# Other options: allkeys-lfu, volatile-lru, volatile-lfu, volatile-ttl, noeviction

Go — Ristretto (W-TinyLFU adaptive cache):

import "github.com/dgraph-io/ristretto"

cache, _ := ristretto.NewCache(&ristretto.Config{
    NumCounters: 1e7,     // track frequency for 10M keys
    MaxCost:     1  str:
    return country_db.lookup(code)   # expensive lookup cached in-process

---

7. Multi-Layer Cache Architecture

Production systems typically use three cache layers. Each layer trades latency, capacity, and consistency differently.

| Layer | Type | Latency | Capacity | Consistency | |-------|------|---------|----------|-------------| | L1 — In-Process | Thread-local / heap map | ();

async function getLayered(key: string, fetch: () => Promise, ttlMs: number): Promise { // L1 check const l1 = l1Cache.get(key); if (l1 && l1.expiresAt > Date.now()) return l1.value as T;

// L2 check (Redis) const l2 = await redis.get(key); if (l2 !== null) { const value = JSON.parse(l2) as T; l1Cache.set(key, { value, expiresAt: Date.now() + ttlMs / 2 }); // shorter L1 TTL return value; }

// DB fetch const value = await fetch(); await redis.set(key, JSON.stringify(value), 'EX', Math.floor(ttlMs / 1000)); l1Cache.set(key, { value, expiresAt: Date.now() + ttlMs / 2 }); return value; }

---

### 8. Cache Invalidation Strategies

Invalidation is the hardest part of caching. Three main strategies: time-based, event-based, and versioned keys.

**Red Flags:**
- No invalidation strategy — stale data blindness after writes; users see outdated records
- Invalidating by scanning key patterns (`KEYS user:*`) in production Redis — O(N) blocks the server
- Event-based invalidation without idempotency — duplicate events cause double invalidation churn
- Versioned keys without expiry — old versioned keys accumulate, exhausting memory

**Time-Based Invalidation:**
```python
# Accept some staleness; TTL controls maximum stale window
redis.set("catalog:featured", json.dumps(products), ex=600)  # stale up to 10 min

Event-Based Invalidation — TypeScript with an event bus:

// Publisher: emit event after DB write
async function updateProduct(id: string, patch: Partial): Promise {
  const updated = await db.products.update(id, patch);
  await eventBus.publish('product.updated', { id });  // trigger invalidation
  return updated;
}

// Subscriber: invalidate on event
eventBus.subscribe('product.updated', async ({ id }: { id: string }) => {
  await redis.del(`product:v1:${id}`);
  await redis.del(`catalog:featured`);  // also bust aggregate keys
});

Versioned Keys — bust without explicit delete:

// Version stored in a separate key or config; bump version to invalidate
const CATALOG_VERSION = await redis.get('catalog:version') ?? '1';
const cacheKey = `catalog:v${CATALOG_VERSION}:featured`;

// To invalidate all catalog entries: increment the version key
await redis.incr('catalog:version');
// Old keys become unreachable and expire via TTL

Redis CLI — targeted delete (safe alternative to KEYS):

# Use SCAN with MATCH for batch deletes — non-blocking
redis-cli --scan --pattern "product:v1:*" | xargs redis-cli del

---

9. Caching Anti-Patterns

| Anti-Pattern | Description | Fix | |-------------|-------------|-----| | Cache Everything | Every DB query result is cached regardless of access frequency | Cache only data with high read-to-write ratio and measurable latency impact | | No TTL | Keys set without expiry; stale data survives forever | Always set a TTL; use short TTLs for frequently updated data | | Unbounded Cache | In-process map or cache with no size limit; memory grows until OOM | Set maxSize or MaxCost; choose an eviction policy | | Stale Data Blindness | Writes go to the DB; cache never invalidated; reads serve wrong data | Implement event-based or write-through invalidation | | Cache Stampede Ignored | Popular key expires; all callers hammer the DB simultaneously | Use singleflight, mutex, or probabilistic early refresh | | Caching Mutable Aggregates | Complex object cached; partial update invalidates only part of it | Cache fine-grained entities; rebuild aggregates at read time | | Hot Key Bottleneck | Single Redis key receives millions of reads/sec; becomes a bottleneck | Shard hot keys with a suffix (e.g., key:shard:{0..N}); use local L1 replica | | Bypassing Cache on Write | App writes directly to DB without updating or evicting the cache key | Always evict or update cache on write; never leave a stale key alive |

Hot Key Sharding — Go:

const numShards = 8

func hotKeyGet(ctx context.Context, base string) (string, error) {
    shard := rand.Intn(numShards)

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [mickeyyaya](https://github.com/mickeyyaya)
- **Source:** [mickeyyaya/refactoring-skills](https://github.com/mickeyyaya/refactoring-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.