Caching, CDN and real-time delivery

The cardinal rule to say out loud before anything else in this area: cache invalidation bugs are consistency bugs, so choose the staleness you can tolerate before you choose a cache. Most candidates answer caching questions by naming Redis. The lead-level answer layers it: browser, CDN, gateway, in-process, distributed, database buffer pool, and picks the layer that matches the invalidation story.

The real-time half of the chapter is where LLM products live now. Token streaming made server-sent events relevant again, and the reasons why are a clean, checkable test of whether someone understands HTTP infrastructure.

What this chapter covers

• [done] Cache stampede on a hot key
• [done] Invalidating with complex dependencies
• [done] SSE vs WebSockets for token streaming
• [done] Reconnect, resume, and client-side dedupe
• [done] WebSocket scaling: sticky routing, backplanes, connection math
• [done] Cache-Control per asset class, validators, stale-while-revalidate
• [done] Vary, cache-key design, and cache poisoning
• [done] CDN tiered caching, origin shield, and what belongs at the edge
• [done] Caching a personalised page: layers, fragments, negative caching
• [done] Redis: eviction policies, hash slots, hot keys, persistence

Source: §18.