Caching, CDN and real-time delivery
The cardinal rule to say out loud before anything else in this area: cache invalidation bugs are consistency bugs, so choose the staleness you can tolerate before you choose a cache. Most candidates answer caching questions by naming Redis. The lead-level answer layers it: browser, CDN, gateway, in-process, distributed, database buffer pool, and picks the layer that matches the invalidation story.
The real-time half of the chapter is where LLM products live now. Token streaming made server-sent events relevant again, and the reasons why are a clean, checkable test of whether someone understands HTTP infrastructure.
What this chapter covers
- [done] Cache stampede on a hot key
- [done] SSE vs WebSockets for token streaming
- [todo] WebSocket scaling: sticky routing vs a pub/sub backplane
- [todo] Connection resource math, ulimit, ephemeral ports, conntrack
- [todo] Reconnect, resume by last-event-id, and client-side dedupe
- [todo] Cache-Control decided per asset class, as a walkthrough
- [todo] stale-while-revalidate and stale-if-error
- [todo] ETag, Last-Modified, and validator semantics
- [todo] Vary and the cache-key design that destroys hit rate
- [todo] CDN tiered caching and origin shield
- [todo] Purge strategies: hard, soft, surrogate keys, with a worked example
- [todo] Edge compute: what belongs at the edge and what does not
- [todo] Caching a personalised page: shell plus fragments, ESI, streaming SSR
- [todo] Cache poisoning and unkeyed input
- [todo] The six cache layers and the patterns that go with them
- [todo] Redis eviction policies, cluster hash slots, hot-key mitigation
- [todo] Redis persistence, and why Redis is not a database
- [done] Invalidating with complex dependencies
- [todo] Negative caching
Source: §18.