CDN tiered caching, origin shield, and what belongs at the edge

What it is

A CDN is not one cache. It is a hierarchy, and the shape of that hierarchy decides your origin load far more than your TTLs do.

        clients
           │
    ┌──────┴──────┬──────────┬──────────┐
    ▼             ▼          ▼          ▼
  EDGE PoP     EDGE PoP   EDGE PoP   EDGE PoP     ~300 locations
  (Toronto)    (London)   (Sydney)   (Tokyo)      small caches, close to users
    │             │          │          │
    └──────┬──────┴────┬─────┴──────────┘
           ▼           ▼
       MID-TIER     MID-TIER                      ~20 locations
       (regional)   (regional)                    large caches
           │           │
           └─────┬─────┘
                 ▼
          ORIGIN SHIELD                           ONE designated location
                 │
                 ▼
              ORIGIN

Without tiering, every edge PoP is a separate cache, so a cache miss at 300 PoPs is 300 origin requests for the same object. With tiering, a miss at the edge goes to the mid-tier, and only a miss there reaches the shield, and only a miss there reaches your origin.

Origin shield is the last layer: a single designated PoP through which all origin fetches pass, so the origin sees at most one request per object per TTL regardless of how many PoPs wanted it.

What this is confused with: more PoPs being better. More PoPs means better latency and worse hit rates, because each one has a smaller share of traffic and therefore a colder cache. The hierarchy exists to recover the hit rate that geographic distribution costs you, and a CDN with 300 PoPs and no tiering can have a worse origin offload than one with 30.

The problem it solves

The arithmetic of a flat CDN:

Object with a 1-hour TTL, requested from 300 PoPs:
  flat:               300 origin requests per hour per object
  + mid-tier (20):     20 origin requests per hour per object
  + origin shield:      1 origin request per hour per object

Three hundred to one, for one configuration change, on every object.

The effect is largest exactly where it hurts most: long-tail content. A popular object is cached everywhere and the tail is not:

Catalogue of 2M product images, global traffic:
  top 1% of objects:     cached at nearly every PoP, hit rate ~99%
  next 9%:               cached at some PoPs, hit rate ~70%
  bottom 90%:            usually cold at any given PoP, hit rate ~12%

Origin requests are dominated by the tail, and the tail is where tiering
helps most: a request for a rare object from Sydney can be served by the
mid-tier copy fetched for a Melbourne user an hour ago.

Two secondary problems tiering solves:

Thundering herd on a purge. Purging an object at 300 PoPs means the next request at each of them is a miss, so a purge of a popular object is an instant 300-request origin spike. Through a shield it is one.

Origin capacity planning becomes tractable. With a shield the origin's request rate is bounded by objects / TTL rather than by user traffic, which is a number you can compute in advance.

Mechanics

Configuring the hierarchy

Cloudflare:   Tiered Cache (Smart or Custom Topology) + "Origin Shield"
              is implicit in the upper tier.
Fastly:       Shielding: designate a POP as the shield in the backend config.
CloudFront:   Origin Shield: a named AWS region, per origin.
Akamai:       Tiered Distribution + SureRoute.

# CloudFront, per origin
origin {
  domain_name = "origin.example.com"
  origin_shield {
    enabled              = true
    origin_shield_region = "eu-west-1"    # pick the region CLOSEST TO YOUR ORIGIN
  }
}

The shield region should be closest to the origin, not to your users. The shield's job is to be a single point of consolidation in front of the origin; putting it near users defeats the purpose because the origin fetch then crosses the ocean anyway.

With multiple origins in multiple regions, you want a shield per origin, each near its own origin, or the consolidation happens at the wrong place.

Cache-key and hierarchy interaction

Tiering only works if the tiers agree on the key. If an edge PoP includes a header in the key that the mid-tier does not, they store different objects and the hierarchy is defeated:

Edge key:      /product/4471 + Accept-Encoding + X-Device-Type
Mid-tier key:  /product/4471 + Accept-Encoding
-> the mid-tier stores one object per encoding; the edge asks for
   three variants and gets the same one back, or misses depending on
   the CDN's behaviour.

Normalise the key at the edge, before the tier boundary (see Vary and cache keys), so every tier is caching the same thing.

When tiering is wrong

Tiering adds a hop, so a miss at every level is slower than a flat miss:

Flat CDN miss:        edge -> origin                  ~180 ms
Tiered miss:          edge -> mid -> shield -> origin ~240 ms

For content with a very low hit rate at any tier, tiering is pure added latency. That is the case for genuinely per-user content, for very-short-TTL data, and for anything where the object population vastly exceeds what the mid-tier can hold. The decision rule: tier when the same object is requested from multiple PoPs within its TTL, which is most content and not all of it.

What belongs at the edge

Edge compute (Cloudflare Workers, Fastly Compute, CloudFront Functions and Lambda@Edge, Akamai EdgeWorkers) runs code at the PoP. The question is not what it can do, it is what benefits from being 20 milliseconds from the user rather than 200.

BELONGS at the edge:
  - request normalisation: cache-key rewriting, header stripping,
    query-parameter cleanup
  - routing and A/B assignment: choose a variant, set a cookie, keep the
    decision consistent
  - redirects: 301/302 without a round trip to the origin
  - auth CHECKS: validate a JWT signature locally, reject early
  - personalisation ASSEMBLY: stitch cached fragments (see below)
  - bot filtering, rate limiting, geo-blocking
  - image transformation: resize and re-encode near the user

DOES NOT belong at the edge:
  - anything needing a database. A DB call from 300 PoPs is 300 clients
    against your database, from everywhere, over the internet.
  - anything needing consistency. Edge state is eventually consistent
    at best and per-PoP at worst.
  - heavy computation. Edge runtimes have tight CPU limits
    (Cloudflare Workers: 50 ms CPU on the paid tier, 10 ms free).
  - anything requiring a large dependency tree. Cold start and bundle
    size limits are real.

"Anything needing a database" is the constraint that decides most cases, and the edge-database products (Cloudflare D1, Durable Objects, KV; Fastly's KV Store) exist precisely to make a bounded subset of that possible: eventually-consistent key-value reads with edge-local replicas.

// A good edge worker: normalise, decide, and get out of the way.
export default {
  async fetch(request, env) {
    const url = new URL(request.url);

    // 1. Cache-key normalisation: strip tracking params, sort the rest.
    ['utm_source','utm_medium','utm_campaign','fbclid','gclid']
      .forEach(p => url.searchParams.delete(p));
    url.searchParams.sort();

    // 2. Auth check: signature only, no network call.
    const token = request.headers.get('Authorization')?.slice(7);
    if (url.pathname.startsWith('/api/') && !(await verifyJwt(token, env.PUBKEY))) {
      return new Response('unauthorized', { status: 401 });   // never reaches origin
    }

    // 3. Stable A/B assignment from a cookie, so the variant is in the key.
    const variant = getOrAssignVariant(request);              // 'a' | 'b'
    url.searchParams.set('_v', variant);

    return fetch(new Request(url, request));
  }
};

Rejecting unauthenticated requests at the edge is the highest-value pattern here: an attack or a misbehaving client never reaches your origin at all, and the rejection costs a signature verification rather than a round trip.

A worked example: 300 PoPs and a 34 percent offload

A retail platform. Product catalogue of 4.2 million SKUs with images, global traffic, a CDN with roughly 300 PoPs.

Baseline:

CDN hit rate:              66%
origin requests:           34% of total
origin instances:          62
p50 latency (global):      41 ms
p95 latency (Australia):   380 ms
purge behaviour:           a catalogue-wide purge caused a 40-minute
                           origin overload
origin cost:               ~$74,000/month

A 66 percent hit rate with correct cache headers (they had already done the work on the HTTP cache semantics page) meant the problem was hierarchy, not policy.

Step 1: measure where the misses come from.

misses by object popularity decile:
  decile 1 (most popular):   0.4% of misses
  decile 2:                  1.1%
  ...
  decile 9:                 14.2%
  decile 10 (long tail):    61.8%      <- almost two thirds

misses by PoP, for the SAME object within its TTL:
  objects fetched from origin by >1 PoP in a TTL window:  84%
  mean PoPs fetching the same object per TTL:             11.3

Eighty-four percent of origin fetches were for an object another PoP had already fetched. That is the number that says "you need tiering" and it is measurable directly from CDN logs.

Step 2: enable tiered caching.

                          before      after
CDN hit rate              66%         88%
origin requests           34%         12%
mean PoPs fetching the
  same object per TTL     11.3        1.9
p50 latency               41 ms       43 ms      (+2 ms: the extra hop on misses)

Twenty-two points of hit rate for a configuration change, at the cost of 2 ms on p50.

Step 3: origin shield.

                          before      after
origin requests           12%         3.1%
mean PoPs fetching the
  same object per TTL     1.9         1.0
origin instances          62          22

The shield took the remaining duplication to zero, because every origin fetch now passes through one location.

Step 4: the purge problem, which the shield fixed as a side effect.

catalogue-wide purge (a pricing update), before:
  affected objects:         4.2M
  PoPs holding them:        ~300
  origin requests in the
    first 60 seconds:       ~180,000/s     <- overload
  recovery:                 40 minutes

after (with shield):
  origin requests in the
    first 60 seconds:       ~14,000/s
  recovery:                 4 minutes

Still a spike, because a purge of 4.2M objects means 4.2M origin fetches eventually, and the shield collapsed the 300x duplication. They additionally moved to surrogate-key purging so a price change purged only the affected SKUs (see cache invalidation).

Step 5: edge compute, for three things.

// 1. Cache-key normalisation, which was previously done at the origin
//    (so every variant reached the origin to be told it was the same object).
// 2. JWT verification, rejecting unauthenticated /api/ requests at the edge.
// 3. Image variant selection: choose format and size from Accept and
//    a width hint, so one stored original serves all variants.

                          before      after
unauthenticated requests
  reaching origin         ~2.4M/day   0
image variants stored     14 per SKU  1 original + edge transform
origin storage            41 TB       6 TB
p95 latency (Australia)   380 ms      88 ms

The 2.4 million daily unauthenticated requests were mostly bot traffic, and moving the rejection to the edge removed them from origin capacity planning entirely.

Step 6: what they tried at the edge and reverted.

attempted: personalised pricing lookup at the edge, hitting a regional
           read replica over the internet.

measured:  p50 edge latency 20 ms -> 210 ms
           database connections from 300 PoPs: ~9,000 concurrent
           replica CPU: 94%
           
reverted after 3 days.

A database call from the edge is a database call from 300 places, and the connection count alone made it unworkable. They moved that lookup back to the origin and cached the result per pricing tier, which was 40 distinct values rather than per user.

Final:

                              before      after
CDN hit rate                  66%         96%
origin requests               34%         2.8%
origin instances              62          14
p50 latency (global)          41 ms       38 ms
p95 latency (Australia)       380 ms      88 ms
origin storage                41 TB       6 TB
purge recovery                40 min      4 min
origin cost                   ~$74,000/mo ~$16,000/mo

The tiering and shield changes were configuration and delivered most of the offload. The edge compute delivered the latency and the storage saving. And the one thing they tried that did not work was the one that needed a database, which is the constraint that decides edge architecture.

Production evidence

Fastly's shielding, Cloudflare's Tiered Cache, CloudFront's Origin Shield and Akamai's Tiered Distribution are the same idea in four products, and all four document the same motivation: without it, origin load scales with PoP count rather than with content.

Cloudflare's Argo Smart Routing and Tiered Cache are marketed separately because they solve different halves: routing improves miss latency, tiering improves the hit rate. That separation is a useful reminder that a hierarchy costs latency on a full miss.

CloudFront's documentation is explicit that Origin Shield should be in the region closest to the origin, and that with multiple origins you want a shield per origin.

Cloudflare Workers' CPU limits (10 ms free, 50 ms paid, with configurable higher limits) are documented, and they are the constraint that keeps heavy computation off the edge. Fastly Compute's WebAssembly model has a comparable posture with faster cold starts.

Cloudflare D1, KV and Durable Objects exist because "no database at the edge" is a real constraint that customers hit immediately, and their consistency models (eventually consistent KV, single-instance-per-object Durable Objects) are the honest shape of what is possible.

Image transformation at the edge (Cloudflare Images, Fastly Image Optimizer, Akamai Image Manager) is the most widely adopted edge-compute use case, precisely because it is CPU-bounded, needs no state, and turns a storage multiplier into a compute cost.

The debate

Should you always enable tiering and shielding? For anything with an object population larger than a single PoP's cache and traffic from many PoPs, yes: it is a configuration change that in the worked example took origin requests from 34 percent to 3. The cases where it is wrong are genuinely per-user content and very short TTLs, where nothing is shared between PoPs and the extra hop is pure latency.

Does more PoPs help? For latency, yes. For hit rate, no: each PoP has a smaller traffic share and a colder cache. The two pull against each other and tiering is what reconciles them, which is why a 300-PoP CDN without tiering can offload less than a 30-PoP one.

What is the right shield location? Closest to the origin, always. Putting it near users means the origin fetch still crosses the distance, and the consolidation happens at the wrong end. With multi-region origins, one shield per origin.

What belongs at the edge? Anything that is a decision rather than a computation, and anything that can reject work before it reaches your origin. Cache-key normalisation, auth checks, routing, redirects, A/B assignment, bot filtering. The bright line is state: if it needs a database, the edge is 300 clients against that database from everywhere, and the connection count alone usually settles it.

Are edge databases changing that? Partially and carefully. Eventually-consistent edge-replicated key-value stores make read-heavy, staleness-tolerant lookups viable, and they do not make transactional or strongly-consistent workloads viable. The honest framing is that they extend the "no state at the edge" rule to "no strongly consistent state at the edge", which is a meaningful widening and not a removal.

Is edge compute worth the operational cost? It is a second runtime, a second deployment pipeline, a second place bugs live, and different observability. For request normalisation and auth rejection it pays immediately, because both remove origin load and neither is complex. For application logic it is a real architectural commitment and should be treated as one.

Follow-up Q&A

"What is origin shield and why does it matter?"

A single designated PoP through which all origin fetches pass, so the origin sees at most one request per object per TTL regardless of how many PoPs wanted it. Without it, each of 300 PoPs is an independent cache, so a miss is 300 origin requests for the same object. It also collapses the purge spike: a catalogue purge went from 180,000 origin requests per second to 14,000 in one case, purely from consolidation.

"Why does adding PoPs sometimes reduce the hit rate?"

Each PoP has a smaller share of traffic, so its cache is colder and long-tail objects are unlikely to be present. Latency improves and hit rate degrades, and the two pull against each other. Tiering is what reconciles them: the edge stays close to users and the mid-tier holds a larger working set fetched on behalf of all the edges beneath it.

"How do you know you need tiering?"

Measure how many PoPs fetch the same object from origin within its TTL. In one case 84 percent of origin fetches were for an object another PoP had already fetched, with a mean of 11.3 PoPs per object per TTL. That number comes straight from CDN logs and it is the direct measure of duplication that tiering removes.

"When is tiering wrong?"

When nothing is shared between PoPs, because then the extra hop is pure added latency on every miss: genuinely per-user content, very short TTLs, or an object population so large that the mid-tier cannot hold a useful working set either. The rule is to tier when the same object is requested from multiple PoPs within its TTL, which is most content and not all of it.

"What belongs at the edge?"

Decisions rather than computation, and anything that lets you reject work early. Cache-key normalisation, JWT signature verification, redirects, A/B assignment, bot filtering, image transformation. In one case moving auth rejection to the edge removed 2.4 million daily requests from the origin, mostly bots. The bright line is state: a database call from the edge is a database call from 300 places, and one team measured 9,000 concurrent replica connections before reverting.

"How do edge KV stores change that?"

They widen the rule from "no state at the edge" to "no strongly consistent state at the edge." Eventually-consistent replicated key-value reads are genuinely viable for configuration, feature flags, routing tables and staleness-tolerant lookups. Transactions, read-your-writes and anything where a stale value is a correctness problem still belong at the origin.

Common misconceptions

"More PoPs is strictly better." Better latency, worse hit rate per PoP. Without tiering, origin load scales with PoP count rather than with content.

"The origin shield should be near users." Near the origin. Its job is to consolidate origin fetches, and putting it near users means the origin fetch crosses the distance anyway.

"Tiering is free." It adds a hop on a full miss, so content that is never shared between PoPs pays latency for nothing.

"Edge compute is just a faster server." It has tight CPU limits, no durable local state, and it runs in hundreds of places at once, which is what makes a database call from it a capacity problem for the database.

"A purge is cheap." A purge of a popular object at 300 PoPs is an instant 300-request origin spike, and a catalogue-wide purge is that multiplied by the object count. Shielding collapses the duplication and surrogate keys reduce the scope.

Interview delivery note

Say this verbatim: "Without tiering, every PoP is an independent cache, so a miss on one object at 300 PoPs is 300 origin requests. The measurement that decides it is how many PoPs fetch the same object within its TTL: in one case 84 percent of origin fetches were duplicates, with a mean of 11.3 PoPs per object, and tiering plus a shield took origin traffic from 34 percent to 3." The mechanism, the measurement that proves it, and the result.

The senior-versus-staff separator is knowing that more PoPs reduces the per-PoP hit rate. A senior engineer configures tiering because the vendor recommends it. A staff engineer explains that geographic distribution and cache efficiency pull against each other, that the hierarchy exists to reconcile them, and therefore that a 300-PoP CDN without tiering can offload less than a 30-PoP one. That reframes tiering from an optimisation into a correction for a cost you already paid.

The second signal is the state boundary at the edge. Saying "a database call from the edge is a database call from 300 places, and one team measured 9,000 concurrent replica connections before reverting" is a concrete number for a constraint people usually state abstractly.

The Staff & Team Lead Interview Reference