Design CDN

Assume a CDN with 60,000 customer zones, 180 points of presence (PoPs) and a mixed workload: many small web assets plus video segments that dominate the bytes. Average response size across all requests is about 60 KB.

Customers interact with two surfaces: a control API for configuration and purges, and HTTP headers on their origin responses that tell the edge how to cache.

Every customer hostname resolves to a small set of anycast addresses that all 180 PoPs announce through BGP. The internet's routing picks the path to the "closest" PoP in network terms, which is usually close in distance too. Inside the PoP, an L4 load balancer spreads connections across edge servers with consistent hashing on the connection tuple, so packets of one TCP or QUIC flow keep landing on the same server.

The edge server terminates TLS, applies the firewall and rate limits, matches the request against the zone's rules and builds the cache key. It then hashes that key to find which server in the PoP owns the object. Sharding cache by key inside the PoP means 26 servers act as one large cache instead of 26 copies of the same hot set. If the owner has the object, it streams it back through the edge server that took the request.

On a miss, the owner asks a parent tier: a handful of large regional hubs, each acting as shield for many edge PoPs. Only a parent miss reaches the customer's origin. The control plane is separate: it stores configurations and certificates, pushes versioned changes to every server, fans out purges, and collects logs. The data plane keeps serving with its last known config if the control plane is unreachable.

There are two ways to steer users. DNS steering answers each lookup with the IP of a PoP chosen for that resolver. It allows fine control over load, but it sees the resolver, not the user, and resolvers can be far from their users. A change also takes as long as the DNS TTL plus whatever caching resolvers do on top.

Anycast announces the same prefix from every PoP and lets BGP choose. Failover is fast: when a PoP stops announcing, routers converge on the next path in seconds. It also spreads attack traffic naturally, because a botnet spread around the world hits many sites at once. The cost is less control. BGP does not know about load, and a routing change can move a live TCP connection to a different PoP, which then has no state for it. In practice, routes are stable enough that this is rare, and QUIC connection ids help.

Most large CDNs mix both. They use anycast for the front door, then tune it with BGP communities and prefix splits when one site is overloaded, and sometimes use DNS to steer very large customers or video traffic with precision.

Cache key design comes first. By default the key is scheme, host, path and the full query string, plus whatever the response lists in Vary. Most real gains come from removing noise: drop tracking parameters such as utm_source, sort the remaining ones, and normalise Accept-Encoding into a few buckets. A single unnormalised parameter can split one object into thousands of copies.

Inside a server, two layers work together: RAM for small, very hot objects and NVMe for everything else. Pure LRU lets a crawl of one-hit objects push out the useful set, so the admission policy matters as much as eviction. A common approach is to admit an object to disk only on its second request within a window, tracked in a small counting filter, and to evict with an LRU variant that protects frequently reused objects.

Large files need a different shape. A 4 GB game patch or an hour of video should not be one cache entry. Store objects in fixed slices of about 2 MB, keyed by object and slice number, and serve range requests by assembling slices. A user who watches the first two minutes then leaves only pulls the slices they watched, and different slices of one file can live on different servers.

Three mechanisms keep origins safe. The parent tier turns misses from many PoPs into a single fetch per region. Request coalescing makes sure that if 3,000 requests for the same uncached object arrive at once, only one goes upstream and the rest wait on it. And stale serving lets the edge return an expired copy while it revalidates in the background (stale-while-revalidate), or for hours if the origin is failing (stale-if-error).

Revalidation is cheap when origins send ETag or Last-Modified: the parent asks "has this changed?" and gets a 304 with no body if not. That keeps expired-but-unchanged content flowing at the cost of one small request.

A purge request lands in the control plane, which assigns it a global sequence number and writes it to a durable log. A fan-out service publishes it to every PoP over persistent connections, and inside the PoP a local relay delivers it to every server. Each server acknowledges, and the control plane reports progress per PoP. Two seconds is achievable because purges are tiny messages pushed over warm connections, not pulls.

Purging by URL is easy; purging by tag is what customers actually need. When an origin response carries Surrogate-Key: product-8812, the server records that key in a local index from tag to cache entries. Purging the tag then removes every page, image and API response that mentioned the product. A soft purge only marks entries stale, so the next request revalidates instead of every PoP hitting the origin at once.

Servers that were offline during a purge must catch up before they serve. Each server remembers the last purge sequence it applied, and on reconnect it replays the missing range from the log. If the gap is too old to replay, it drops the affected zones' cache entirely.