Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

CDN & static delivery

How a CDN serves the same bytes to millions of users from points of presence near them: reaching the nearest edge, the hit ratio and the cache key behind it, an origin shield so misses do not all land on the origin, purges that must reach every edge, versioned URLs, and short-lived dynamic responses at the edge.

One bucket, or copies everywhere

The studio behind an online game has a decision to make before its first expansion ships: keep serving patches the way it always has, from an object storage bucket in one region, or put something in front of the bucket. Small updates never strained it. This patch is 18 GB, and about two million players will want it in the first weekend. That is 36 PB, which through a single 100 Gbps link would take about a month to leave the bucket, most of it bound for players an ocean away.

A CDN (content delivery network) is the something in front. It runs servers in many points of presence (PoPs), data centres close to where users are, and each PoP keeps a cache. The launcher asks a nearby PoP for a chunk of the patch. The first request for that chunk there is a miss: the PoP fetches it from the origin, here the bucket, keeps a copy and answers. Every later request for it at that PoP, while the copy is kept and still fresh, is a hit, served without touching the origin. The bucket can stay private, since the CDN fetches with credentials of its own (Object storage & blobs), and the CDN is one of several places a system keeps copies (Caching layers).

Finding the nearest door

First the launcher has to reach a nearby PoP, and CDNs do it in one of two ways, often both. With anycast, every PoP announces the same IP addresses to the internet's routers, and each player's packets go to the PoP nearest in routing terms, usually but not always the closest on the map. When a PoP fails and its announcement is withdrawn, routers move its traffic to the next one without waiting for any DNS record to expire. With DNS steering, the CDN's name servers answer each lookup with the address of a nearby PoP, and a changed answer waits on the record's TTL; how "nearby" is judged, and why from the resolver rather than the player, is in DNS, proxies & the network edge.

The number to watch: hit ratio

On release day the studio watches one number, the hit ratio: the share of requests a PoP answers from its own cache. With every miss going to the origin, at 98% it carries two requests in every hundred; at 90%, it carries ten, five times the load.

Ten minutes after the release, it read under 1%. The launcher added each player's download token to every chunk URL, such as /patch/chunk-0412.bin, as a query parameter t, and the edge, by default, used the whole URL, query string included, as the cache key: the name a copy is stored and looked up under. Two million players made two million keys for one file, and every request missed. The studio changed one rule at the edge: verify the token's signature there, which most CDNs can do for signed URLs or cookies, then leave t out of the key.

A cache key should hold exactly what changes the response. Leave out something that matters, such as a language parameter, and the edge serves one variant to people who asked for another; put in something that does not, a tracking parameter or a session id, and the same bytes are stored under thousands of names. The key is never a way to share what belongs to one player, though: a response built for one person stays off the shared edge altogether, and how the origin says so is HTTP caching: Cache-Control, ETags & revalidation, along with the headers that make other responses miss by design. And each PoP's cache is its own, so one file asked for once in each of 250 PoPs is 250 misses.

Two hundred and fifty misses for one file

That is what the release looks like from the bucket. Every PoP starts cold, and each one's first request for each chunk goes to the origin: 250 PoPs times 18 GB is 4.5 TB, much of it in the same few minutes. An origin shield, one form of tiered caching, adds a layer between them: edges send their misses to a few regional shield caches, and only a shield's miss reaches the bucket. With four shields, the bucket serves each chunk about four times, roughly 72 GB.

Edges miss to a shield

A Seoul miss goes to the regional shield, and only the shield's misses reach the bucket. A purge has to reach both tiers.

Many edges also ask a shield for the same chunk at the same moment, and many CDNs collapse those concurrent misses into one origin fetch, request collapsing. How a burst of misses becomes an outage, and the other defences, is Hot keys & cache stampedes.

Taking a file back

On the second day, a bug report: chunk 0412 was built from the wrong branch. Chunks are named by position, so the corrected one has the same URL, and every PoP holding the old copy has to stop serving it.

Waiting is one way: each cached copy has a lifetime set by the origin's headers, and a TTL bounds staleness with no coordination at all. A purge is the other. The release job calls the CDN's API, and the CDN's control plane delivers the invalidation to every PoP, and to the shields, which would otherwise hand the old copy straight back. Providers take from under a second to several minutes, and until it lands some PoPs still serve the old file. Some CDNs also let the origin tag responses, Fastly with a Surrogate-Key header, so one call purges everything with a tag. A purge does not touch the bucket, either: a file that must not be served at all is deleted there first, or the next miss fetches it right back.

A purge reaches the CDN and nothing else: a launcher or browser that already holds the file keeps it until its own copy expires. So for the next patch the studio names each chunk after a hash of its bytes. A changed chunk is a new URL, the old one never needs purging, and both can be cached for a year. Only the patch manifest, the small list of chunk names in the current build, still changes under a fixed name; it gets a one-minute lifetime and is the one file a release purges. Naming by hash is Content-addressed storage & deduplication.

Answers that change every few seconds

A CDN is not only for files. Every running launcher asks GET /api/servers/status every 30 seconds, about 66,000 requests a second at the peak, with the same answer for everyone. Cached for 5 seconds at the edges and shields, it reaches the origin only a few times every 5 seconds, and unless an expired copy is served, each answer is seconds behind the origin at most, a staleness the studio chose. Sign-in and purchases cannot be cached, but they still pass through the edge, where a nearby PoP ends the player's connection and reuses its own open connections to the origin; why that saves time is Network protocols: TCP, UDP, HTTP/2 & HTTP/3.

Checking a design against the brief

The Content Delivery Network Challenge is this system at 50,000 requests a second, and its brief doubles as a checklist for the design you submit. Does something steer each player to the nearest healthy edge? Does every edge miss pass through a shield before it reaches the origin? Do purges have a path of their own that reaches every edge? And do the edges' logs feed whatever reports the hit ratio? Diagnose: Image Origin Under a Viral Post is the short version, one flash crowd and no edge yet, and Diagnose: Streaming Platform With No Edge Delivery and Video Streaming Platform put the same questions to video segments.

Practice this