Ask anyone to design a URL shortener and the first answer arrives in seconds: store code → url, look the code up, redirect. That answer is correct, and it is also the smallest part of the design. A shortener is a good exercise precisely because its core is trivial — every decision that matters is about what happens around it: which path has to be fast, which work can wait, and which failure is allowed to reach the person who clicked.
This post builds one architecture in four stages. Each stage starts from a problem the previous one cannot handle, adds only what that problem requires, and says what it costs. If you already know the territory, jump to the final architecture. If you are here to learn, read it in order: the final diagram is only obvious once you have seen why each box arrived.
The assumptions
Numbers decide architecture, so here are the ones this design works from. They are assumptions for the exercise, round on purpose — not measurements of any real service:
- 50 million new links a month, about 20 creations per second on average.
- 100 redirects for every link created: roughly 2,000 redirects per second on average, with peaks ten times higher when one link spreads.
- About 500 bytes per link (the long URL, the code, an owner, timestamps): around 25 GB of new data a month, 300 GB a year.
- One click event per redirect, kept for analytics.
Two things already stand out. Writes are tiny: 20 per second is nothing for any database. And the redirect is the product: it is the request real people wait on, thousands of times a second, standing between them and the page they actually wanted. Creating a link can take a few hundred milliseconds without anyone noticing; a redirect that slow makes every link feel broken.
The worked design in Learn starts from a smaller service, with different numbers. The shape of the answer is the same, which is the point: the estimate changes the sizes, not the reasoning.
Stage 1: make it work
Start with the smallest thing that is correct. A client talks to a stateless service, and the service stores links in a relational database. There are two operations: POST /links stores a long URL and returns a code, and GET /{code} looks the code up and answers with a redirect.
Even this first version runs at least two instances of the service behind a load balancer. That is not about scale — one instance could serve this traffic — it is the minimum for availability: with a single instance, every deploy and every crash is an outage. The service keeps no state of its own, so any instance can answer any request, and adding instances later is a configuration change rather than a redesign.
Learn moreWorked design: a link shortenerLoad balancing & horizontal scalingUnique ID generation
The short code itself
The data model is a single table — code, long URL, owner, creation time — with the code as its primary key. The only real decision is how codes are made, and there are two reasonable answers.
Random codes. Draw seven characters from an alphabet of 62 (base62: 26 lowercase letters, 26 uppercase letters and 10 digits) and insert. The unique constraint is the collision check: when it rejects the row, draw again, a few times at most — never look the code up before inserting, because two requests can both find it free. Seven characters give about 3.5 trillion combinations. After a year of this traffic 600 million are taken, so about one draw in 6,000 hits an existing code: a single insert almost never retries, yet the system as a whole still retries around 50,000 times that year. Collisions are routine for the system and harmless to each request, which is exactly what the retry is for. Random codes also have no order: with sequential codes, anyone holding one link can try the codes next to it, and people often shorten links they never meant to publish.
Counter-based codes. Take the next number from a sequence the database already provides and encode it in base62 — no collisions, no retries. The cost is predictability, since consecutive numbers give consecutive codes. The usual fix is to pass the number through a reversible scramble before encoding it — a keyed permutation of the seven-character range: the codes stay unique but are no longer in order.
This design uses the scrambled counter: uniqueness by construction, codes that reveal neither their order nor their neighbours, and no new component, because the number comes from the database we already have. An insert that rolls back leaves its number unused — sequences do not give numbers back, and they need not: codes must be unique, not consecutive. Neither scheme makes a link private, though. With 600 million of the 3.5 trillion codes in use, about one guess in 6,000 lands on a real link, so a scanner finds links either way; a short code is an address, not a permission. Random codes would be just as defensible. What matters is that the choice is small, contained and easy to change later.
Stage 2: the redirect becomes the hot path
At 2,000 redirects a second the first stage is fine. At the peak — 20,000 a second while one link is everywhere — every redirect is still a database query, and the database has to be sized for the worst minute of the month. That is an expensive way to serve a read that almost never changes.
A link's destination is written once and read thousands of times, which makes it the textbook case for a cache. The service uses cache-aside: on a redirect it looks the code up in the cache first. On a hit it answers at once; on a miss it reads the database, stores the result in the cache with a time-to-live (TTL), and answers.
The service asks the cache first; the database only answers the misses.
Learn moreCaching layersHot keys & cache stampedes
Three details decide whether this cache helps or hurts:
- The database stays the source of truth. Creating a link writes to the database and returns only after the commit; the cache is filled on the first read. If the cache vanished, every link would still exist — redirects would just be slower.
- The TTL is a statement about change. Links rarely change, so a long TTL — hours or days — is safe. When a link is deleted or its destination edited, the service deletes the cache entry too, and the TTL bounds how long a missed deletion can survive.
- Unknown codes get a cached answer too. Deleted links, expired links and typos get requested again and again. Caching “not found” for a minute answers those repeats without the database. It does not stop a scanner that tries a different code every time: each guess is a new miss, and limiting how fast one client may ask is the defence there. Only a real “not found” is cached that way — never a database error — and creating a link deletes any “not found” entry for its code.
The cache changes how the system fails, too. If it goes down, redirects fall through to the database — still correct, but slower — and the database is suddenly asked for the full load the cache was absorbing. Either the database is sized to survive that, or the service sheds load (answers some requests with an error immediately instead of queueing them) until the cache is back. Popularity adds a second risk: a viral link is a single hot key, and if its entry expires at the worst moment, a crowd of requests misses at the same time and every one of them goes to the database for the same row. The fixes need no new box: let one request refill the entry while the others wait for it (request coalescing), or keep serving the old value while it is refreshed in the background (stale-while-revalidate). That failure has a post of its own on this blog, linked at the end.
Stage 3: count clicks without slowing redirects
Now the product wants analytics: how many times each link was clicked, from where, and when. The obvious implementation writes a row on every redirect. It is also the wrong one, because it puts a write — slower than the cached read it sits next to — on the latency path of every single click.
The lesson of this stage fits in one sentence: redirecting the reader and recording the click do not need the same latency contract. The reader needs a redirect in milliseconds. Analytics needs the clicks within a minute or so, each counted once, and can afford to lose a few. So the service puts a small click event in an in-memory buffer and answers the redirect at once; a background task in the same service publishes the buffered events to a queue. The redirect never waits for the queue — not even for its acknowledgement. A worker consumes the queue and writes the events, in batches, to a store built for analytics.
Learn moreMessage queues & async workersIdempotency & exactly-once illusions
The assumptions explain why clicks get their own store. Links grow by 25 GB a month. Clicks arrive at 2,000 a second — about 170 million events a day — and at roughly 100 bytes each that is around 500 GB a month, twenty times the links. The largest dataset in a URL shortener is not the links, and the database that answers redirects should not also be the one counting clicks per country for last week.
The asynchronous boundary has costs, and they are worth stating plainly:
- From the queue on, delivery is at least once. A worker that crashes after writing a batch but before acknowledging it will receive that batch again. Each event carries an ID and the analytics side ignores IDs it has already seen, so the consumer is idempotent and a retry cannot double a count.
- Counts lag. A click appears in analytics seconds after the redirect, or longer if the worker falls behind. For analytics that is fine, and it is a decision rather than an accident.
- Before the queue, clicks can be lost. If the queue is unreachable, redirects carry on and events wait in the buffer, which holds a few seconds of them; once it is full, new events are dropped and counted as dropped. An instance that crashes loses whatever its buffer still held. Losing a handful of clicks during an outage is the price this design pays so that an analytics problem never becomes a redirect problem.
301 Moved Permanently lets browsers remember the redirect and skip the shortener on the next visit: cheaper, but those repeat clicks are never seen, and an edited destination does not reach a browser that cached the old one until that entry expires — which, without caching headers, may be never. A 302 Found keeps every click observable and every edit effective. (308 and 307 are the same pair with the request method preserved; for a link that is only ever fetched with GET, they behave alike.) If analytics is part of the product, 302 fits better; the opposite choice is defensible, as long as it is made on purpose.Stage 4: when the database is the single point of failure
Look at what is still alone. The service runs several instances. The cache can disappear without losing data. The queue absorbs a slow or broken worker. The database is one machine holding the only copy of every link: if it fails, creation stops and every cache miss fails, and a lost disk loses everything written since the last backup.
The fix is a standby replica that receives a copy of every write shortly after it is committed, and can be promoted to take the primary's place. Notice what it is for: availability and durability, not read capacity. The cache already took the read load off the database, so a replica added “for reads” would mostly sit idle. A replica and a cache answer different questions, and another post on this blog explains that distinction.
Learn moreReplication & read replicasSharding & partitioning
The replication here is asynchronous, which is a trade-off of its own: the primary confirms a write without waiting for the replica. If the primary fails, the last moments of writes may never reach the replica, so a link created a second before the failure could be lost. With counter-based codes that loss has a second edge: the promoted replica's sequence can lag behind numbers the old primary already handed out, so a lost link's code could be issued again for a different destination — while the old link is still in someone's message, or still in the cache. Moving the sequence well past its last value as part of every promotion closes that. Synchronous replication closes the first gap, the lost write, at the price of slower writes and a creation path that depends on two machines. At 20 creations a second either is affordable; the choice depends on whether “your new link disappeared” is acceptable once in a long while.
What this stage deliberately does not add is sharding. Splitting the links across several databases solves a different problem — more data or more writes than one primary can hold — and at 300 GB a year and 20 writes a second that problem is years in the future. When it does arrive, the partition key is already obvious: every lookup is by code, so partitioning by a hash of the code makes each redirect touch exactly one partition. Partitioning by ranges of the raw sequence number instead would send every new link to the same newest partition; hashing the key is what spreads the writes.
The final architecture, read by its failures
Four stages and nine boxes, each of them there for a reason the stages gave. The most useful way to read the final diagram is by failure — what breaks, and who notices:
- A service instance dies → the load balancer stops sending it traffic. Nobody notices.
- The load balancer fails → everything fails. It is drawn as one box because it is usually a managed, redundant service rather than a machine you run; if it is a machine you run, it needs a pair.
- The cache is down → redirects are served from the database, more slowly, for as long as the database can take the load. Nothing becomes incorrect.
- The queue or the worker is down → redirects are unaffected; clicks are buffered, delayed or, in a long outage, partly lost.
- The analytics store is slow → the worker falls behind and counts lag. Redirects do not care.
- The primary database fails → link creation stops until the standby is promoted — by a failover manager or an operator, once the old primary can no longer accept writes. Popular links keep redirecting from the cache; the rest fail until the promotion completes.
That list is the real design. The boxes are the means; the decisions are which path each failure is allowed to reach. The redirect depends on as little as possible — the load balancer, the service, the cache, and the database only for misses — and everything else is attached to it asynchronously.
What this design leaves open
Some decisions belong to the product rather than the architecture, and a design review should name them instead of pretending they are solved:
- Abuse. A shortener hides where a link goes, which makes it attractive for phishing and malware. Real services check destinations when a link is created and rate-limit who can create links.
- Custom aliases. Letting people choose their own code turns it into a user-supplied unique key, with reserved words and squatting to deal with.
- Expiry and deletion. Links that expire need a cleanup job and a cache that forgets them on time.
- Regions. Readers on several continents would want the cache, and perhaps the redirect itself, closer to them. The traffic in this exercise does not need it yet.
The short code took two paragraphs. Everything else — the hot path, the asynchronous edge, the source of truth, the failure each box is allowed to cause — is the actual design, and it is the part that carries over to every other system you will draw.