Designing a URL shortener: the short code is the easy part

Ask anyone to design a URL shortener and the first answer arrives in seconds: store code → url, look the code up, redirect. That answer is correct, and it is also the smallest part of the design. A shortener is a good exercise precisely because its core is trivial — every decision that matters is about what happens around it: which path has to be fast, which work can wait, and which failure is allowed to reach the person who clicked.

This post builds one architecture in four stages. Each stage starts from a problem the previous one cannot handle, adds only what that problem requires, and says what it costs. If you already know the territory, jump to the final architecture. If you are here to learn, read it in order: the final diagram is only obvious once you have seen why each box arrived.

The assumptions

Numbers decide architecture, so here are the ones this design works from. They are assumptions for the exercise, round on purpose — not measurements of any real service:

• 50 million new links a month, about 20 creations per second on average.
• 100 redirects for every link created: roughly 2,000 redirects per second on average, with peaks ten times higher when one link spreads.
• About 500 bytes per link (the long URL, the code, an owner, timestamps): around 25 GB of new data a month, 300 GB a year.
• One click event per redirect, kept for analytics.

Two things already stand out. Writes are tiny: 20 per second is nothing for any database. And the redirect is the product: it is the request real people wait on, thousands of times a second, standing between them and the page they actually wanted. Creating a link can take a few hundred milliseconds without anyone noticing; a redirect that slow makes every link feel broken.

The worked design in Learn starts from a smaller service, with different numbers. The shape of the answer is the same, which is the point: the estimate changes the sizes, not the reasoning.

Stage 1: make it work

Start with the smallest thing that is correct. A client talks to a stateless service, and the service stores links in a relational database. There are two operations: POST /links stores a long URL and returns a code, and GET /{code} looks the code up and answers with a redirect.

Even this first version runs at least two instances of the service behind a load balancer. That is not about scale — one instance could serve this traffic — it is the minimum for availability: with a single instance, every deploy and every crash is an outage. The service keeps no state of its own, so any instance can answer any request, and adding instances later is a configuration change rather than a redesign.

Learn moreWorked design: a link shortenerLoad balancing & horizontal scalingUnique ID generation

The short code itself

The data model is a single table — code, long URL, owner, creation time — with the code as its primary key. The only real decision is how codes are made, and there are two reasonable answers.

Random codes. Draw seven characters from an alphabet of 62 (base62: 26 lowercase letters, 26 uppercase letters and 10 digits), insert, and retry if the unique constraint rejects the row. Seven characters give about 3.5 trillion combinations, so even after a year of this traffic a collision is rare and the retry seldom runs. Random codes are also impractical to guess, which matters more than it seems: with sequential codes, anyone holding one link can try the codes next to it, and people often shorten links they never meant to publish.

Counter-based codes. Take the next number from a sequence the database already provides and encode it in base62 — no collisions, no retries. The cost is predictability, since consecutive numbers give consecutive codes. The usual fix is to pass the number through a reversible scramble, a keyed permutation, before encoding it: the codes stay unique but are no longer in order.

This design uses the scrambled counter: uniqueness by construction, codes nobody can enumerate, and no new component, because the number comes from the database we already have, in the same transaction as the insert. Random codes would be just as defensible. What matters is that the choice is small, contained and easy to change later.

Stage 2: the redirect becomes the hot path

At 2,000 redirects a second the first stage is fine. At the peak — 20,000 a second while one link is everywhere — every redirect is still a database query, and the database has to be sized for the worst minute of the month. That is an expensive way to serve a read that almost never changes.

A link's destination is written once and read thousands of times, which makes it the textbook case for a cache. The service uses cache-aside: on a redirect it looks the code up in the cache first. On a hit it answers at once; on a miss it reads the database, stores the result in the cache with a time-to-live (TTL), and answers.

The service asks the cache first; the database only answers the misses.

Learn moreCaching layersHot keys & cache stampedes

Three details decide whether this cache helps or hurts:

• The database stays the source of truth. Creating a link writes to the database and returns only after the commit; the cache is filled on the first read. If the cache vanished, every link would still exist — redirects would just be slower.
• The TTL is a statement about change. Links rarely change, so a long TTL — hours or days — is safe. When a link is deleted or its destination edited, the service deletes the cache entry too, and the TTL bounds how long a missed deletion can survive.
• Unknown codes get a cached answer too. Scanners request random codes constantly. Caching “not found” for a minute keeps them off the database. Only a real “not found” is cached that way — never a database error — and creating a link deletes any “not found” entry for its code.

The cache changes how the system fails, too. If it goes down, redirects fall through to the database — still correct, but slower — and the database is suddenly asked for the full load the cache was absorbing. Either the database is sized to survive that, or the service sheds load (answers some requests with an error immediately instead of queueing them) until the cache is back. Popularity adds a second risk: a viral link is a single hot key, and if its entry expires at the worst moment, a crowd of requests misses at the same time and every one of them goes to the database for the same row. That failure has a post of its own on this blog, linked at the end.

Stage 3: count clicks without slowing redirects

Now the product wants analytics: how many times each link was clicked, from where, and when. The obvious implementation writes a row on every redirect. It is also the wrong one, because it puts a write — slower than the cached read it sits next to — on the latency path of every single click.

The lesson of this stage fits in one sentence: redirecting the reader and recording the click do not need the same latency contract. The reader needs a redirect in milliseconds. Analytics needs the click to arrive eventually, counted correctly. So the service publishes a small click event to a queue and answers the redirect without waiting for anything else. A worker consumes the queue and writes the events, in batches, to a store built for analytics.

Learn moreMessage queues & async workersIdempotency & exactly-once illusions

The assumptions explain why clicks get their own store. Links grow by 25 GB a month. Clicks arrive at 2,000 a second — about 170 million events a day — and at roughly 100 bytes each that is around 500 GB a month, twenty times the links. The largest dataset in a URL shortener is not the links, and the database that answers redirects should not also be the one counting clicks per country for last week.

The asynchronous boundary has costs, and they are worth stating plainly:

• Delivery is at least once. A worker that crashes after writing a batch but before acknowledging it will receive that batch again. Each event carries an ID and the analytics side ignores IDs it has already seen, so the consumer is idempotent and a retry cannot double a count.
• Counts lag. A click appears in analytics seconds after the redirect, or longer if the worker falls behind. For analytics that is fine, and it is a decision rather than an accident.
• The queue can fail. If publishing fails, the redirect still succeeds: the service can hold a few seconds of events locally and drop the rest. Losing a handful of clicks during an outage is the price this design pays so that an analytics problem never becomes a redirect problem.

Stage 4: when the database is the single point of failure

Look at what is still alone. The service runs several instances. The cache can disappear without losing data. The queue absorbs a slow or broken worker. The database is one machine holding the only copy of every link: if it fails, creation stops and every cache miss fails, and a lost disk loses everything written since the last backup.

The fix is a standby replica that receives a copy of every write shortly after it is committed, and can be promoted to take the primary's place. Notice what it is for: availability and durability, not read capacity. The cache already took the read load off the database, so a replica added “for reads” would mostly sit idle. A replica and a cache answer different questions, and another post on this blog explains that distinction.

Learn moreReplication & read replicasSharding & partitioning

The replication here is asynchronous, which is a trade-off of its own: the primary confirms a write without waiting for the replica. If the primary fails, the last moments of writes may never reach the replica, so a link created a second before the failure could be lost. Synchronous replication closes that gap at the price of slower writes and a creation path that depends on two machines. At 20 creations a second either is affordable; the choice depends on whether “your new link disappeared” is acceptable once in a long while.

What this stage deliberately does not add is sharding. Splitting the links across several databases solves a different problem — more data or more writes than one primary can hold — and at 300 GB a year and 20 writes a second that problem is years in the future. When it does arrive, the partition key is already obvious: every lookup is by code, so partitioning by a hash of the code makes each redirect touch exactly one partition. Partitioning by ranges of the raw sequence number instead would send every new link to the same newest partition; hashing the key is what spreads the writes.

The final architecture, read by its failures

Four stages and nine boxes, each of them there for a reason the stages gave. The most useful way to read the final diagram is by failure — what breaks, and who notices:

• A service instance dies → the load balancer stops sending it traffic. Nobody notices.
• The load balancer fails → everything fails. It is drawn as one box because it is usually a managed, redundant service rather than a machine you run; if it is a machine you run, it needs a pair.
• The cache is down → redirects are served from the database, more slowly, for as long as the database can take the load. Nothing becomes incorrect.
• The queue or the worker is down → redirects are unaffected; clicks are buffered, delayed or, in a long outage, partly lost.
• The analytics store is slow → the worker falls behind and counts lag. Redirects do not care.
• The primary database fails → link creation stops until the standby is promoted. Popular links keep redirecting from the cache; the rest fail until the promotion completes.

That list is the real design. The boxes are the means; the decisions are which path each failure is allowed to reach. The redirect depends on as little as possible — the load balancer, the service, the cache, and the database only for misses — and everything else is attached to it asynchronously.

What this design leaves open

Some decisions belong to the product rather than the architecture, and a design review should name them instead of pretending they are solved:

• Abuse. A shortener hides where a link goes, which makes it attractive for phishing and malware. Real services check destinations when a link is created and rate-limit who can create links.
• Custom aliases. Letting people choose their own code turns it into a user-supplied unique key, with reserved words and squatting to deal with.
• Expiry and deletion. Links that expire need a cleanup job and a cache that forgets them on time.
• Regions. Readers on several continents would want the cache, and perhaps the redirect itself, closer to them. The traffic in this exercise does not need it yet.

The short code took two paragraphs. Everything else — the hot path, the asynchronous edge, the source of truth, the failure each box is allowed to cause — is the actual design, and it is the part that carries over to every other system you will draw.