The estimate makes most of the decisions
The service takes a long URL, hands back a short code, and sends anyone who opens the code on to the long URL, counting the click on the way. Say 100 million new links a year and 100 redirects for every link created. A year has about 31.5 million seconds, so that is ~3 writes a second and ~300 redirects a second on average. At ~500 bytes per link (the URL, the code, an owner, timestamps), storage is 100 million × 500 bytes ≈ 50 GB a year: one database holds a decade of it. Back-of-envelope estimation has the method behind these numbers.
Those numbers settle most of the design. Writes are trivial. Reads are modest too, but each one stands between a person and the page they wanted, so a redirect should cost one lookup and nothing that can wait. This is a read-path latency problem, not a scale problem. A viral link that multiplies redirects tenfold for an hour, to about 3,000 a second, is work for a cache, not a reason to shard; Sharding & partitioning covers the day the links outgrow one database, which this estimate puts years away.
Where seven characters come from
The API is two calls: POST /shorten takes a URL (and optionally a custom alias) and returns a code; GET /{code} answers with a redirect. The data is one row per link, links (id, url, owner, created_at). The first real decision is how codes are made, and the first idea, hashing the long URL, goes wrong three ways. Two people shortening the same URL get the same code: whose link is it, and whose clicks? A hash cut to seven characters collides anyway, so you need a collision path regardless. And anyone who knows the scheme can hash a URL to learn whether it was shortened, and at which code.
A counter encoded in base62 (26 lowercase letters, 26 uppercase, 10 digits) cannot collide, because every integer has exactly one encoding. Seven characters give 62^7 (62 characters in each of 7 positions) ≈ 3.5 trillion codes. To keep the counter off the request path, an ID generator hands each API instance a range of, say, 1,000 integers; at 3 writes a second a range lasts minutes, so the generator is asked about once per thousand writes. Unique ID generation compares ranges with Snowflake-style IDs and UUIDs.
A plain counter has one visible flaw: consecutive IDs give consecutive codes, so anyone holding one link can walk its neighbours, and people shorten links they never meant to publish, such as a draft document. The fix is a keyed permutation of the seven-character range, built from a real cipher (format-preserving encryption) rather than a homemade scramble, applied before encoding: still one code per ID and no collisions, but the codes no longer reveal their order. It is reversible with the key, so the redirect can turn a code back into its ID and read the row by primary key. The alternative is a random code stored in its own column under a unique constraint, drawing again when the constraint rejects it; never look the code up first, because two requests can both find it free. Even at the end of the first year a draw collides about once in 35,000 (100 million taken out of 3.5 trillion), so at 3 writes a second the retry is free. Both are defensible: the counter needs a generator, the random code needs a retry.
Custom aliases share that namespace: the permutation gives every seven-character code to some ID, so an alias like launch7 can one day collide with a generated code. An alias cannot be turned back into an ID either, so it needs its own unique column, and the redirect has to know which lookup a code wants. Give aliases a shape the generator never produces, such as another length: the two namespaces stay apart, and the shape tells the redirect where to look.
A short code is an address, not a lock
The permutation hides order; it does not make live codes hard to find. A random guess lands on a real link with probability live links ÷ 62^7: after one year, 100 million ÷ 3.5 trillion, about one in 35,000, whichever scheme made the codes, and about one in 6,000 once 600 million are live. A scanner making 1,000 guesses a second finds about 100 live links an hour (1,000 × 3,600 ÷ 35,000), and capping each client's rate (Rate limiting) slows one scanner, not many. So a short code is an address, not an access control: anything behind a link that must stay private needs its own check, such as a login or a signed URL that expires (Authorization: roles, relationships & policies).
The redirect: one lookup, and a status code you choose
At 300 reads a second, 100 reads per write and destinations that almost never change, the read path is the textbook case for cache-aside on code → URL: look in the cache; on a miss, read the database, store the answer with a long TTL, and redirect (Cache-Aside covers the pattern). The database stays the source of truth, and a link reaches the cache on its first read. Editing or deleting a link deletes its cache entry; the TTL bounds how long a stale entry survives, whether the delete was missed or a read racing the edit put the old URL back. Requests for codes that do not exist are misses that reach the database too, and the same dead code tends to come back (a deleted link still shared, a client retrying), so cache the not-found answer for a minute. It does little against a scanner, which rarely asks for the same code twice; that load is what rate limiting is for. Cache only a real not-found, never a database error, and have link creation delete any not-found entry for its code.
Then the response. A 301 Moved Permanently is cacheable by default, so browsers remember it and send repeat visits straight to the destination: fewer requests to your servers, but those clicks never reach you, and an edited destination does not reach a browser holding the old answer until its copy expires, which without caching headers may be never. A 302 Found is not cached unless you say so, so every click arrives and every edit takes effect. This design returns 302, paying for repeat redirects to keep click data. The opposite choice is defensible; making it silently is not.
Counting clicks stays off that path: a database insert per redirect would sit next to a cache read on every click. The service drops a small click event into an in-memory buffer and answers at once; a background task publishes the buffer to a queue, and a consumer adds up counts in batches (High-write ingestion describes that buffer-and-batch shape). The redirect never waits for the queue, not even for its acknowledgement. What that buys is paid for in three ways: counts lag by seconds, a crashed instance loses the events still in its buffer, and a batch delivered twice is counted twice unless the consumer deduplicates on an event ID (Idempotency & exactly-once illusions).
What breaks, and who notices
- The cache is down. With a short timeout on the cache call, redirects fall through to the database: 300 primary-key reads a second on average is a modest load for one database, so they get slower, not wrong, though a viral peak ten times higher is the load to test it against. The sharper risk is one viral link whose entry expires at its peak, sending every concurrent miss to the database for the same row (Hot keys & cache stampedes).
- The ID generator is down. Redirects keep working, and so do writes until each instance's range runs out. The generator must record a range durably before giving it away; one that forgets on restart hands out IDs that already have rows, and those inserts fail on the primary key.
- The click pipeline is slow or down. Redirects do not notice. Click counts lag by design: a few seconds is invisible on an analytics dashboard and buys a redirect that never waits. In a long outage the buffer fills and new events are dropped, so the counts are approximate, not merely late.
Practise it on the URL Shortener Challenge
Everything above is in the brief of the URL Shortener Challenge: the 100:1 read path, click analytics, and codes that are unique and "not guessable". Start in the requirements stage by writing down what that last phrase means here, codes that do not reveal their order, not codes nobody can find. On the Grid, keep creation and redirect as two visibly separate flows, give the click events their own branch off the redirect, and label the redirect edge with the status code you chose. There is no ID generator in its palette; a Postgres sequence that hands each instance a block of IDs (each nextval names one block) plays that part, and Kafka carries the clicks. Diagnose: URL Shortener Under Launch Traffic then sends a flash crowd at a version with no cache, and Pastebin is the same skeleton with bigger payloads: large bodies move to object storage while the row keeps the key and metadata.