Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

Cache-Aside

The application owns the cache: check it, load from the database on a miss and fill it, and on a write update the database and then delete the key. Why deleting beats overwriting, the race that can still put an old value back, and what bounds it.

"Seat limit reached (5)"

A support ticket for a project-management SaaS: the admin of team 7 upgraded from 5 seats to 20, got the receipt, and the invite for a sixth person was still refused with "Seat limit reached (5)". An hour later it worked, and nobody had touched anything. The cause is a few lines added to make every API request faster: a pattern done halfway.

Ask the cache, then the database

Every API request checks the caller's plan: how many seats, which features. That is one row in PostgreSQL, SELECT plan, seat_limit FROM teams WHERE id = 7, and across all teams the API was running it 8,000 times a second for data that changes a few times a day. So the team put Redis next to the API. The API now asks Redis for team:7 first; on a hit it is done, and on a miss it reads the row from PostgreSQL, writes it into Redis with a TTL, and carries on.

That is Cache-Aside (also called lazy loading). The application owns the cache: it decides when to look, when to load and what to store. The cache sits beside the data path rather than in it. Redis never talks to PostgreSQL and has no idea where its values come from, which is why any cache that can get, set and delete with a TTL will do. Only rows someone asked for take up memory, a miss costs three round trips (cache, database, cache) instead of one, and if Redis goes down, an API that treats cache errors as misses still works by reading PostgreSQL at full load, something Caching layers says to plan for.

The application owns every arrow

Reads go to the cache first and to the database only on a miss, then fill the cache. A write updates the database, then deletes the key. The cache and the database never talk to each other.

Delete the key, don't overwrite it

Now the ticket. The upgrade handler wrote the new plan to PostgreSQL and returned. Nothing touched team:7, so Redis kept serving the five-seat row until its one-hour TTL ran out. Cache-aside hands the write path to the application as well: whatever changes a team must also deal with the cached copy, by overwriting the key with the new row or by deleting it. Overwriting looks better, because the next read is a hit. Then a billing webhook and an admin change team 7's plan at nearly the same moment:

  1. Request A writes seat_limit 10 to PostgreSQL.
  2. Request B writes seat_limit 20 to PostgreSQL, which now holds 20.
  3. Request B sets team:7 to 20 in Redis.
  4. Request A, held up by a garbage-collection pause or a slow network, sets team:7 to 10.

The database says 20, the cache says 10, and it stays that way until the TTL runs out. The database put the two writes in one order; the cache received them in another, and the last set won. A delete has no such problem: two deletes in either order leave the same empty key, and the next read loads whatever the database holds by then. Deleting is also cheaper when the cached value takes several queries to build: an overwrite pays for that build on every write, even if nobody reads the value before the next change.

The order of the two steps matters as well: update the database, then delete the key. Delete first and there is a gap between the delete and the commit in which any reader that misses loads the old row and caches it for a full TTL. For a busy key, that gap almost always catches a reader. And if the delete fails after the commit, because Redis timed out or the process died between the two steps, the old row stays until it expires. Retry a delete that times out; a process that died retries nothing, so keep the TTL as the backstop.

A reader that arrives at the wrong moment

Update-then-delete narrows the window. It does not close it. All it takes is a write that lands while one reader is between its database read and its cache fill:

  1. Reader R misses team:7.
  2. R reads the row from PostgreSQL: 5 seats.
  3. The upgrade commits: 20 seats.
  4. The upgrade deletes team:7, which is already empty.
  5. R writes its 5-seat row into team:7.

The cache now holds the value from before the upgrade, and no later write is coming to remove it. The upgrade had to commit and delete between steps 2 and 5, a short gap unless R is slowed down, so this is rare, but at thousands of reads a second rare things happen. Misses served by a read replica make it far likelier: the replica can return the old row after the primary has committed and the key is gone, for as long as it lags (the lag itself is explained in Replication & read replicas).

What bounds it is the TTL. Whatever the race leaves behind expires, so the TTL caps how long each stale entry can be served, counted from the moment that entry was written. It bounds each fill, not the time since the upgrade: the stall comes on top, and a replica still lagging when the entry expires can load the old row again. That is why every cache-aside key gets one, even when every write deletes its key correctly. Pick it from how long this data may be wrong: a seat limit that is stale for an hour blocks a paying customer for an hour, while one that is stale for 60 seconds costs a retry. Choosing TTLs in general belongs to Cache eviction: TTL, LRU, LFU. And for a decision that costs money, such as accepting the sixth invite, the API can read PostgreSQL directly: invites are rare, and the cache is there for the 8,000 reads, not for that check.

Facebook's memcache deployment, described in the 2013 paper "Scaling Memcache at Facebook", closes this window instead: a miss hands the reader a lease token, and a delete of the key invalidates it, so the stalled reader's set in step 5 is refused. Without leases like these, the TTL is what bounds the race.

The price is every writer

Every writer must remember the delete: the upgrade handler, but also a migration script, a support tool, someone fixing a row from a SQL console. Each one that forgets leaves a stale copy until the TTL. Teams that cannot vouch for every writer drive the deletes from the database's own stream of committed changes; that technique has its own article, Change data capture. A hot key that expires at peak sends all of its readers to the database at once, the burst named in Hot keys & cache stampedes. And when you would rather the cache load from and write to the store itself, turn to Cache read and write strategies.

Practice: five strategies, one owner each

Caching Strategies puts cache-aside beside four other strategies on one Grid, sharing one application and one store, and grades whether each says who owns the store access. Your cache-aside corner passes the test this article set if a reader can replay the ticket on it: the lookup, the fill on a miss, the store written before the entry is invalidated, and a TTL, so that a good run never lets the cache serve a stale entry longer than its TTL. The read path alone is a ten-minute flowchart in Cache-Aside Lookup Flow.

Practice this