Cache read and write strategies

Who talks to the database?

With Cache-Aside, the application does. Every service that reads checks the cache, queries the database on a miss and fills the cache itself; every service that writes updates the database and deletes the key. The cache sits beside the data path, and the database has as many clients as you have services.

Four of the strategies here put the cache in the path and hand it that job. Each way of doing so moves three things: when a write counts as done, what a crash loses, and how stale a read can be. A city marathon's live-tracking app, where friends follow runners from their phones, needs four of them at once.

Diagram components: Read-Through Cache, Write-Through Cache, Write-Behind Cache, Refresh-Ahead Cache.

Interactive diagram — open this article in RodGrid to explore it.

Let the cache do the loading

Start with the runner page. The follower app, the results kiosk at the finish and the commentators' screen all read runner:18204. With read-through, they ask the cache for that key and nothing else. On a miss the cache calls a loader, a function registered with it once that fetches the runner from the database, stores the result and returns it. The query, the TTL and the key format live in that one place instead of in three copies that can drift apart.

A miss is no faster; the caller waits on the cache instead of the database. A plain Redis server has no loader, so read-through in front of Redis is a thin layer of your own code that every service calls instead. In-process libraries such as Guava's and Caffeine's loading caches take the loader as a function and merge concurrent misses for one key into one query, but only within one process: 20 processes can still send 20 queries.

The result everyone opens at once

Writes are where the strategies part ways. Runner 18204 crosses the line, the timing system writes finished, 3:41:07, and every follower's phone gets a notification. Under cache-aside that write deletes the cached runner, so the first follower to open the page, seconds later, pays a database round trip for the very result that was just written.

Write-through sends the write to the cache, which writes the database and keeps the new value. The writer is acknowledged only when both have it:

1. The timing system sends runner:18204 = finished, 3:41:07 to the cache.
2. The cache writes the row to the database and waits for the commit.
3. The cache stores the new value.
4. The cache acknowledges the timing system.
5. A follower's page reads runner:18204 from the cache and gets the result without touching the database.

The order matters. If the database write fails at step 2, the cache must not keep the new value, or it would show a finish the database never recorded. If the cache dies between steps 2 and 4, the database has the result but the timing system never heard back and retries, so the retry must be harmless; writing the same result again is. And every write pays the commit plus a hop through the cache, whether anyone reads it or not.

That last cost is why some data should not go through. After the race, each runner's watch uploads a full activity file that will be opened a few times at most, days later. Writing those through the cache only fills memory with entries waiting for eviction, in an order Cache eviction: TTL, LRU, LFU decides. So they are written around: the database gets them, and the cache gets one only if a read ever asks, through the loader. This is cache-aside's write side without the delete, and it shares the stale-copy catch that Cache-Aside explains; an activity file is written once and cannot be cached before it exists, so it never meets that catch.

Eight thousand pings a second

While the race runs, each of 40,000 runners' phones reports its position every 5 seconds: 8,000 writes a second, and the map shows only the latest one per runner. Write-through would wait on 8,000 commits a second, nearly all overwritten 5 seconds later.

Write-behind, also called write-back, acknowledges from the cache and writes the database later. The cache marks the entry dirty, newer than the database, and a background flush writes dirty entries out in batches. A dirty entry must be written before it is evicted, or a full cache drops an acknowledged write with no crash at all. Flush every 30 seconds and a runner's six pings overwrite each other in the cache; only the last reaches the database. 8,000 writes a second become at most 40,000 rows per flush, one per runner, about 1,333 a second on average, sent in batches.

• A crash loses what was not flushed. An acknowledged position lives only in cache memory until the flush, so a dead node takes about 30 seconds of positions for every runner it held, more if a flush had failed. For a runner who pings again in 5 seconds, that hardly matters; a finish time lost this way is a result the runner never gets. Some caches replicate or persist the queue to shrink that window; check whether yours acknowledges a write before or after that copy exists.
• The database lags. The results service, a report, anything reading the database directly sees positions up to 30 seconds old, longer if a flush falls behind, while readers going through the cache see the latest.
• Order can break. If two flushes for one runner overlap, or a failed one is retried late, an older position can overwrite a newer one. Flush each row with the ping's timestamp and let the database keep it only when that timestamp is newer.

So write-behind fits writes that are frequent, overwrite each other and are cheap to lose. Coalescing discards the intermediate pings by design, so the route a runner wants to replay afterwards needs a durable path of its own; building one is the subject of High-write ingestion. The batching itself is the instinct behind write combining in Counting at scale, applied to whole values.

A projection nobody should wait for

Each runner page also shows a projected finish time, computed from the runner's splits and the course profile ahead. The query takes about 800 ms, the result is cached for 60 seconds, and an elite runner's key is read by thousands of followers. Each expiry makes the next reader wait 800 ms, and readers who arrive in that moment pile onto the same miss: a stampede, with its own defences in Hot keys & cache stampedes.

Refresh-ahead reloads a key before it expires. A read late in the entry's life, say after 45 of its 60 seconds, gets the current value at once and starts a reload in the background; when the reload lands, the entry's clock starts again. A runner people keep watching stays cached, and as long as each reload lands before the entry expires, nobody waits on the query. Caffeine's refreshAfterWrite triggers its reload on a read this way; other caches refresh on a timer whether anyone reads or not.

A 10-minute TTL would hide the query too, but by showing projections up to 10 minutes old; refresh-ahead keeps the 60-second freshness and pays for it in queries. That payment is a bet that a key read now will be read again. For a runner nobody checks twice, the reload is wasted work, and a window that opens after 10 seconds instead of 45 recomputes a watched runner about four times as often.

One app, four strategies

Diagram components: Component, Write-Through Cache, Write-Behind Cache, Refresh-Ahead Cache, Data Store.

Runner results are read-through and write-through, positions are written behind, projections are refreshed ahead. Only the activity files go from the app straight to the database.

Read-through pairs with either write strategy, since both still miss on keys evicted or never written, and with write-behind a read through the cache even sees the latest ping while the database lags. Neither pairing sees a write that bypasses the cache: when officials correct a time directly in the database after a protest, the cached runner stays wrong until its TTL ends, which is why runner entries keep one. One way to catch every change, whoever made it, is to follow the database's own log, as Change data capture does.

What each crash costs

The Caching Strategies Challenge sets cache-aside and the four in-path strategies side by side around one application and one store, in at most ten nodes, and its brief asks who owns the store access in each: the application in cache-aside, the cache in the other four. The marathon gives you a test to run on the finished Grid. Take each cache in turn and say what its crash costs. Read-through, write-through and refresh-ahead lose nothing the store does not already have; write-behind can lose every acknowledged write it had not flushed. Someone reading your Grid should be able to tell which cache that is.