Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

Back-Pressure

How a slow consumer's "slow down" travels upstream hop by hop, through a blocking queue, the network's own flow control or explicit demand, and what to do at the edge where the source cannot be slowed.

A backfill that outruns its cluster

A backfill job loads two billion historical click events from files in object storage into a search cluster. The reader could produce 80,000 events a second. The cluster normally indexes 50,000 a second, and while it merges its storage files in the background it drops to 10,000. For every second of that merge, 70,000 events have to go somewhere: into a buffer that grows, into the bin, or back to the reader as a reason to send less.

Back-Pressure is the third choice. The overloaded consumer signals upstream, and the producer slows to the pace the consumer can sustain, rather than buffering without limit until memory runs out. Unlike a Rate Limiter, whose limit is set by policy rather than by the consumer, back-pressure carries the consumer's actual capacity back upstream as it changes.

Consumer pushes back

The consumer's signal goes back to the producer instead of into a bigger buffer. In Reactive Streams the signal is a number: request(n) grants n more items, and a producer that has used up its grants waits.

Read from the slow end

The backfill has more than two stages. A reader streams events over a network connection to an enricher service, which adds account details and hands each event to an indexer thread in the same process, which writes them to the cluster in bulk requests. The signal has to cross each of those boundaries, and each crossing works differently.

Data flows right, pressure flows left

Each edge is labelled with how that hop pushes back. Nobody tells the reader to slow down; its writes simply start to wait.

The cluster refuses. The cluster cannot slow the indexer down; it can only say no. When its write queue is full, it rejects writes with 429 Too Many Requests, for a whole bulk request or for single items inside an otherwise successful response. That is an explicit signal, and it becomes back-pressure only if the indexer treats it as one: fewer bulk requests in flight, and retries after a growing delay. An indexer that resends every rejected request at once does the opposite, adding load to a cluster that is already behind; backoff itself is the business of Retry.

A queue makes the enricher wait. The indexer is now slower, and the enricher keeps handing it events. Between two threads of one process, the simplest signal is a bounded queue: when its 5,000 slots are full, the enricher's put blocks until the indexer takes an event. The enricher does nothing to slow down; it simply cannot go faster than the queue drains. How big that buffer should be, and what else a full one can do, is the territory of Producer-Consumer. What matters here is the consequence: the enricher's thread is blocked, so it stops reading from its network connection.

The network closes the window. Nobody wrote code for this hop; TCP does it. Each side of a TCP connection advertises a receive window, how many more bytes it can buffer, and the sender may not have more unacknowledged data in flight than that window allows. When the enricher stops reading, its operating system's receive buffer fills and the advertised window shrinks to zero. The reader's own send buffer then fills, and its next write has to wait: a blocking write blocks, and a non-blocking one reports that it cannot proceed, so the reader must wait rather than pile the data up in memory. So the reader reads its files more slowly, because it is waiting to write. The signal has reached the source, and nothing was lost or buffered without limit on the way.

Built as stages inside a streaming library instead, the same backfill makes demand explicit: under Reactive Streams, the specification that Project Reactor, RxJava's Flowable and Akka Streams implement, a subscriber calls request(n) and the publisher may not send more than the total requested so far. That suits code that must never block a thread, such as handlers on an Event Loop.

One unbounded buffer is enough to lose it

Back-pressure is a chain, and one missing link breaks it. Say the enricher hands events to the indexer through an executor whose queue has no limit, which is what Java's Executors.newFixedThreadPool gives you. The enricher never blocks, it keeps reading at full speed, TCP never closes the window, and during the merge 70,000 surplus events a second pile up in that one queue until the process runs out of memory. Every other hop was right; one buffer without a bound absorbed the signal. A fire-and-forget call, where a stage starts work without waiting for it to finish and without capping how much is in flight, swallows the signal the same way.

Done right, back-pressure has an honest cost: the whole pipeline runs at the speed of its slowest stage. During the merge the backfill crawls at 10,000 events a second from end to end. For a backfill that is fine, because the files will still be there in an hour.

When the source is a person

Feed the indexer live clicks from mobile apps instead, and there is nobody to slow: a phone cannot stop its user from tapping, and a request held too long times out and is often sent again. At that edge the surplus is refused cheaply (Load Shedding), capped per client (Throttling), or absorbed by a durable queue that helps only if consumers keep up on average (Queue-Based Load Leveling). How these choices combine across a system is laid out in Backpressure & load shedding.

Slow down or drop, on a Grid

The Backpressure & Load Shedding challenge asks for both halves: a fast producer, a slower consumer, a bounded queue whose fullness signals upstream, the producer slowing to the consumer's pace, and shedding where slowing is impossible. Its last requirement, making clear when to slow down and when to drop, is the line this article drew between the backfill and the live clicks. A drawing that answers it puts the Back-Pressure node on an edge leading back to the producer, with a maximum size written on the queue.

Practice this