Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

Scatter-Gather

Send one request to several responders at once and return one answer by a deadline taken from the caller's own time budget: the best of whatever came back, labelled as partial when it is, at the cost of asking several responders every time.

A price in three seconds, from six carriers

A freight broker's quote page makes the shipper one promise: describe a pallet shipment, 480 kg from Lyon to Milan, collected on Thursday, and see a price within 3 seconds. The broker does not set that price. Six road carriers do, each through its own rate API, and for the same pallet on the same lane their prices can be far apart. A good quote means asking more than one of them.

Asking them one after another does not fit the promise. The carriers' APIs usually answer in 300 to 900 milliseconds, so six calls in a row take from just under two seconds to over five before the broker has done anything with the answers. The six questions do not depend on each other, though. They can all be asked at once.

Ask everyone, answer once

Scatter-Gather is that move: one request goes to several responders at the same time (the scatter), their replies are collected (the gather), and the caller receives a single answer built from them. Here every carrier gets the same request, the pallet's details, and the answer is the one price the page shows.

One quote round, three carriers

The quote round sends the same request to each carrier, gathers the replies until its deadline, and answers the page with the best quote and the list of carriers that did not reply.

NoteBuilt from a Recipient List or a Publish-Subscribe Channel (scatter) plus an Aggregator (gather).

The two ways to scatter differ in who decides who is asked. With a Recipient List the broker computes the set of carriers for each request, leaving out those that do not serve the lane or the weight, so it knows how many replies to expect. With a publish-subscribe channel any subscribed carrier answers, and the broker may not know in advance how many will. The gather half is an Aggregator: it groups the replies by the quote's id, decides when the group is complete, keeps its state while it waits and deals with replies that turn up after it has closed. What follows is what changes when someone is waiting on the other side of that group.

The deadline belongs to the page

Every reply is worth having, so the first instinct is to wait for all six. Then one carrier's API has a slow afternoon and takes 11 seconds to answer, and every quote on the page takes 11 seconds, whatever the other five said. Waiting for everyone makes the slowest responder the speed of the whole request.

So the gather needs a deadline, and the number comes from the caller's own budget rather than from a guess. The page promised 3 seconds. The broker spends about 300 milliseconds receiving the request and looking up the lane, and needs about 400 at the end to add its margin, save the quote and send the page its answer. That leaves 2,300 milliseconds for the carriers, so the gather closes 2,300 milliseconds after the scatter, with whatever has arrived. Each request can also tell the carrier how much time is left, so a carrier that reads it can stop working on an answer nobody will use; passing a remaining budget down a chain of calls is in Timeout.

A carrier reply that comes in 2,600 milliseconds after the scatter finds the gather already closed; what happens to a part that arrives after its group has closed is settled in Aggregator.

The best of four is not the best

At 2,300 milliseconds the broker holds four quotes. Picking one takes more than sorting by price.

  • Quotes are not comparable as they arrive. One carrier prices in euros with the fuel surcharge included, another sends a base rate and a surcharge percentage, one includes the tail lift the shipper asked for and another charges it as an extra. Each reply is converted into one shape before any comparison.
  • "Best" is defined before the replies arrive, from what the shipper asked for. If the pallet must be in Milan by Monday, €190 with four working days in transit loses to €214 with two.
  • Some replies are not quotes: "lane not served", an error, a price outside any plausible range. They tell the gather that a carrier has answered, but none of them is a candidate.

Then the answer has to say what it is. Two carriers did not reply, so €214 is the best of four quotes, not the best price on the market that Thursday, and the page shows "4 of 6 carriers responded" next to it. Presented as complete, it invites a salesperson to promise the lowest rate available when the carrier that timed out would have been cheaper. When no usable quote arrives in time, the page needs a decision made in advance, an honest error or the published rate card labelled as an estimate, which is the question Fallback answers.

Six calls for every quote

Scatter-Gather spends other people's capacity. Every quote request becomes six carrier calls, and most quotes never become a booking: shippers compare, change the weight and ask again. If one quote in ten is booked, the broker makes sixty carrier calls for every pallet it moves. Carrier rate APIs usually come with rate limits, the carrier computes every quote whether or not anyone books it, and the broker's own outbound connections grow with each responder it adds.

Two habits keep the fan-out in proportion. The recipient list asks only the carriers that serve the lane and the weight, which on Lyon to Milan may be four instead of six. And a carrier that misses the deadline or fails most of the time comes off the list for a while, instead of being called on every request only to have its reply thrown away.

Same shape, different question

Gateway Aggregation draws the same fan-out but asks each service something different (a shipment's tracking, its invoice, its documents) and puts the pieces side by side, while Request Hedging repeats one request to a second replica only when the first is slow and keeps whichever interchangeable answer arrives first. Scatter-Gather asks every responder the same question because their answers differ, and that is why it has to choose among them or merge them.

Drawing the quote round

The Scatter-Gather Challenge asks for this design: a requester broadcasting one request, independent recipients such as competing quote providers, a gather step, and an aggregation that chooses or merges the best answer. Draw the broker's version, with the quote page as the client and the carriers as services. Then give the most attention to the last requirement, a label on the gather step's timeout. Write the number the page's budget leaves, such as "closes at 2,300 ms", not a round figure: that label is where the drawing shows that the deadline belongs to the caller, and that the wait for carriers stops there, leaving the time the broker reserved to finish the answer.

Practice this