Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

Gateway Aggregation

One endpoint at a shared gateway that calls several services and returns one answer, and how to keep that answer on time when a service is slow, with any missing part marked as missing.

The chain the phone was paying for

Here is what a banking app sent when its account overview opened, on a train where each round trip to the bank took 300 ms:

  1. At 0 ms the app asks for the customer's accounts, their cards and their scheduled payments, together, over one HTTP/2 connection.
  2. At about 300 ms the accounts arrive, and only now does the app know which ids to ask about.
  3. At 300 ms the app asks for each account's balance.
  4. At about 600 ms the balances arrive, and the screen can show the numbers the customer opened it for.

The independent requests cost one round trip between them. The chain cost two, because the balance request could not leave the phone until the account list arrived. Inside the bank's data centre the same chain is cheap, a millisecond or so per hop, so the platform team moved it there. The shared gateway that already fronted every service got one new endpoint, GET /overview: the app calls it once, and the gateway calls the services, composes their answers and sends back one response. That is Gateway Aggregation: an endpoint at the gateway that turns one client request into several backend calls, and their answers into one response.

One overview, four services

The balance call waits for the account list; cards and scheduled payments start at once, in parallel with that chain.

What sets the time

The gateway keeps the shape the phone had: calls that need nothing from each other start together, and a call that needs another's answer starts when that answer arrives. With typical times of 30 ms for Accounts, 50 ms for Balances, 40 ms for Cards and 70 ms for Scheduled payments, the longest path is Accounts then Balances, 80 ms, and the response is ready after about that plus the gateway's own work. Called in sequence, the same four would take 190 ms.

So independent calls cost about the slowest of them, and each chained step adds its own time on top. When Balances added an endpoint that accepts a list of account ids, the gateway could ask for every balance in one call instead of one per account; the chain was still two steps, but its second step became one call for any customer whose accounts fit in one batch.

A part that does not answer

One Monday the Cards service slowed down after a release and some of its calls hung, never returning. The first version of /overview set no timeout on its backend calls and had no notion of answering without a part, so every overview waited on Cards. Customers whose accounts, balances and payments were all fine watched a spinner until the app gave up and showed its error screen.

Watch outCommon mistake: fanning out to dependencies with no per-call timeout or partial-result handling, so the whole response waits on the slowest (or a hung) one instead of degrading gracefully.

The fix has two halves. Each backend call gets its own timeout, all of them inside a deadline for the whole response, so a chained step starts with only the time its predecessors left it; how long each should be is the subject of Timeout. And each part is declared required or optional, because the gateway cannot work that out for itself. The overview means nothing without the account list, so Accounts is required: if it fails, the whole response is an error. Balances, Cards and Scheduled payments are optional. Now the overview costs about as much as its slowest path, each call capped by its timeout, rather than the sum of all of them, and a hung Cards service costs the customer the card section, not the screen.

Saying which part is missing

The app has to be able to tell a missing part from an empty one. An empty list of scheduled payments tells a customer that the rent will not leave their account tomorrow, and they may move money on that belief, when in fact the service that knows about the rent did not answer.

HTTP has no status code for "mostly worked" (206 Partial Content belongs to range requests), so a common choice is to answer 200 when every required part is present and put each part's state in the body. The overview carries a status per section, ok or unavailable, and the same field per account, because the batch answer from Balances can lack one balance out of five while carrying the other four. When a required part fails, the gateway answers with an error status, such as 503. The field belongs in the contract from the endpoint's first release, because an app version built without it can draw a missing section as an empty one.

One tap, many calls

Every opening of the app is now several backend calls. With one balance call per account, as in the first version, a customer with six accounts cost nine; the batch endpoint brought it to four, as long as the accounts fit in one batch. A per-item fan-out grows with the customer's data as well as with the traffic, and is easy to miss in a load test whose test customers hold one account each. Retries multiply it again, since an app that retries /overview makes this gateway repeat every call, including the ones that succeeded; Retry covers how attempts at several layers add up.

When the composition belongs to a screen

Later the app team wanted accounts listed in the order each customer opens them most, and the home-screen widget wanted balances only. Each request puts knowledge of one screen into the shared gateway, whose configuration every client depends on. Generic composition, the same parts in the same shape for whoever calls, is fine at the gateway; composition that knows one screen belongs in code owned by that screen's team, a backend for that client. Backends for Frontends (BFF) decides when a client deserves one, and The BFF tier: gateway, sessions & fan-out runs one behind the gateway. The messaging shape that sends one request to many recipients and combines their replies is Scatter-Gather.

On the Challenge

The API Gateway Patterns Challenge asks for one entry point that does routing, aggregation and offloading in at most 10 nodes, with each gateway responsibility labelled with its pattern. Give the Gateway Aggregation node a client edge for one composed request, such as the overview, and an edge to each Service it calls. Label each of those edges with its timeout and with required or optional, and number the edges of a chain so the order shows. A good run calls the independent dependencies in parallel, so together they cost about the slowest one, capped by a timeout, not the sum of all of them; a chained call still adds its time to the call it waits for. Then read your labels as the gateway would on the day one optional Service hangs: the response should still leave on time, with that part marked missing.

Practice this