Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

API gateway & the edge

Which checks belong in the one entry point every request passes through, which rules stay in the services, and what that shared front door costs to run.

Who checks the key?

An API split across several services has to decide where the shared entry work is done: terminating TLS connections, checking whether the caller's key is valid, counting its requests against its limit. Done in every service, it is done several times, by several teams on several schedules, and the copies drift apart.

A city's transit agency publishes an open API that third-party journey apps are built on: live departures, a journey planner, service disruptions and stop data, each served by its own service with its own copy of all three checks. When a partner's key leaked and the agency revoked it, three services refused it within minutes. Disruptions cached key lookups for a week, and kept answering the leaked key for six more days, until its cached entry expired.

The agency's fix was an API gateway: a single entry point that every client request passes through, which runs those checks once and then routes the request to the service that owns it. The services drop their copies of the checks.

The gateway is in front of the services

Every request passes a load balancer and one of three identical gateway instances, which check the key, the limit and the size, then route by path.

What the front door can decide

The duties that moved have one thing in common: the gateway can decide each of them from the request, the caller's credentials and its own records of keys and counts, without knowing anything about departures or disruptions.

  • TLS termination. One certificate, renewed in one place, at the gateway or at a load balancer just in front of it, instead of four certificates expiring on four dates.
  • Key and token validation. The gateway checks that an API key exists and has not been revoked, or that a token's signature, expiry and audience are valid, and turns the request away before any service sees it. Revoking a key becomes one change, enforced in one place; if the gateway caches key lookups, it should be for minutes, not a week. Tokens raise their own version of the question, how long a revoked one can still work, and Sessions vs tokens: JWT, refresh & revocation answers it.
  • Coarse rate limits. A ceiling per API key, say 50 requests a second, that keeps one runaway client from flooding every service. The algorithms, and the shared counter that lets several gateway instances agree on a count, are in Rate limiting.
  • Request size limits. A journey-planner request with a 40 MB body is refused with a 413 before any service tries to read it.
  • Routing. /departures goes to Departures, /journeys to the planner. Read Gateway Routing for how rules match, which one wins when two do, and how paths are rewritten on the way.

The move of these duties off the services has a pattern name, Gateway Offloading, and two limits. The gateway's checks protect the services only if nothing can reach the services around it, so they sit on a private network or accept only traffic that comes through the gateway. And a valid key says who is calling, not what they may see: whether a partner may read a particular resource stays the owning service's decision. Floods from thousands of addresses and requests crafted to exploit a bug need other layers, set out in Edge protection: WAF, DDoS & defence in depth.

Work that knocks on the gateway's door

Once the gateway exists it attracts work, because it is the one place every request passes. Two requests reached the platform team in the first month.

The partner programme wanted free-tier partners to see unplanned disruptions but not planned engineering works. The gateway could have done it, with a rule that inspects each Disruptions response and drops some entries. But that rule is about what a disruption is, which only the Disruptions team understands. It would live in a configuration the Disruptions team does not deploy, outside the service's tests, and a later change to the disruption format could break it with no test of the service to catch it. It went into the Disruptions service, which learns the partner's tier from the caller identity the gateway passes along, in a header the gateway sets itself after removing any copy the client sent.

The agency's own app wanted a single call returning a stop's departures together with the disruptions affecting it, in a shape tuned to its home screen. Combining several backend calls into one response is a pattern of its own (Gateway Aggregation), and a shape built for one client and owned by its team belongs in a backend for that client, which The BFF tier: gateway, sessions & fan-out follows into production. One shared gateway configuration that makes the app and the partners take the same response shape serves neither of them well.

NoteA rule that needs the domain's data, or has to change when the domain changes, belongs in the service that owns that domain. The gateway routes and enforces policy; services own the domain.

Every request, one component

The gateway now sits in the path of every call, and that has three costs.

The first is a hop: one more network round trip and the gateway's own processing on every request, typically a few milliseconds within one region, paid by every request.

The second is a shared blast radius. Before, a bad deploy of Stops broke stop data. Now a mistake in the gateway cuts clients off from all four services at once: a route file with a syntax error, a certificate nobody renewed, a gateway release that runs out of memory under load. A single gateway instance with no redundancy makes it a single point of failure for the whole API. So it runs as several identical instances, at least two and preferably across availability zones, behind a load balancer (Load balancing & horizontal scaling), with anything they must share, such as the rate-limit counters, in a store they all reach.

The third cost is organisational. Every new service or path prefix needs a gateway change, and if the configuration is one file reviewed by the platform team and released every Thursday, every team's release waits for Thursday. The agency split it: each team owns the routes for its own paths, checked automatically before release (does this path collide with another team's, and does every route point at a backend that exists?), and ships them on its own schedule, to one gateway instance first and then the rest. The platform team keeps what is shared: TLS, key validation and the default limits.

Where you draw the line

The API Gateway Patterns Challenge asks for one entry point that does the three gateway patterns, routing, aggregation and offloading, in at most 10 nodes, with every responsibility labelled with the pattern it belongs to. Model the agency's API there, and the labels become the decision this article kept making: for each duty, the gateway or the service. Expect the planned-works rule to look like a gateway job, since it filters a response on the caller's tier. A design you can defend gives a reason for every placement: for each duty on a gateway node, why it decides no domain rule, even when it combines the services' answers, and for each rule left in a Service, which of the domain's data it needs.

Practice this