Learn

Architecture & pattern guides

Read the idea, build it on a Grid, prove it in a challenge. Short, practical guides — illustrated with RodGrid's own nodes — on system design, design patterns, flow modeling, and how to get the most out of the app.

All articles

Timeout

Bound how long a caller waits: what giving up does and does not undo, connect and read timeouts, choosing the number from measured latency and the caller's deadline, and passing the remaining time down a chain.

"The payment timed out. Was the customer charged?"

The checkout service calls a payment provider with a 2-second timeout. At 2.0 seconds it stops waiting, logs payment call failed: timeout after 2000 ms, and shows the customer an error. The honest answer to the question above is that nobody knows yet. The request may never have reached the provider, the provider may still be working on it, or the charge may have gone through and only the reply was late.

A timeout caps how long a caller waits for a call. When it fires, the caller gives up and reacts with a retry, a Fallback or an error, instead of holding a thread and a connection indefinitely. A remote call with no timeout is the classic start of a cascading failure, because every blocked caller holds a resource until nothing is left for anyone, the story Bulkhead tells in full; so every remote call needs one. It is the foundation the other resilience patterns build on, and Cascading failures: how one dependency takes down its callers shows which link of a spreading failure each of them breaks.

Give up after a deadline

What giving up does not do

A timeout is the caller's decision, made on the caller's side. Usually nothing tells the provider, and it carries on: the charge it was halfway through can complete a few hundred milliseconds after checkout stopped listening. The timeout ended the wait, not the work, so for a write like this the caller has to assume it might have happened. Two safe moves follow. Retry with the same Idempotency Key, so the provider answers with the charge that already went through instead of making a second one, or ask the provider for the payment's status before trying again, keeping in mind that not found a moment after the timeout may only mean the charge is still in flight, and only a definite answer settles it. What is not safe is telling the customer the payment failed and inviting them to pay a second time.

Two waits inside one call

"2 seconds" was one number, but an HTTP client usually has at least two. The connect timeout bounds setting up the connection, which on a healthy network takes a round trip or a few, so it can be much shorter. The read timeout bounds waiting for the response, and in many clients, Python's requests among them, it is the longest gap allowed between bytes, not the time for the whole response: a server that sends one byte a second never trips a 2-second read timeout. Bounding the whole call takes an overall deadline as well. And check the defaults, because some clients have none: requests waits indefinitely unless you pass a timeout.

Picking the number

Measured over a busy week, half of the provider's calls finish within 300 ms, 99% within 1.2 s and 99.9% within 1.8 s. Those are percentiles, the subject of Latency percentiles & tail latency: p99 is the latency that 99% of calls beat. A timeout based on the average, say twice its 350 ms, would cut off a slice of perfectly healthy calls every minute, and if they are retried, the timeout itself adds load. A timeout just above a high percentile, p99.9 plus a margin, which is how 2 seconds was chosen, fails only calls that are already far slower than normal.

The other bound comes from above. The mobile app gives up on checkout after 5 seconds, so checkout's own work plus everything it waits for has to fit inside that. When a dependency's slow tail does not fit inside the caller's budget, no timeout value fixes it, and unless failing the slow tail into a Fallback is acceptable, the call needs a different shape: racing a second request once the first is slower than normal, for reads that are safe to repeat (Request Hedging), or accepting the work now and delivering the result later (Asynchronous Request-Reply).

A budget that travels

Checkout is one hop in a chain. The app's 5 seconds start when the user taps, and the gateway and checkout each spend some of them before the payment call begins. If every hop sets its own fixed timeout, the hops disagree: the gateway may keep waiting on a call that the app abandoned a second ago. Deadline propagation removes the disagreement. The first hop sets the deadline, every call carries the time remaining, and each hop gives its own calls the smaller of its usual timeout and what is left, minus a little for its own reply. A hop that receives a request with almost nothing left skips the work and fails fast, since nobody will be waiting for the answer.

Each hop passes on what is left

The payment call still gets its 2 seconds here, because 4.6 are left. Had the app's deadline been nearly spent, checkout would have used the remainder, or not called at all.

gRPC carries the remaining time with every call that has a deadline, though it sets none by default and some languages pass it on to outgoing calls only when enabled; over plain HTTP it is a header of your own that every hop must read and pass on. Sending time remaining rather than a clock time avoids depending on two machines' clocks agreeing. Where the platform allows, also cancel work nobody is waiting for: a database query may keep running after its client has given up unless it is cancelled or has a timeout of its own.

Timeouts times retries

Retries spend the same budget, and they multiply across layers: if the app, the gateway and checkout each make up to three attempts, one tap can become as many as 27 calls to the payment provider, and the inner layers can keep retrying attempts the outer ones have already abandoned. Retry at one layer, inside the deadline; backoff and which errors deserve a retry belong to Retry, and a Circuit Breaker can count timeouts as failures so a provider that keeps timing out stops being called.

One sum to check

The Resilient Service Calls challenge asks for a Timeout bounding how long each call may hang, alongside a Retry, a Circuit Breaker and a Fallback, in a sensible order. The test of a good answer is arithmetic: with the Timeout inside the Retry, nearest the dependency, the timeout per attempt times the number of attempts, plus every backoff wait, has to fit inside the caller's deadline with a little left for the reply. Write all three numbers on the nodes, and the sum either fits or shows you the bug.

Practice this