Pattern · seen in 2 breakdowns across 2 companies

Retry Budget

Definition

Bound retry amplification mechanically by giving each client a local budget — typically a token bucket — that retries spend. While tokens remain, transient failures are masked freely; when the bucket empties, retrying continues only at a fixed, low rate. The budget converts the retry storm from a behavioral risk (hoping clients back off enough) into an arithmetic bound: no client can more than marginally multiply its offered load, no matter how long the dependency stays down.

The pattern's strongest form ships the budget as default platform behavior — in the SDK, the service mesh, or the RPC framework — so the safe behavior is ambient rather than per-team discipline.

When it applies

01Client fleets calling shared dependencies whose failures are sometimes caused by overload — where retries would amplify the cause

02Platforms (SDKs, meshes, RPC layers) positioned to make bounded retrying the default for every consumer

03Systems where backoff and jitter alone cannot bound aggregate retry load during extended outages

Tradeoffs

Budgets throttle retries during long runs of genuinely transient faults, where more retrying would have been safe

The bound is per-client and local; aggregate fleet amplification is emergent, not centrally guaranteed

Sizing the bucket is a policy decision — too small masks nothing, too large bounds nothing

The same move, 2 ways

Every row is a production system that bet on this pattern — the note says how, in that system's own terms.

LinkedIn Engineering

2023

The server-assisted form of the pattern: Hodor refuses a request before the application code runs, so retrying it on another copy is safe whatever it does. Both caller and server keep a retry budget, an idea from Google's SRE book, to bound the storm risk. When the server's budget runs out, that itself signals a widespread overload and switches retries off entirely, accepting failed requests to protect the traffic still being served. Read the breakdown →

Amazon (AWS)

Amazon Builders' Library

2019

Amazon's local token bucket bounds each client's retry amplification mechanically: retry freely while tokens remain, at a fixed rate once exhausted — shipped as default AWS SDK behavior in 2016 so the safe behavior is the ambient one. The pattern is the difference between advising clients not to storm and making storms arithmetically impossible per client. Read the breakdown →

Problems this pattern answers

The walls where its breakdowns live — each opens the cross-company comparison.

Priority-blind load sheddingLINKEDIN →Retry-amplified overloadAMAZON (AWS) →