Retry Budget

resilience

Definition

Bound retry amplification mechanically by giving each client a local budget — typically a token bucket — that retries spend. While tokens remain, transient failures are masked freely; when the bucket empties, retrying continues only at a fixed, low rate. The budget converts the retry storm from a behavioral risk (hoping clients back off enough) into an arithmetic bound: no client can more than marginally multiply its offered load, no matter how long the dependency stays down.

The pattern's strongest form ships the budget as default platform behavior — in the SDK, the service mesh, or the RPC framework — so the safe behavior is ambient rather than per-team discipline.

When it applies

  • Client fleets calling shared dependencies whose failures are sometimes caused by overload — where retries would amplify the cause
  • Platforms (SDKs, meshes, RPC layers) positioned to make bounded retrying the default for every consumer
  • Systems where backoff and jitter alone cannot bound aggregate retry load during extended outages

Tradeoffs

  • Budgets throttle retries during long runs of genuinely transient faults, where more retrying would have been safe
  • The bound is per-client and local; aggregate fleet amplification is emergent, not centrally guaranteed
  • Sizing the bucket is a policy decision — too small masks nothing, too large bounds nothing

Seen in

  • Amazon Builders' LibraryNov 25, 2019

    The Selfish Retry: Timeouts, Backoff, and Jitter at Amazon

    Amazon's local token bucket bounds each client's retry amplification mechanically: retry freely while tokens remain, at a fixed rate once exhausted — shipped as default AWS SDK behavior in 2016 so the safe behavior is the ambient one. The pattern is the difference between advising clients not to storm and making storms arithmetically impossible per client.