Retry with Backoff and Jitter

When a client keeps retrying a call that failed against a shared service, just trying again right away, over and over, is dangerous. The failure might be a brief blip that clears on its own, in which case a quick retry works. Or the server itself might be in trouble, in which case every retry piles on more load at the worst possible time. The client cannot tell which of the two it is facing. Exponential backoff handles both without the client knowing which: after each failure it waits longer than the last (each wait roughly doubling), so a brief blip recovers fast while a real outage sees the retries slow to a trickle.

Backoff by itself is not enough when many clients fail together. One server incident knocks out thousands of clients at almost the same instant, and if they all follow the same backoff schedule they all retry at almost the same instant too. That is the thundering herd: each wave of retries re-creates the very overload that caused the failure. Adding a little randomness to each client's wait (jitter) breaks the clients out of lockstep. Each one waits a slightly different amount, so together their retries spread into a smooth wave the server can absorb while it recovers.

WHY JITTER WORKS

This is one of the oldest tricks in networking, and it shows up everywhere. Ethernet used it to let many machines share one wire, and TCP uses it to space out resends. The AWS and Stripe client libraries do backoff and jitter by default, often bundled with idempotency keys so a retry is both safe and polite.

The bigger point: a retry is not just the client's own business - it is load that the rest of the system has to absorb. Backoff and jitter are the client's half of protecting against overload; server-side load shedding is the other half, and a system under real pressure usually needs both.