While the Rest Is on Fire: Stripe's Layered Rate Limiters
Stripe runs four admission-control layers in production, with a clear division of labor. Two are rate limiters that shape each user's pace day to day: a token-bucket request limiter (each user draws from a steadily refilling allowance of requests) that rejects millions a month, and a concurrent-requests limiter that caps how many of a user's requests can be in flight at once. The other two are load shedders. They fire only during incidents, and instead of looking at any one user, they look at the whole system and decide which traffic gets the capacity that is left. The shedders encode a criticality ladder: critical methods like creating a charge are protected by a permanently reserved slice of the fleet, and when workers back up, a four-tier drop order (test mode first, critical last) sheds from the bottom and restores slowly, so the system doesn't thrash between dropping traffic and letting it back in. The whole design serves one goal, in the post's words: keep the core of your business working while the rest is on fire.
Flood the API with a runaway test script and watch charges die first-come-first-served — then arm the layers one at a time: the per-user limiter catches the script, the fleet reservation holds 20% for charges through an internal slowdown, and the worker shedder walks the ladder. Flip FLAP MODE to shed and restore fast, and meet the oscillation Stripe tuned away.
Problem
The post opens with the everyday situations that make rate limiting necessary. One user spikes and everyone else still has to stay up. A misbehaving script, or a deliberate attacker, floods the API. A user sends a high volume of low-priority requests, and the post names the collision directly: someone pulling large amounts of analytics data can starve the critical transactions of other users. Or something inside Stripe goes wrong, normal capacity can't be served, and low-priority requests have to be dropped. In every case the same question decides the outcome: when there isn't room for everything, what gets in?
The post draws its central distinction before offering any machinery. A rate limiter controls the pace of one user's traffic, and it is only the right tool when pacing is negotiable, meaning clients can spread their requests out without changing the result, which Stripe finds true for almost all of its batch-heavy payment traffic. A load shedder is a different instrument: it decides from the whole state of the system rather than from who is asking, it fires rarely and only in emergencies, and its job is explicitly triage. Rate limiters keep users fair to each other on a normal day; load shedders choose what survives an abnormal one. A system with only the first still has no answer for the day capacity itself collapses, and a system that sheds without a model of what matters answers with whatever happened to be in the queue.
Solution
The four layers, in the order Stripe recommends building them. The Request Rate Limiter restricts each user to a set number of requests per second using token buckets in Redis: one bucket per user, tokens dripping in at a steady rate, one token spent per request, and an empty bucket means the request is rejected. It is the workhorse, constantly triggered, rejecting millions of requests a month, mostly runaway test-mode scripts. Two refinements carry the operational lessons: rate limiting behaves the same in test and live modes, so scripts don't pick up different behavior on the way to production, and after studying traffic patterns Stripe added brief burst allowances above the cap for real spikes like flash sales.
The Concurrent Requests Limiter changes the unit from rate to how many are in progress at once: not 'a thousand a second' but 'twenty running at the same time.' It guards CPU-heavy endpoints, where a few slow responses breed retries that pile still more demand onto the already-strained resource. Stripe reports it fires far less often (about 12,000 requests a month) and that it fully solved a recurring resource-contention problem. It also, on purpose, nudges integrators toward a better pattern: run a pool of workers draining a queue, rather than hammer the API and back off on rejections.
The Fleet Usage Load Shedder is where the criticality ladder becomes structure. Traffic divides into critical methods (creating a charge) and non-critical ones (listing charges); a Redis cluster counts how many of each are in flight; and a fixed fraction of the fleet is reserved for critical traffic at all times. With a 20% reservation, any non-critical request beyond its 80% allocation is rejected with a 503 (the status code that says the server is shedding load, as opposed to a 429, which says a single user is being paced). The post is candid about how rarely it fires, and that rarity is the point: it triggered for only a tiny fraction of requests this month, and in other months it has prevented outages.
The Worker Utilization Load Shedder is the last line of defense, watching each worker's spare capacity and shedding when a box is too busy to keep up. Traffic sits in four categories (critical methods, POSTs, GETs, test-mode traffic) and shedding starts at the bottom: drop test mode; if that restores health, slowly bring it back; if not, climb the ladder. The word slowly is load-bearing in both directions, and the post's own dialogue makes the failure vivid: shed fast and restore fast and you flap, 'I got rid of test-mode traffic! Everything is fine! I brought it back! Everything is awful!' The shed rate was tuned by trial and error to move a lot of traffic within a few minutes. Only 100 requests were rejected by this layer in the month reported, and it has repeatedly bought faster recoveries during real incidents. The first three layers prevent trouble; this one is damage control.
Around all four sits the operational discipline the post insists on. The limiters hook into middleware so that any bug or a Redis outage fails open and the API keeps serving. Rejections return deliberate, meaningful status codes (429 for pacing, 503 for load shedding). Every limiter ships behind a kill switch. And each one is dark-launched first: run in a watching-only mode that logs what it would have blocked, tune the thresholds against real traffic, and sometimes work with specific users to fix their integration before enforcement ever begins.
Tradeoffs
- Rate limiters only work when pacing is negotiable, and the post scopes its own tool honestly: if clients can't spread their requests out without changing the result (real-time events), a limiter doesn't apply and the real answer is more capacity. That precondition is a property of the workload, not the code. Stripe's batch-heavy payment traffic happens to satisfy it almost all the time, which is why the tool fits so well here and might not elsewhere.
- The criticality split is a business judgment wearing an engineering hat. Someone has to decide that creating a charge is critical and listing charges is not, keep that classification current as the API grows, and pick the reservation number: the 20% of the fleet that analytics can never touch is 20% of steady-state capacity paid for as insurance. A wrong or stale classification quietly turns the protection into misallocation, protecting the wrong thing while charging you for it.
- Shedding has to move slowly in both directions, and that stability is paid for in reaction time. Fast shedding plus fast restoration produces the flapping loop the post narrates; the tuned compromise (move a lot of traffic within a few minutes) means the first minutes of an incident are absorbed, not deflected. A steadier control loop and a slower one are the same setting: you can't have the calm without the lag.
- The protection layer must fail open, which means it is absent exactly when its own infrastructure fails. Catching errors at every level so a Redis outage can't take down the API is clearly right, and it means the limiters offer no protection during a failure of the limiter stack itself. The safeguard is designed never to become the risk, and the price of that is guaranteed windows where nothing is guarding the door.
- The concurrency limiter shapes how clients are built, not just how fast they go. 'Twenty in flight at once' pushes integrators toward running a pool of workers draining a queue instead of hammering and backing off, which is a better citizen for expensive endpoints but a real migration cost for users who built the other way. Choosing a limiter type is choosing a programming model for everyone downstream of you.
- Dark launching treats every limiter as guilty until calibrated. Watching what a limiter would have blocked before it blocks anything, tuning thresholds against real traffic, and working with individual users to fix their integrations costs weeks of patience per layer, and the alternative is discovering the right thresholds on live customers. The kill switches say the same thing: the protective machinery is assumed capable of becoming the outage.
Patterns in this article
- Priority-Aware Load Shedding
When a system has to drop load, it decides in advance which traffic matters most and drops the least important first, rather than dropping whatever happens to be in the queue. Stripe's version is the most legible example on the site: a fixed slice of the fleet is permanently reserved for critical methods, and a four-tier ladder (test mode, then GETs, then POSTs, then critical) sheds from the bottom up. The reserved-capacity idea is what makes it distinctive: priority is enforced by setting aside capacity ahead of time, not only by choosing a drop order once the pressure hits. Uber, Netflix, and DoorDash each solve the same problem differently, with tiers and controllers, playback protection, and priority headers respectively; Stripe's is the plainest to read and, from 2017, the earliest.
- Feedback-Controlled Load Management
The worker shedder is a feedback loop: it watches how busy the workers are and sheds more when they're overloaded, less when they recover. Its lesson is a tuning scar Stripe published: if it sheds and restores too fast, it oscillates, dropping traffic, seeing health return, restoring it, and immediately overloading again ('I brought it back! Everything is awful!'). The fix is to move slowly in both directions, which trades some reaction speed for stability. Uber and DoorDash build the same kind of loop with more formal control math (PID, AIMD); Stripe found the right damping by trial and error and, usefully, wrote up the failure it hit on the way.
- Layered Admission Control
The article's whole architecture is layers: two per-user rate limiters that prevent trouble day to day, in front of two whole-system load shedders that react during incidents. Each layer is scoped to a different job and fires at a different frequency, from millions of rejections a month at the first layer down to about a hundred at the last. The point of the arrangement is that each layer exists so the next one rarely has to fire. This is the same shape Stripe's own AWS-side cousin describes as protection in layers, built out here as four concrete, separately tunable stages.
Also solving this
Other systems in behindscale's Priority-blind load shedding class: