Pattern · seen in 2 breakdowns across 2 companies

Layered Admission Control

Guard a service with several layers that decide which requests to let in: cheap per-client rate limiters in front that fire all the time, and whole-system load shedders behind that fire only in emergencies.

resilience

The mechanism

The pattern at its core: two admission layers with different jobs - a per-client rate limiter for one noisy client, a whole-system load shedder for a surge from every client at once - and what goes wrong without the layer a situation needs.

LIVE ARTIFACTTHE PATTERN, GENERALOPEN FULL SCREEN ↗
THE IDEAA single admission mechanism can't do both jobs. A per-client rate limiter stops one client from taking more than its fair share of capacity, but it can't catch a surge where every client stays within its own limit yet the total number of requests overwhelms the service. A whole-system load shedder catches that surge, but on its own, faced with one noisy client, it drops low-priority work from every client instead of just reining in the noisy one. Stack them: the front layer handles per-client fairness and fires constantly, the back layer handles system-wide emergencies and fires rarely, and each exists so the next barely has to.
WHAT TO TRYSwitch between one noisy client and a surge from every client at once, and toggle each layer off. The noisy client needs the front rate limiter; the surge needs the back load shedder. Remove the layer the scenario needs and the service is overloaded, or protected only by shedding unfairly across every client.

Toggle a rate limiter and a load shedder against a noisy client and a system surge - each covers a job the other can't.

Definition

Protect a service with a stack of admission checks - the gates that decide which requests to let in - instead of just one. The stack has two kinds of layer:

  • a rate limiter in front - scoped to one client at a time, it paces each client so no single one takes more than its share of capacity; usually, it fires constantly and cheaply
  • a load shedder behind - scoped to the whole system, it decides under emergency which work deserves the capacity that is left; it fires rarely, only when the system is genuinely in trouble

Each layer has its own scope (one client versus the whole system), its own trigger (a client's pace versus the system's state), and its own firing rate. They are ordered so that each layer existing means the next one rarely has to fire. You can read the stack's health straight from the rejection counts: a healthy stack rejects orders of magnitude more at the front than at the back.

A STACK OF LAYERS
Traffic flows through a per-client rate limiter, then a whole-system load shedder, into the service.
Requests pass through a stack of admission layers: a per-client rate limiter in front that fires constantly, then a whole-system load shedder that fires only in emergencies. Each layer exists so the next rarely fires.

Two nearby patterns are worth telling apart:

  • Priority-Aware Load Shedding decides the drop order within a single shedding decision - which requests to drop first; this pattern decides the stack of decisions, which mechanism gets to reject first and on what evidence
  • Circuit Breaker cuts off the calls a service makes to a dependency when that dependency is failing; admission layers instead decide which incoming requests the service accepts

When it applies

01One client can eat the capacity everyone else needs. On public or multi-tenant APIs, a single client - by accident or on purpose - can burn through the capacity every other client depends on.
02Normal days and incidents need different defenses. On a normal day you want fairness between clients; in an emergency you want to protect the most important work - two different jobs.
03One mechanism is being asked to do two jobs. When a single guard has to be both the everyday abuse limiter and the last-resort emergency triage, it is stretched across two jobs with different scopes and triggers.

Tradeoffs

Every layer is more to build and watch. Each one is configured, rolled out carefully, and capable of misfiring on its own, so the stack multiplies the tuning and monitoring work.
Layers interact: a well-tuned front layer means the back layers almost never fire, so their settings quietly go stale - precisely because the design is working.
Every layer has to fail open, which leaves a gap. If a layer breaks it must let traffic through rather than block everything, so if the whole limiting system fails, the service runs on with no protection at all.

The same move, 2 ways

Every row is a production system that bet on this pattern — the note says how, in that system's own terms.

Amazon (AWS)
Amazon Builders' Library
2019
Protection stacks in layers: WAF and API Gateway at the edge, iptables at the operating system, then the framework, then the code. Each outer layer is cheaper to reject at but knows less about the request; each inner layer knows more but can least afford the work of rejecting. They cooperate so the server that understands the most is shielded from volumes it couldn't even say no to. The catch this article names: cheap early rejection costs visibility, so every layer has to log what it drops and why, and keep false rejections at zero. Stripe's four stacked limiters are the same idea built out. Read the breakdown →
Stripe
Stripe Engineering
2017
The article's whole architecture is layers: two per-user rate limiters that prevent trouble day to day, in front of two whole-system load shedders that react during incidents. Each layer is scoped to a different job and fires at a different frequency, from millions of rejections a month at the first layer down to about a hundred at the last. The point of the arrangement is that each layer exists so the next one rarely has to fire. This is the same shape Stripe's own AWS-side cousin describes as protection in layers, built out here as four concrete, separately tunable stages. Read the breakdown →

Often used together

Patterns sharing breakdowns with this one — derived from co-occurrence, threshold ≥2 shared.

priority-aware-load-shedding ×2

Problems this pattern answers

The walls where its breakdowns live — each opens the cross-company comparison.

Priority-blind load sheddingAMAZON (AWS) · STRIPE →