Pattern · seen in 14 breakdowns across 13 companies

Fault Isolation

Fault isolation means putting boundaries between the parts of a system so that when one part fails, the damage stays contained instead of spreading to everything else.

resilience

The mechanism

The pattern at its core: four services, one pool of shared resources, and a toggle that gives each service its own isolated slice.

LIVE ARTIFACTTHE PATTERN, GENERALOPEN FULL SCREEN ↗
THE IDEAFour services draw on one pool of resources. Fault isolation gives each its own dedicated slice, so when one service gets stuck it can only starve itself - not the others.
WHAT TO TRYClick a service to make it STUCK. With ISOLATE off it drains the shared pool and every service starves; turn ISOLATE on and the damage stops at its own boundary.

Break a service, then flip ISOLATE - the blast radius snaps from the whole system to one box.

Definition

Fault isolation is a way of building systems so that when one part fails, the damage is caught at a boundary and cannot spread to the rest. The boundary is the whole idea: you make the lines between components explicit and make sure they hold when something breaks - through dedicated resources, separate data paths, separate locations, or sandboxed code.

Systems naturally grow shared dependencies over time, and a shared dependency is the path a small, local failure travels along to take down everything connected to it. Fault isolation makes the opposite trade on purpose: you spend a little more up front - duplicated resources, some extra work - so that when one part breaks, the breakage stays where it started instead of spreading.

ISOLATED ON PAPER, COUPLED UNDERNEATH
Two isolated services share hidden dependencies below them; a failure in a shared dependency reaches both.
Service A and Service B each sit behind their own boundary, so they look isolated. But both still depend on the same DNS, identity, and control plane underneath, and a failure there crosses the boundary into both.

The same move applies at many layers:

  • Process isolation: each service runs in its own container.
  • Customer isolation: each customer's data and resources are kept separate.
  • Location isolation: run the same work in more than one place - different data centers, regions, or providers - so losing one doesn't take the rest down.
  • Traffic isolation: different classes of traffic get separate network paths.
  • Capability isolation: risky code runs sandboxed, with limited permissions.

The logic is the same at every layer: a failure should not spread past the boundary that is meant to contain it.

THE SAME MOVE, EVERY LAYER
Four panels show the isolation boundary at process, customer, location, and sandbox scopes.
The same boundary shows at every scale: a process in a container, a customer with separated data, work run across more than one location, code in a sandbox. At each layer, one unit's failure is kept from becoming everyone's.

When it applies

01One component's failure can cascade into a broad outage: shared dependencies give a local problem a path to spread system-wide, and cutting that path is the goal.
02One customer must not drag down service for the others: on shared infrastructure a single heavy or misbehaving customer can starve everyone else unless the boundaries hold (noisy-neighbor protection).
03A monitor has to keep working while the thing it watches fails: if it shares machines with what it watches, it fails at the same moment - so it needs its own separate resources.
04You run untrusted or auto-generated code: the host has to survive whatever the guest does, so the guest runs sandboxed with limited reach.
05Sharing resources for efficiency has started causing shared failures: at scale, the savings from sharing are now outweighed by the blast radius when the shared thing breaks.

Tradeoffs

Isolation costs duplicated resources: separate boundaries use more compute, network, and storage than sharing would. It only pays off above a certain scale, or when the stakes are high enough.
It adds operational work: separate boundaries mean separate deploys, monitoring, and runbooks - and extra coordination across teams.
Isolation can be an illusion: a system can look separate while quietly sharing hidden dependencies like DNS, identity, or the control plane. You have to actually find and cut every shared dependency that matters, not just the obvious ones.
Too much isolation backfires: too many separate compartments scatter your debugging, slow down changes that legitimately cross boundaries, and pile complexity onto the boundaries themselves. The right number depends on which failures you are actually trying to contain.

The same move, 14 ways

Every row is a production system that bet on this pattern — the note says how, in that system's own terms.

Airbnb
Airbnb Engineering
2026
Every layer of Airbnb's redesign embodies fault isolation: dedicated compute clusters isolate the metrics pipeline from app workloads, a separate Envoy L7 plane isolates telemetry traffic from business traffic, and the Dead Man's Switch isolates failure detection from the failing system. The same principle applied at three different layers. Read the breakdown →
Discord
Discord Engineering
2025
Cell architecture is fault isolation at the level of a group of clusters. A failure in the BFG cell doesn't touch the user-dm-messages cell; a bad deploy of guild-message routing doesn't break DM search. The redesign sets up a ladder of isolation boundaries (cluster, then cell, then use case) that the original two-cluster setup couldn't express. Airbnb applies the same pattern in its monitoring infrastructure, at a different layer. Read the breakdown →
Netflix
Netflix Technology Blog
2024
This article is really about picking the cheapest isolation that's still strong enough. Separate clusters give the firmest fault isolation (a failure in one physically can't touch the other) but cost the most; the partitioned limiter trades that hardware boundary for a software one at a single cluster's cost, accepting that the boundary is now code the team has to keep correct. It's a clear worked example of fault isolation as a spectrum with a cost gradient, not an all-or-nothing choice. Read the breakdown →
Slack
Slack Engineering
2023
Siloing converts an AZ from a slice of every request's fan-out into a contained failure domain: a failure inside one AZ presents errors only in that AZ's frontends, instead of surfacing in all of them. The June 30 incident is the pattern's negative proof — pre-cellular, one link's gray failure was everyone's outage. Read the breakdown →
DatadogMonitor the Monitors: How Datadog Watched Its Own Global Outage2023ShopifyTen Bounds on Failure: Resilient Payment Systems at Shopify2022MetaNo Trace in Any Log: How Meta Hunts Silent Data Corruption2022RobloxDark Dashboards: The 73 Hours When Roblox Couldn't See Itself2022GitHubVirtual Before Physical: Partitioning GitHub's Relational Databases2021UberThe Ledger Above the Log: Uber's Kafka Consumer Proxy2021CloudflareNeither Dead nor Alive: Cloudflare's Real-World Byzantine Failure2020Amazon (AWS)One Twenty-Eighth: Shuffle Sharding and the Arithmetic of Blast Radius2019SegmentWhen Queues Stop Working: Segment's Database-as-a-Queue2018ShopifyOne Pod Online: Shopify's Cure for Platform-Wide Failure2018

Often used together

Patterns sharing breakdowns with this one — derived from co-occurrence, threshold ≥2 shared.

cell-architecture ×3generic-mitigation ×3independent-observability ×3

Problems this pattern answers

The walls where its breakdowns live — each opens the cross-company comparison.

Blast radius scales with cluster sizeAMAZON (AWS) · DISCORD · SHOPIFY →Gray failure defeats automatic detectionCLOUDFLARE · META · SLACK →Observer shares fate with observedAIRBNB · DATADOG · ROBLOX →Buffer degrades under backlogSEGMENT · UBER →Ambiguous failure under retrySHOPIFY →Priority-blind load sheddingNETFLIX →Single-cluster scaling ceilingGITHUB →