The catalog

Browse by problem class.

Grouped by the crux — the bottleneck that made each system hard. Filter by company to see who else hit the same wall.

Showing 14 of 14 breakdowns across 10 problem classes.

Ambiguous failure under retry

2 systems

A request that times out or fails mid-flight leaves the caller unable to tell whether it already took effect, so a retry risks repeating an operation that in fact succeeded.

SEEN AT Shopify · Stripe

Partial completion under crashes

2 systems

A multi-step process interrupted between steps leaves its outcome and its side effects undetermined — some work done, some not, and no safe way to resume without either dropping or duplicating it.

SEEN AT Airbnb · Uber

Priority-blind load shedding

2 systems

Under overload the system drops work without regard to its importance, so shedding meant to protect the service also sacrifices its most critical traffic.

SEEN AT Netflix · Uber

Single-table scaling ceiling

2 systems

A single table's size or write volume outgrows what any one database instance can serve, and because a table is the smallest unit vertical partitioning can move, only splitting the table itself across shards relieves it.

SEEN AT Figma · Notion

Blast radius scales with cluster size

1 system

In one large shared cluster the cost of a single failure grows with node count — a lone slow or dead node degrades a disproportionate share of operations, and coordination overhead worsens with size — so the fix is smaller isolated cells.

SEEN AT Discord

Buffer degrades under backlog

1 system

A component whose job is to absorb backlog is served by a substrate that slows down as the backlog grows, so the very condition it exists to handle erodes its ability to handle it.

SEEN AT Meta

Gray failure defeats automatic detection

1 system

When a failure is partial and inconsistent, different components hold contradictory views of what is healthy, so automated detection cannot converge on a fault the system can't agree exists — and mitigation must fall to a human acting on ambiguous evidence.

SEEN AT Slack

Observer shares fate with observed

1 system

A system depends on the very infrastructure it exists to watch or protect — most commonly monitoring running on the servers it monitors — so a failure in that foundation disables the safeguard exactly when it is needed.

SEEN AT Airbnb

Retry-amplified overload

1 system

When failures are caused by overload, clients retrying to improve their own odds spend the struggling dependency's scarce capacity, multiplying load at the worst moment — and naive backoff does not fix it, because clients that failed together retry in sync.

SEEN AT Amazon (AWS)

Single-cluster scaling ceiling

1 system

A single database cluster becomes both a capacity ceiling and a shared-fate domain: it can only be scaled by ever-larger machines, and any incident on it degrades everything built on top — so relief means moving whole tables or domains onto separate clusters.

SEEN AT GitHub