Problem · seen at 3 companies
Partial completion under crashes
The process died halfway through, and now nothing can tell you which half actually happened.
The wall
A multi-step process interrupted between steps leaves its outcome and its side effects undetermined — some work done, some not, no safe way to resume without either dropping or duplicating it — and, at scale, often no way even to enumerate which steps remain. 3 teams hit this wall; the breakdowns below are the evidence.
Same wall, 3 systems
Airbnb
Airbnb Engineering
A multi-step process crashing between steps leaves its outcome, and its side effects, to chance. Read the breakdown →
Uber
Uber Engineering
Every service rebuilds the same crash-durable machinery — the duplicated scaffolding is what doesn't scale. Read the breakdown →
Netflix
Netflix Technology Blog
Half-done multi-day processes lived inside pub/sub code; nobody could answer what remained. Read the breakdown →