Pattern · seen in 3 breakdowns across 3 companies

Durable Workflows

A durable workflow lets you write a multi-step process as plain top-to-bottom code, while the system saves each step as it finishes, so a crash resumes from the last completed step, not the start.

resilience

The mechanism

The pattern at its core: a multi-step workflow that crashes partway, and a runtime that replays the code from the top but returns the saved result for every step that already finished, so no step's effect runs twice.

LIVE ARTIFACTTHE PATTERN, GENERALOPEN FULL SCREEN ↗
THE IDEAYou write the workflow as plain top-to-bottom code. When it crashes, the runtime runs that same code again from the beginning - but every step that already finished returns its saved result instead of re-running, so its effect fires only once. Execution reaches the point where it crashed and carries on from there.
WHAT TO TRYRun the workflow, crash it partway, then replay. Watch the runtime re-enter from the top: the finished steps return their saved results instantly, the crashed step runs for real, and every effect still fires exactly once.

Crash a workflow partway, then replay it - each finished step returns its saved result, so every effect fires once.

Definition

A durable workflow is a multi-step process - a payment, an onboarding, an approval - that remembers how far it has got, even through crashes, restarts, and long waits. Your code never has to handle the saving, retrying, or recovering itself. The person writing the workflow writes plain, top-to-bottom code, and the engine underneath promises that if the process is interrupted anywhere, it picks back up from the last good point when it restarts.

The key idea is to split the workflow into two kinds of code:

  • the flow - the ifs, loops, waits, and order of steps; it must be deterministic (same inputs, same result every time) so it can be replayed exactly
  • the effects - the parts that change the outside world, like calling an API or writing to a database; each one is wrapped in a named step whose result is saved (checkpointed) when it succeeds

After a crash, the workflow runs again from the top. But wherever it reaches a step that already finished, it uses the saved result instead of running that step again - the same checkpoint-and-resume idea, now applied to a whole multi-step process at once. So the author just sees plain, top-to-bottom code, while the engine underneath does the work of surviving crashes.

FLOW VS EFFECTS
Two lanes: a deterministic flow on top, checkpointed effects below, with replay reusing the saved results.
A durable workflow has two layers: a deterministic flow that replays exactly on each restart, and effects that run once and save a checkpoint. On replay the flow reuses those saved results instead of firing the effects again.

When it applies

01The process runs for minutes to hours, and the middle matters. Things like payments, claims, or multi-stage approvals, where losing the half-finished state would be a real problem.
02The process has to wait on the outside world. It waits for a human to approve, a callback to arrive, or a scheduled time, and holding a thread open or polling the whole time would be wasteful.
03A crash must not leave a mess behind. The process has to survive failures without half-finished state, duplicate effects, or work that got started and then forgotten.
04You are drowning in cleanup and reconciliation scripts. When failures keep spawning one-off fix-up jobs, a durable workflow folds that scattered recovery logic into one place.

Tradeoffs

The flow code has to be deterministic. Anything that could differ between runs - a random number, the current time, a branch that depends on either - has to be wrapped in a saved step, or the replay won't match.
Effects can run more than once. If a step succeeds but its checkpoint fails to save, the replay will run it again, so each step has to be safe to repeat.
Changing a running workflow is hard. If you edit the logic while old runs are still in flight, replaying them against the new code can go wrong, so you have to version the workflow.
Debugging means thinking in replays. What actually happens may not match a plain top-to-bottom read of the code, because the runtime is replaying and short-circuiting underneath.

The same move, 3 ways

Every row is a production system that bet on this pattern — the note says how, in that system's own terms.

Airbnb
Airbnb Engineering
2026
Skipper's Workflow + Action model is the canonical embedded implementation: deterministic orchestration logic, side effects checkpointed behind a one-annotation boundary, and crash recovery by replay from the last good state. The library form (versus a central cluster) changes where durability executes, not the pattern itself — which is exactly why the same pattern recurs across centralized engines too. Read the breakdown →
Uber
Uber Engineering
2023
Cadence delivers the same guarantee Skipper delivers — multi-step programs that survive crashes and resume with committed work intact — through the same fundamental mechanism family (persisted state, replayable execution, versioning for in-flight changes). The recurrence across opposite topologies is the strongest evidence the pattern is real: the guarantee is the invariant; where the engine lives is the variable. Read the breakdown →
Netflix
Netflix Technology Blog
2016
Third company, a decade early: declarative flow definition, durable per-task state, timeouts and retries as configuration, pause/resume/restart as operations. The Conductor-specific contribution is searchability — execution state indexed in Elasticsearch, making the workflow store a queryable system of record rather than just a recovery log. Read the breakdown →

Often used together

Patterns sharing breakdowns with this one — derived from co-occurrence, threshold ≥2 shared.

embedded-vs-centralized-orchestration ×3

Problems this pattern answers

The walls where its breakdowns live — each opens the cross-company comparison.

Partial completion under crashesAIRBNB · NETFLIX · UBER →