Pattern · seen in 3 breakdowns across 2 companies

Atomic Phases

Atomic phases means splitting a long job into steps that each save their result when they finish, so an interrupted job resumes from the last finished step instead of starting over or repeating work.

resilience

The mechanism

The pattern at its core: a job split into a few steps, each one saving a checkpoint when it finishes, and a retry that reads those checkpoints to pick up where it left off.

LIVE ARTIFACTTHE PATTERN, GENERALOPEN FULL SCREEN ↗
THE IDEAA job that runs as one big step has to start over from the beginning after any crash, redoing work that already happened - and redone work can mean charging a card twice. Splitting the job into steps that each save a checkpoint lets a retry skip what already finished and resume from the exact point that failed.
WHAT TO TRYRun the workflow, then crash it partway. As one big step it restarts from the top and charges the card twice; with atomic phases the retry resumes from the last checkpoint and every step runs exactly once.

Crash a workflow partway, then retry - resume from the last checkpoint, or restart from zero and charge the card twice.

Definition

Break a long job into separate steps, each able to finish and be saved on its own, so that if the job is interrupted it can pick up from the last saved point without redoing or skipping work. Each phase has three parts:

  • a clear input, so it knows what it starts from
  • one real change to the outside world, such as charging a card or saving a record, kept to that one change or a small set of closely related ones
  • a saved marker that records it finished

When the job retries, it reads what was saved, sees which phase finished last, and carries on from the next one. A long operation that reaches across the network becomes a line of small steps you can think about one at a time - each free to succeed, fail, or run again on its own without breaking the rest.

RESUME, DON'T RESTART
A four-phase workflow; a crash during phase three; a retry arrow resuming at phase three, not the start.
A crash partway through does not send the job back to the start. Each finished phase saved a checkpoint, so the retry skips what already committed and resumes at the phase that failed, redoing nothing and skipping nothing.

The rule that makes it work: each phase is all-or-nothing as far as the outside world can see. Either the phase finished and its result is there for the next phase, or it didn't and the next try sees the world exactly as it was before the phase started. Nothing half-done is ever visible.

When it applies

01A long job has to survive a crash partway. A process crash, a dropped network connection, or a lost machine should not force the whole thing to start over.
02A retry must be safe from any point. Whatever step it stopped on, running it again should never do the same work twice.
03The job touches several outside systems. Each call to a database, a payment provider, or another service can fail on its own, so recovery has to happen one step at a time.
04Redoing earlier work is expensive or harmful. When re-running a step costs real money or cannot be undone, the retry needs an exact point to restart from instead of starting over.

Tradeoffs

Every phase has to save its state durably. That saved checkpoint takes time to write, so each boundary adds a little latency.
Choosing where the phases begin and end is hard. Too few and a failure replays a lot of expensive work; too many and the bookkeeping costs more than the work itself.
The data model gets more complicated. Every job now stores its progress, which phase it reached, next to its actual business data.
Making a phase truly all-or-nothing is hard across systems. When one phase touches several services at once, it is difficult to guarantee they all commit together or not at all.

The same move, 3 ways

Every row is a production system that bet on this pattern — the note says how, in that system's own terms.

Airbnb
Airbnb Engineering
2026
Skipper's checkpointed Actions are the pattern applied inside a single workflow method: each action commits its result durably, and replay resumes from the last committed boundary rather than redoing or skipping work — resume-from-last-committed-point, with the workflow method as the sequence and the checkpoint as the phase boundary. Read the breakdown →
Amazon (AWS)
Amazon Builders' Library
2021
Recording the idempotency key and making all the changes that service the request have to commit as one all-or-nothing unit (an ACID transaction). The post is blunt about why: if the key is saved but the work fails, a retry replays a success that never happened; if the work commits but the key isn't saved, a retry does the work a second time. The guarantee that makes retries safe is not the key by itself but the key and the work landing together or not at all. Read the breakdown →
Airbnb
Airbnb Engineering
2019
The same all-or-nothing idea appears here at a smaller scale than in Skipper. Skipper draws its phase boundaries around whole workflow steps (checkpointed actions); Orpheus draws them inside a single request, splitting it into Pre-RPC / RPC / Post-RPC and fencing the phases with two rules: no network calls inside a database transaction, and no database work during the network call. Both give the same property, that you can be interrupted partway and still recover cleanly, just at different sizes: one around a whole workflow step, the other around the parts of one request. Read the breakdown →

Often used together

Patterns sharing breakdowns with this one — derived from co-occurrence, threshold ≥2 shared.

idempotency-keys ×2retryable-error-classification ×2

Problems this pattern answers

The walls where its breakdowns live — each opens the cross-company comparison.

Ambiguous failure under retryAIRBNB · AMAZON (AWS) →Partial completion under crashesAIRBNB →