Idempotency Keys

consistency

Definition

A client-supplied stable identifier that the server uses to deduplicate retries of the same logical operation. The first request creates a record of the outcome under the key; subsequent requests with the same key return that recorded outcome rather than re-executing the work. Same key in, same outcome out, regardless of how many times the request is sent.

Idempotency keys are the user-facing contract that makes safe retry possible. They sit above the wire protocol -- the network can drop, the server can crash, the client can time out and retry, and as long as both sides agree on the key, the outcome is deterministic. The implementation underneath (atomic phase commits, dedup storage, etc.) is a separate concern; the pattern is the agreement on the identifier.

When it applies

  • Any non-idempotent operation that may be retried -- payment charges, resource creation, irreversible state changes.
  • APIs where the caller is responsible for retry policy and the server cannot trust the network.
  • Workflows that span multiple systems where partial completion on retry would diverge state.

Tradeoffs

  • Requires durable storage of (key -> outcome) records with TTLs that outlive realistic retry windows.
  • Adds per-request bookkeeping cost: every request reads and (on first execution) writes a key record.
  • Key reuse is a real hazard -- a client that reuses an old key for a new logical operation gets the old outcome, silently. Key generation discipline (UUIDs, monotonic IDs, scope-prefixed) becomes part of the contract.

Seen in

  • Shopify EngineeringJul 28, 2022

    Ten Bounds on Failure: Resilient Payment Systems at Shopify

    Shopify's instantiation is provider-side: the centralized payment service tracks each attempt's completed steps under the key and runs recovery steps to rebuild state before continuing, so a retry never re-reaches the financial partner. The ULID key format makes the key itself index-friendly — a 50% INSERT duration reduction in one high-throughput system.

  • Amazon Builders' LibraryNov 25, 2019

    The Selfish Retry: Timeouts, Backoff, and Jitter at Amazon

    EC2 RunInstances' client token is the same mechanism Stripe exposes as the Idempotency-Key header: the client names the operation, retries repeat the name, and the server deduplicates — converting an unsafe-to-retry, side-effecting call into a safe one. Second company, same contract: retry safety is purchased at API-design time.

  • Stripe EngineeringFeb 22, 2017

    Designing robust and predictable APIs with idempotency

    The source of the pattern's canonical form: a client-generated unique ID sent with each mutating request (Stripe's Idempotency-Key header on all POST endpoints), letting the server resolve all three network-failure cases — process fresh, pick up and carry through, or replay the cached result. Same key in, same outcome out.