Pattern · seen in 1 breakdown across 1 company

Hot-Data-First Migration

Definition

Order a live data migration by access recency and frequency rather than by key range or table order: replicate the hot set — recently created, updated, and read entities — ahead of the archive, so the strained source system sheds its serving load as early as possible and the destination becomes useful long before the migration completes. The mechanism is typically a live change/access feed capturing the hot set in real time, plus a background scan walking the remainder most-recent-first under backpressure, advancing only when the live feed's queue runs empty so the archive never competes with the present.

The pattern's defining honesty is that capability arrives per access pattern, not per percentage: point lookups on hot entities serve from the destination almost immediately, while any query that must see the full population — listings, aggregates, anything not keyed by ID — structurally waits for the scan to finish. Migration progress dashboards measured in rows misstate this; the real milestones are 'which query shapes are now servable.' Boundary against uniform backfills (key-ordered or table-ordered): those optimize for scan throughput and completeness accounting, and relieve the source only at the end; hot-first optimizes for load relief and early utility, and pays with a longer, throttled tail.

When it applies

01The source system is actively failing or near its ceiling — shedding its serving load early is worth more than completing the copy early
02Access is heavily recency-skewed, so a small hot set carries most traffic and can be captured live while the long tail scans in the background
03The destination can usefully serve partial data (point reads by ID) while population-complete query shapes wait

Tradeoffs

Capability arrives per query shape: full-population queries (listings, aggregates) are blocked until the scan completes, however high the row-percentage reads
Two migration mechanisms must run correctly together — the live feed for the hot set and the backpressured scan for the tail — with the scan throttled to never starve the present
Recency ordering assumes the access distribution; a workload with a heavy cold-read tail (archives, compliance reads) undercuts the early-relief payoff

The same move, 1 ways

Every row is a production system that bet on this pattern — the note says how, in that system's own terms.

Canva
Canva Engineering Blog
2022
The post's first lesson, 'be lazy' (know your access patterns and migrate the busy data first), made mechanical. Recently created, updated, and read media replicated ahead of the archive, with a newest-first scan feeding the backlog under backpressure, so load came off the strained MySQL cluster as early as the hot set allowed. The honest cost is that capability arrives per access pattern: queries without an ID had to wait for the scan to finish. Read the breakdown →

Problems this pattern answers

The walls where its breakdowns live — each opens the cross-company comparison.