Simulated Policy Selection

When a system must choose among competing policies — what to cache, where to place data, how long to retain it, when to admit or evict — and the right answer varies by workload, the common failure is picking one policy in a design review and letting it govern everything. The pattern replaces the argument with an experiment that never ends: run cheap online simulations of every candidate policy against the live, observed workload, segment by segment, and let each segment be governed by whichever policy currently wins its simulation. Google's L4 is the reference form: applications pass features with each new file, L4 segregates files into categories, observes each category's real I/O over time, and continuously simulates the candidate placements — SSD for one hour, for two hours, not at all — with the winner governing that category's new files and the losers running on as counterfactuals, ready to take over when the workload drifts.

The pattern pays a second dividend that is easy to miss: the same simulations answer sizing questions. Because the machinery can already evaluate 'what would this policy have done,' it can equally evaluate 'what would I have done with more or less capacity' — turning placement simulation into a capacity oracle that predicts, per unit of additional fast storage, how much load could be offloaded from the slow tier. In L4's case those signals drive SSD hardware purchases and tell planners where shifting capacity between applications buys the most efficiency: the component that places files also sizes the fleet.

Boundary against neighbors: this is not A/B testing — no user-visible variation is served; the losing policies run only in simulation against the same observed trace. And it is not offline modeling — the simulations run continuously against live traffic, so a policy that stops winning stops governing. The pattern applies when a policy's counterfactual cost can be computed cheaply from an observed trace; if evaluating a candidate requires actually serving traffic under it, you are in experiment-design territory, not simulation territory.