Sharding Postgres Without Leaving Postgres

Figma's database stack has grown almost 100x since 2020 — the kind of problem a business wants, and the kind an infrastructure team has to stay ahead of rather than react to. In 2020 the company ran a single Postgres database on AWS's largest physical instance. By the end of 2022 the databases team had built out a distributed architecture with caching, read replicas, and a dozen vertically partitioned databases — groups of related tables like 'Figma files' or 'Organizations' split into their own partitions. The team's earlier post on that work (https://www.figma.com/blog/how-figma-scaled-to-multiple-databases/) frames vertical partitioning exactly as this article treats it: a relatively easy and very impactful scaling lever that bought significant runway quickly, and a stepping stone on the path to horizontal sharding rather than a destination.

Staying ahead meant measuring where the ceiling actually was. As the fleet grew larger and more heterogeneous, the team combined historical data and load testing to quantify each database's scaling limits — from CPU and IO to table size and rows written — so it could predict the remaining runway per shard and prioritize scaling problems before they ballooned into reliability risks. The data showed that some tables, containing several terabytes and billions of rows, were becoming too large for any single database: reliability began to suffer during Postgres vacuums, the essential background operations that keep Postgres from running out of transaction IDs and breaking down, and the highest-write tables were growing so quickly that they would soon exceed the maximum IO operations per second supported by Amazon RDS. Vertical partitioning could not save them here, because its smallest unit of partitioning is a single table. To keep the databases from toppling, the team needed a bigger lever.

The obvious exits were all one-way doors. The team evaluated CockroachDB, TiDB, Spanner, and Vitess, but switching to any alternative database would have required a complex data migration to ensure consistency and reliability across two different database stores, and would have set aside years of accumulated expertise in running RDS Postgres reliably and performantly. With only months of runway remaining at Figma's very aggressive growth rate, de-risking an entirely new storage layer while completing an end-to-end migration of the most business-critical use cases was judged extremely risky; the team explicitly favored known low-risk solutions over potentially easier options with much higher uncertainty and less control over the outcome. NoSQL was ruled out on different grounds: Figma's product is powered by a fairly complex relational data model — file metadata, organization metadata, comments, file versions — that NoSQL APIs do not offer the versatility for, and adopting one would have meant rewriting almost the entire backend application.

That left building horizontal sharding on top of the vertically partitioned RDS Postgres infrastructure the team already ran — without falling into the trap of reimplementing a generic horizontally sharded relational database in-house, which would put a small team in competition with tools built by large open source communities and dedicated database vendors. Because the solution was tailored to Figma's specific architecture, it could get away with a much smaller feature set: no atomic cross-shard transactions, a colocation strategy that minimized changes at the application layer, support for the subset of Postgres compatible with the majority of the product logic, and easy backwards compatibility between sharded and unsharded Postgres — so that if unknown unknowns surfaced, rolling back remained possible.

The project began with an unusually explicit goals list, and the goals explain most of the design. Minimize developer impact, so product teams keep building features instead of refactoring large parts of the codebase. Scale out transparently, so that after a table's initial preparation, future shard splits happen in the background with no application-level changes. Skip expensive backfills, which at Figma's table sizes and Postgres throughput would have taken months. Make incremental progress, avoid one-way migrations — retaining the ability to roll back even after a physical sharding operation completes — maintain strong data consistency without complex mechanisms like double-writes, and play to the team's existing strengths under tight deadline pressure.

The first decision was the shard key, and Figma's answer diverges from the textbook. There was no single good candidate in the existing data model, and adding a unified composite key would have meant a new column on every table's schema, expensive backfills to populate it, and substantial refactoring of product logic — several goals violated at once. Instead the team selected a handful of keys — UserID, FileID, OrgID — such that almost every table at Figma could shard by one of them. Related tables sharing a key are grouped into colocations, affectionately 'colos', which share the same sharding key and physical sharding layout: within a colo, cross-table joins and full transactions are supported as long as they are restricted to a single value of the sharding key. Most application code already interacted with the database that way, which is precisely what made the abstraction cheap for product developers.

Many of the chosen keys were auto-incrementing or prefixed with Snowflake timestamps, which would have concentrated the majority of data on hot shards; rather than run an expensive, time-consuming migration to more randomized IDs, the team routes on a hash of the sharding key. The accepted cost is that range scans on shard keys become less efficient — sequential keys hash to different shards — a query pattern uncommon enough in Figma's codebase to trade away.

The serving stack was re-architected around DBProxy, a new golang service sitting between the application layer and PgBouncer, carrying load-shedding, request hedging, improved observability, transaction support, database topology management, and a lightweight query engine — the heart of the system. A query parser turns application SQL into an abstract syntax tree; a logical planner extracts the query type and the logical shard IDs; a physical planner maps logical shard IDs to physical databases and rewrites the query to execute on the appropriate shard. A query filtered to a single shard key routes to one database, pushing execution down into Postgres. A query without a sharding key becomes a scatter-gather — fanned out to every shard, results aggregated back — which gets very complex for aggregations, joins, and nested SQL, and is expensive by construction: because it touches every database, each scatter-gather contributes the same load as if the fleet were unsharded. Keeping DBProxy from growing into a reimplementation of Postgres's own query engine meant choosing the supported subset deliberately. A shadow planning framework let engineers define potential sharding schemes for their tables and run the logical planning phase against live production traffic, logging queries and query plans to a Snowflake database for offline analysis. From that data the team picked a query language that supported the most common 90% of queries while avoiding worst-case complexity in the engine — all range scans and point queries allowed; joins only between two tables in the same colo, and only on the sharding key. The same shadow machinery doubled as an application-readiness map, showing product teams exactly which logic needed refactoring before their tables could shard.

The signature move is the separation of logical sharding from physical sharding. Once a table is logically sharded, all reads and writes behave as if the table were horizontally sharded — from a reliability, latency, and consistency perspective the system appears sharded — while the data still physically sits on a single database host. Figma represents the logical shards with Postgres views over the unsharded table, one per shard, each selecting the rows whose hashed shard key falls within that shard's range, and each accessed through its own sharded connection pooler still pointing at the unsharded instance. Rollout proceeded gradually behind feature flags in the query engine, and rolling back when bugs surfaced was a simple configuration change, rerouting traffic back to the main table within seconds. The views themselves were treated as a risk to validate rather than an assumption: views add performance overhead and can fundamentally change how the Postgres query planner optimizes, so the team collected a corpus of sanitized production queries and load-tested with and without views — confirming minimal overhead in most cases and less than 10% in the worst — then built a shadow reads framework that sent all live read traffic through the views, comparing performance and correctness against non-view queries. By the time the team ran its first reshard, the sharded topology had already been running in production.

Query routing depends on topology — which table maps to which shard key, which logical shard ID lives on which logical and physical database. Vertical partitioning had gotten by with a simple hard-coded configuration file mapping tables to partitions, but shard splits change the topology dynamically, and routing a request to the wrong database is precisely the failure that cannot be tolerated. The team built a topology system that encapsulates the horizontal sharding metadata and delivers real-time updates in under a second, with every topology change backwards compatible so that changes never sit on the critical path for the site. The separation of logical from physical topology pays operational dividends too: non-production environments keep production's logical topology while serving from many fewer physical databases, and the topology library enforces invariants — every shard ID maps to exactly one physical database — that keep the system correct as it grows.

The physical sharding operation reused much of the vertical partitioning playbook with two differences: data now moves from one database to N rather than one to one, and the failover had to be made resilient to new failure modes in which the operation succeeds on only a subset of databases. Notably, the team avoided implementing filtered logical replication — copying only each shard's subset — and instead copies the entire dataset to each new shard, then restricts reads and writes to the subset belonging to it. The first horizontally sharded table shipped in September 2023 — roughly 9 months end to end — failing over with only 10 seconds of partial availability on database primaries, no availability impact on replicas, and no regressions in latency or availability afterward. Since then the team has been sharding relatively simple, high-write-rate tables, with increasingly complex databases — dozens of tables, thousands of code call-sites — ahead, along with a named backlog: horizontally sharded schema updates, globally unique ID generation for sharded primary keys, atomic cross-shard transactions for business-critical cases, distributed globally unique indexes, an ORM model seamlessly compatible with sharding, and fully automated one-click reshard operations. The post closes with a notable piece of honesty: once sufficient runway is banked, the team will reassess the in-house RDS approach against the maturing landscape of open source and managed NewSQL solutions.