Application-Layer Sharding

throughput

Definition

Application-layer sharding moves the decision of where data lives — which shard, which cluster, which storage node — into the application code that reads and writes the data, rather than relying on the underlying storage system's internal sharding. The application knows which tenant or entity each piece of data belongs to and uses that knowledge to route reads and writes deterministically. The underlying storage system sees only the operations it's told to perform; it doesn't make routing decisions on the application's behalf.

The motivating insight is control. When sharding is the storage system's responsibility, the application is at the mercy of the storage system's coordination logic, scaling characteristics, and operational boundaries. When sharding is the application's responsibility, the application can use information the storage system doesn't have (tenant priority tiers, traffic patterns, retention policies, locality requirements) to make routing decisions that better match its actual needs. The application also gains the ability to evolve the sharding strategy without depending on the storage system's migration tooling, which is often the largest operational constraint when scale outgrows the initial design.

The canonical implementation is a mapping layer — a lookup from tenant identifier to physical destination — backed by a fast cache and a slower persistent source of truth. Reads consult the cache; writes go to the destination resolved through the cache. The storage system itself is treated as a pool of independent capacity units (clusters, partitions, nodes) that the application orchestrates. This decouples capacity scaling from sharding strategy: adding capacity becomes 'add another unit to the pool'; changing sharding becomes 'reroute the mapping layer.'

The pattern is most visible in systems that have explicitly rejected their storage system's built-in sharding. Discord routes messages to Elasticsearch clusters in application code rather than using Elasticsearch's own shard distribution. Vitess and Citus implement application-layer sharding for relational databases as a deliberate alternative to the database engine's native partitioning. Many large-scale services route requests at the application layer to specific Redis or memcached nodes rather than using cluster-mode sharding. The shared characteristic: the system reaches a scale or operational requirement where the storage system's sharding logic becomes a constraint rather than a feature.

When it applies

  • Multi-tenant systems where tenants vary widely in size and behavior, and where the application can use that knowledge to make better routing decisions than a generic hash function would
  • Systems where the storage layer's native sharding cannot be evolved without significant downtime or data migration, but the data volume or access pattern is likely to require sharding changes over time
  • Architectures where failure isolation between groups of tenants matters more than uniform load distribution — application-layer routing lets you keep specific tenant cohorts together (for failure isolation) or apart (for noisy-neighbor protection)
  • Operational contexts where the team needs to perform cluster-level operations (upgrades, replacements, capacity additions) without coordinating with the storage system's internal rebalancing — application-layer routing lets you drain a cluster by changing the mapping, not by waiting for the storage system to rebalance
  • Systems where the routing logic needs information the storage system doesn't have access to (tenant priority, regulatory locality requirements, customer-specific configuration)

Tradeoffs

  • The application is now responsible for maintaining the mapping layer and ensuring its consistency. Every routing decision must consult the mapping; every change to the mapping must be propagated carefully. This introduces a new class of bugs (mapping inconsistency, stale cache reads) that don't exist when the storage system handles routing internally.
  • The application's complexity grows. Code that would otherwise be 'INSERT into messages' becomes 'look up the destination, route to the correct cluster, handle the case where the lookup fails, retry against the persistent source of truth.' This complexity is non-trivial and must be maintained as the system evolves.
  • Some storage-system features become harder to use. Built-in cross-shard queries, automatic rebalancing, and global indexes often assume the storage system controls sharding. Application-layer sharding either gives these up or rebuilds them at the application layer (which is significant additional engineering work).
  • The mapping layer itself becomes a critical dependency. If the persistent source of truth (often a relational database or a strongly-consistent KV store) is unavailable, the application cannot make routing decisions. The mapping cache provides resilience for hot reads but doesn't help for new tenants or for cache misses — both of which require the source of truth.

Seen in

  • Discord EngineeringApr 24, 2025

    How Discord Indexes Trillions of Messages

    Discord's foundational decision in 2017, preserved through the 2025 redesign. The routing logic lives in application code, not in Elasticsearch's internal sharding. This gives Discord control over which clusters and indices receive each message — a control that proved essential when the time came to evolve the architecture. The pattern's value is most visible in retrospect: the redesign was possible *because* sharding was already an application-level concern that the team could refactor without depending on Elasticsearch's internal coordination.

  • Figma BlogMar 14, 2024

    Sharding Postgres Without Leaving Postgres

    Figma shards its primary OLTP store through DBProxy's query engine — hashed shard keys, colocated table groups preserving single-key joins and transactions — where Discord shards a search-index fleet. Same principle, different substrate: the application layer owns placement and routing so the storage engine underneath can stay stock.

  • Notion BlogOct 6, 2021

    Before the Wraparound: Sharding Postgres at Notion

    Notion names the approach outright: their own partitioning scheme, queries routed from application logic, chosen over Citus and Vitess because packaged clustering logic is opaque and they wanted control of data distribution. Third company in the library to make this exact call — Discord over Elasticsearch, Figma over Postgres with a query-routing service, Notion over Postgres with routing directly in application code.