Skip to content

Consumer apps at scale · 8 of 13

Step 1 · The pattern

Reads spread across partitions

On a quiet day, readers read many different channels, so their queries spread over many partitions of the database. Each partition answers 3 queries at a time. (All numbers are illustrative; the dots stand for many readers.)

  • read
  • answered

In short

Request coalescing: how it works and how it breaks

In a consumer app, the busiest data is also the most shared: a channel with a million members, a live score, a viral post. When something happens there, everyone asks for the same thing within the same second. If each request becomes its own database query, they all land on the same partition, which is flooded while the rest of the cluster idles; queries queue, time out and are retried. Request coalescing puts a data service in front of the database that merges identical requests in flight. The first request for a key queries the database; every request for the same key that arrives while it runs waits for that answer. Routing the same key to the same data-service instance, with consistent hashing, makes most duplicates meet in one place. A messaging company described adding exactly this layer to protect its database from hot partitions.

What this simulation simplifies

  • A request that finds a query in flight waits a fixed half second. Real requests wait only for the rest of the running query.
  • One data service sees every request. A real fleet needs requests for the same key routed to the same instance.
  • Every read for the busy channel asks for exactly the same data. Real readers ask for slightly different pages, so fewer merge.

How this comes up in interviews

"Millions of people in one channel read its latest messages within the same second after a big moment. How do you keep the database alive?"

Then they ask

  1. Why doesn't adding database nodes help?
  2. How do you make sure identical requests meet in the same place?
  3. What happens to all the waiting requests if the shared query fails?

A strong answer covers

  • All requests for one key go to one partition, so it is flooded however big the cluster is.
  • A data service in front of the database merges identical in-flight requests into one query.
  • Route by key with consistent hashing so duplicates reach the same instance.
  • Bound waits with a timeout on the shared query and return its result or error to everyone.
  • Coalescing handles bursts of identical reads; a too-large partition still needs a better key.

Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.

On one page: the track cheat sheet.

Last reviewed: Engineering write-up

Diagrams are our own simulations: the timings, rates and counts in them are illustrative, not measurements. Captions are original wording.