Skip to content
Interview rounds

Databases & caching · Concept deep-dive

A cache on four machines loses one

How do you design a cache that runs on many machines, so that losing or adding a machine does not flood the database?

BackendStaff14 min
Interview-style scenarioLast reviewed: Checked against official documentation

Concept deep-dive · a cache spread over several machinesIllustrative scenario, not a real incident

Review: last month one of four cache nodes failed, and the database was saturated for 20 minutes

A catalogue service keeps product data on four standalone Redis nodes. Its client library picks the node for a key with hash(key) mod 4, and on a miss the service reads PostgreSQL and stores the result.

Last month one node failed. The clients dropped it from their list and carried on with three. A fifth node is planned for next week, because memory is filling up.

What the dashboards and logs show

Reads
200,000 a second, 96% answered from the cache
Database
serves 8,000 reads a second normally, 80,000 at most
Cache nodes
4 × 32 GB, each about 70% full
Last month
one node lost, database saturated for 20 minutes

5 questions on one topic, each at least as hard as the last. Answer each before the next appears. The last one plays out.

Question 1 of 5 · Mid · scored under "Hypothesis before change"

The failed node held a quarter of the keys. Why was the database saturated, and not a little busier?

Your rubric score, the model answer, the follow-up questions and the related lessons appear once you have answered the last step.