Incident · Easy · Databases
One sale, every store
- SEV1
- SALE DAY
- ONE SHARD
90s
The situation
You run a platform that hosts online stores, many to a database shard. A celebrity make-up brand announces a sale and fans rush checkout the moment it opens. Her store goes down, and so does every other store on its shard. You have just added heavier page caching; it helped, though not enough. Her next sale is in a week.
The evidence
- attempted
- completed
Logs and signalsReconstructed
sale announced, new products go livepage cache: hit rate highcheckout: one record per session, rewritten at every stepshard: write load past its limitother stores on the shard: checkout failing
Your call
What do you ship before next week's sale?
Charts and logs are reconstructions. Aim for 90 seconds; going over costs nothing.