Step 1 · The pattern
People open the app at their own pace
On a normal day people open the app whenever they like: here 2 requests a second, spread out. The home feed API serves 6 at a time, so it is never close to full. (All numbers are illustrative; one dot is a large group of phones.)
- request
- home screen loaded
In short
Push notification stampede: how it works and how it breaks
A push notification reaches millions of phones within seconds, and many apps fetch fresh data the moment a push arrives so the screen is ready when the person taps it. Those fetches arrive together, far faster than anything people opening the app ever produce. The API fills up, refuses the rest, and the app retries a second later, so the retries keep it saturated long after the push has gone. The traffic is the app's own doing, so the fix starts there. Make the app wait a random moment before a push-triggered prefetch, so the requests spread out. Teach the edge to tell a prefetch from a person: while the API is busy, answer push prefetches with "nothing new", because the app fetches again when it is opened. And send large campaigns in waves.
What this simulation simplifies
- One dot stands for a large group of phones, and the whole push lands within one second. Real pushes reach phones over several seconds to minutes.
- The API refuses a request as soon as twelve are waiting or in progress. A real service slows down first, then times out.
- Every refused app retries after about a second. Real apps differ, and some give up until the person opens the app.
How this comes up in interviews
"Your API falls over for a few minutes every time marketing sends a push to all users, though daily traffic is flat. Walk me through the cause and the fix."
Then they ask
- The product team insists the screen must be ready when the person taps. How do you keep that and still protect the API?
- How would the edge know which requests it may shed?
- How would you load test a push campaign before it goes out?
A strong answer covers
- A push lands on every phone within seconds, and prefetch on arrival turns it into one synchronised wall of requests.
- Refused requests retry and keep the API full, so the errors outlast the push.
- Add a random delay before push-triggered fetches, and send large campaigns in waves.
- Tag requests with their trigger and shed push prefetches at the edge while the API is busy; the app fetches again on open.
- Make unchanged data cheap with conditional requests, and load test scheduled peaks like pushes.
Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.
On one page: the track cheat sheet.
Sources: Monzo: Protecting our platform from spikes in usage by reducing load from the Monzo app (2023) ↗ · Amazon Builders' Library: Timeouts, retries and backoff with jitter ↗ · InfoQ: Monzo sheds app traffic to protect its platform from spikes (2023) ↗ · Report a problem
Diagrams are our own simulations: the timings, rates and counts in them are illustrative, not measurements. Captions are original wording.