Step 1 · The pattern
One quota, many clients
Every client calls the service through one limiter. It admits 8 requests per window, then the window resets and the quota is whole again. Normal traffic stays well under it.
- request
- served
In short
Client backoff after a 429: how it works and how it breaks
A 429 with Retry-After tells a client when it may try again. That works for one client. Now picture hundreds refused in the same moment. They are all given the same answer, so they all come back together, right as the limit reopens. The quota is used up at once, the service is handed a spike it cannot work through, and everyone else is refused again and lines up for the next reset. Between spikes the service sits idle. The server's hint has turned a crowd into a marching column. The clients have to break step. Treat Retry-After as the earliest moment you may return, then add a random extra wait on top. If you are refused again, widen that random range, up to a cap. Stop after a few attempts and report the failure. Arrivals spread out, and far fewer requests are wasted.
How this comes up in interviews
"Hundreds of clients receive a 429 with the same Retry-After at the same moment. What happens when that time is up, and how should clients behave?"
Then they ask
- The clients already use exponential backoff with jitter on later failures. Why is that not enough?
- How wide should the random range be, and when should a client give up?
- You do not control every client. What can the server do to spread the crowd out?
A strong answer covers
- Clients refused together are given the same wait, so they return together as the limit reopens and the quota goes at once.
- The rest are refused again and line up for the next reset in step, while the service sits idle between spikes.
- Treat Retry-After as the earliest moment to return and add a random extra wait on top, starting with the first wait.
- After each further 429, widen the random range up to a cap. Stop after a few attempts and report the failure.
- A retry budget bounds the extra load from a whole fleet, which a cap on attempts does not.
Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.
On one page: the track cheat sheet.
Sources: Exponential Backoff And Jitter (AWS Architecture Blog, 2015) ↗ · RFC 9110: HTTP Semantics (Retry-After) ↗ · Report a problem
Diagrams are our own simulations: the timings, rates and counts in them are illustrative, not measurements. Captions are original wording.