Java/Spring/JVM · Production incident
Every test passed, and the app crashed
A change to a REST response breaks some clients after the deploy. How do you change an API without breaking the clients already out there?
Wednesday 16:05 · orders-apiIllustrative scenario, not a real incident
Mobile: the crash rate on the order screen went from 0.2% to 31% at 15:40
Version 2.14 of the orders API went out at 15:40. Its build was green, and the API has returned 200 to every request since, at its usual speed.
The website shows orders as it always has, and so does the newest mobile app. The crashes come from phones that open an order.
What the dashboards and logs show
- Order screen crash rate
- 31%, from 0.2% before 15:40
- API errors
- 0.0% 5xx, p99 unchanged
- Clients
- website fine, app 4.2 fine, app 4.1 and older crashing
- Daily active devices
- about 200,000
Clarify · optional · scored under "Hypothesis before change"
Ask the interviewer
0 of 3 asked
Each question reveals a fact. What you ask before you act is part of your score.
First move · scored under "Mitigate before root cause"
A third of order screen views are crashing right now. What do you do first?
Your rubric score, the model answer, the follow-up questions and the related lessons appear once you have answered the last step.