Observability · Production incident
"The service is slow"
A service is reported as slow, with no alert and no obvious cause. How do you troubleshoot it?
Thursday 10:50 · orders-apiIllustrative scenario, not a real incident
Support: customers say placing an order is slow. No alert has fired
The orders API runs on twelve hosts behind a load balancer. Support has had a dozen complaints in ten minutes that placing an order takes several seconds, for some customers and not others.
The service's dashboard shows a p99 of 2.4 seconds, against 180 ms an hour ago. The median has not moved, and the error rate is flat.
What the dashboards and logs show
- p99 latency
- 2.4 s (was 180 ms)
- Median latency
- 60 ms, unchanged
- Error rate
- 0.1%, unchanged
- Request rate
- normal for a Thursday morning
Clarify · optional · scored under "Hypothesis before change"
Ask the interviewer
0 of 3 asked
Each question reveals a fact. What you ask before you act is part of your score.
First move · scored under "Mitigate before root cause"
Customers are waiting several seconds right now. What do you do first?
Your rubric score, the model answer, the follow-up questions and the related lessons appear once you have answered the last step.