Observability · Production incident
The trace ends where the queue begins
A request is slow end to end and its trace stops at a message queue. How do you get one trace across the queue?
Wednesday 11:10 · order-api → queue → notify-workerIllustrative scenario, not a real incident
Support: order confirmations arrive up to 6 minutes after checkout
The order API accepts an order, publishes an order.placed message to a queue and answers in under 100 ms. A worker takes the message, renders an invoice and sends the confirmation.
Since Monday, customers have waited minutes for the confirmation. The trace of a late order has three spans and ends at the publish. The worker's log lines carry a time and a level, and no trace id.
What the dashboards and logs show
- Order API p99
- 90 ms, unchanged
- Checkout to confirmation, p99
- 6 minutes (5 seconds last week)
- Trace of a late order
- 3 spans, the last is the publish, 4 ms
- Worker errors
- none
Clarify · optional · scored under "Hypothesis before change"
Ask the interviewer
0 of 3 asked
Each question reveals a fact. What you ask before you act is part of your score.
First move · scored under "Mitigate before root cause"
Confirmations are minutes late right now. What do you do first?
Your rubric score, the model answer, the follow-up questions and the related lessons appear once you have answered the last step.