Skip to content
Interview rounds

Observability · Design under changing requirements

The slow trace that was never kept

When would you use tail-based sampling over head-based, and what does each lose?

Platform-SRESenior10 min
Interview-style scenarioLast reviewed: Checked against official documentation

Wednesday 10:15 · incident reviewIllustrative scenario, not a real incident

Checkout p99 crossed 8 seconds for ten minutes, and no trace of a slow request can be found

Every service traces with the same SDK settings, and one trace in ten ends up in the trace store.

During last night's latency spike about one request in two hundred was slow. The team has plenty of traces from that window, and nearly all of them are fast, healthy requests.

What the dashboards and logs show

Sampling
10% of traces are stored
Traces kept from the spike
thousands, nearly all under 200 ms
Customer's trace ID
not found in the trace store
Tracing budget
fixed: storage cannot grow much

Clarify · optional · scored under "Hypothesis before change"

Ask the interviewer

0 of 3 asked

Each question reveals a fact. What you ask before you act is part of your score.

First move · scored under "Mitigate before root cause"

The spike may return tonight, and the new design will take weeks. What do you do first?

Your rubric score, the model answer, the follow-up questions and the related lessons appear once you have answered the last step.