Skip to content
Interview rounds

Observability · On-call judgement

Pages for the CPU, silence for the customers

How do you design alerting that pages on customer impact, not CPU?

BackendSenior12 min
Interview-style scenarioLast reviewed: Checked against official documentation

Monday 10:00 · on-call reviewIllustrative scenario, not a real incident

On-call was paged at 01:24 for nothing, and nobody was paged when 5% of orders began to fail

The order API has an SLO: 99.9% of requests succeed over 30 days. Its only paging alert is on CPU, written years ago.

Last night the alert paged for a report job that harmed nobody. An hour later a deploy made one request in twenty fail. The CPU stayed flat, the pager stayed silent, and support found out from customers.

What the dashboards and logs show

SLO
99.9% of requests succeed, over 30 days
Paging alert
CPU above 85% for 5 minutes
Pages last night
one, at 01:24

Clarify · optional · scored under "Hypothesis before change"

Ask the interviewer

0 of 3 asked

Each question reveals a fact. What you ask before you act is part of your score.

First move · scored under "Mitigate before root cause"

The new alerting will take a week to design and review. What do you do first, for tonight?

Your rubric score, the model answer, the follow-up questions and the related lessons appear once you have answered the last step.