Skip to content
Interview rounds

Kubernetes & deploys · Production incident

The deploy succeeds, and customers see errors

Every rolling deploy reports success, and every one produces a burst of 5xx. What do you change so that a rollout drops no requests?

BackendSenior12 min
Interview-style scenarioLast reviewed: Checked against official documentation

Thursday 15:10 · checkout-api, 12 replicasIllustrative scenario, not a real incident

checkout-api: 5xx rate 2.1% for four minutes, third time this week

The checkout API runs as a Deployment of 12 pods behind an ingress gateway. Each of this week's three releases finished with kubectl rollout status reporting success.

During each rollout the gateway logged about 300 responses with status 502 or 503. Between rollouts the count is zero, and the three releases changed unrelated code.

What the dashboards and logs show

5xx at the gateway
about 300 per rollout, none between rollouts
Rollout
about four minutes, reported as successful each time
Pod restarts
0 on every pod, old and new
Application logs
no exceptions during the rollouts

Clarify · optional · scored under "Hypothesis before change"

Ask the interviewer

0 of 3 asked

Each question reveals a fact. What you ask before you act is part of your score.

First move · scored under "Mitigate before root cause"

The fix needs a release of its own. What do you do about the errors until then?

Your rubric score, the model answer, the follow-up questions and the related lessons appear once you have answered the last step.