Skip to content
Interview rounds

Kafka & events · Production incident

The lag climbs while the consumers work

A consumer group's lag keeps growing although its consumers are alive. How do you find out what is stuck, and what do you change?

BackendSenior14 min
Interview-style scenarioLast reviewed: Checked against official documentation

Thursday 10:05 · notification-serviceIllustrative scenario, not a real incident

Lag for group notifier on order-events is 410,000 records and rising

The notification service reads order-events with eight consumers and sends one email per event through a mail provider. It commits its offsets after each batch.

The group's lag was near zero until 09:31 and has grown in a straight line since. All eight pods are running and pass their health checks. One pod was restarted by hand at 09:50 and has read nothing since.

What the dashboards and logs show

Lag, group notifier
near 0 at 09:31, 410,000 at 10:05
Emails sent
about 200 a second, the usual rate
Pods
8 of 8 running and ready, one restarted at 09:50
Topic order-events
every partition has a leader, produce rate as usual

Clarify · optional · scored under "Hypothesis before change"

Ask the interviewer

0 of 3 asked

Each question reveals a fact. What you ask before you act is part of your score.

First move · scored under "Mitigate before root cause"

Emails are going out, but none of that progress is being recorded. What do you do first?

Your rubric score, the model answer, the follow-up questions and the related lessons appear once you have answered the last step.