Skip to content

Path 4 of 10

Kafka Consumers

The poll loop, consumer groups, offsets, rebalancing and lag.

0 / 5 lessons completed ยท about 124 min in total

After this path you can

Outcomes

  • Explain how partitions are shared in a group
  • Pick a commit strategy and know what it risks
  • Diagnose a rebalance or a growing lag

Lessons

  1. 1The consumer and its poll loopA Kafka consumer pulls records by calling poll in a loop. See what one poll does, why the time between polls matters, and how Spring wraps the loop.Foundation22 min
  2. 2Consumer groups and partition assignmentHow a group id turns many consumers into one subscriber, how partitions are shared out by each assignor, and why parallelism stops at the partition count.Intermediate24 min
  3. 3Offset management and commitsWhat a committed offset is, where it is stored, when automatic and manual commits happen, what Spring's ack modes do, and how to reset a group's position.Intermediate26 min
  4. 4Consumer rebalancingWhat starts a rebalance, what each protocol and assignor revokes, what the three timeouts detect, and what happens to work in progress when a partition moves.Advanced28 min
  5. 5Consumer lagWhat lag measures, how to read it per partition, how to tell a capacity problem from a stuck or hot partition, and what to change in each case.Intermediate24 min

Related labs

Simulations, not a real cluster.

  1. Lab 3Consumer group assignmentFoundation20 min
  2. Lab 4Adding and removing consumersIntermediate25 min
  3. Lab 5Duplicate message processingIntermediate25 min
  4. Lab 6Consumer crash before offset commitIntermediate25 min
  5. Lab 7Poison message and dead-letter queueIntermediate25 min
  6. Lab 8Retry and backoff strategyIntermediate30 min
  7. Lab 10Consumer lag during high loadIntermediate30 min
  8. Lab 11Hot partitionAdvanced35 min
  9. Lab 13Database unavailable while consuming Kafka messagesAdvanced35 min