Path 4 of 10
Kafka Consumers
The poll loop, consumer groups, offsets, rebalancing and lag.
0 / 5 lessons completed ยท about 124 min in total
After this path you can
Outcomes
- Explain how partitions are shared in a group
- Pick a commit strategy and know what it risks
- Diagnose a rebalance or a growing lag
Lessons
- 1The consumer and its poll loopA Kafka consumer pulls records by calling poll in a loop. See what one poll does, why the time between polls matters, and how Spring wraps the loop.Foundation22 min
- 2Consumer groups and partition assignmentHow a group id turns many consumers into one subscriber, how partitions are shared out by each assignor, and why parallelism stops at the partition count.Intermediate24 min
- 3Offset management and commitsWhat a committed offset is, where it is stored, when automatic and manual commits happen, what Spring's ack modes do, and how to reset a group's position.Intermediate26 min
- 4Consumer rebalancingWhat starts a rebalance, what each protocol and assignor revokes, what the three timeouts detect, and what happens to work in progress when a partition moves.Advanced28 min
- 5Consumer lagWhat lag measures, how to read it per partition, how to tell a capacity problem from a stuck or hot partition, and what to change in each case.Intermediate24 min
Related labs
Simulations, not a real cluster.
- Lab 3Consumer group assignmentFoundation20 min
- Lab 4Adding and removing consumersIntermediate25 min
- Lab 5Duplicate message processingIntermediate25 min
- Lab 6Consumer crash before offset commitIntermediate25 min
- Lab 7Poison message and dead-letter queueIntermediate25 min
- Lab 8Retry and backoff strategyIntermediate30 min
- Lab 10Consumer lag during high loadIntermediate30 min
- Lab 11Hot partitionAdvanced35 min
- Lab 13Database unavailable while consuming Kafka messagesAdvanced35 min