Path 8 of 10
Performance and Scalability
Throughput, latency, partition counts, hot partitions, backpressure and scaling the cluster.
0 / 5 lessons completed ยท about 132 min in total
After this path you can
Outcomes
- Find the real bottleneck before tuning
- Size partitions and consumers for a target rate
- Tune producers and consumers for a high-volume stream
Lessons
- 1Throughput and latencyWhat sets each of them in a Kafka pipeline, why raising one usually costs the other, where the milliseconds go end to end, and how to measure with percentiles.Intermediate24 min
- 2Partition sizing and consumer parallelismHow to choose a partition count from a target rate and a processing time, what too many partitions cost, why the count is hard to change, and how to add parallelism safely.Intermediate26 min
- 3Tuning producers and consumer fetchingA procedure for tuning batching, compression, buffers, fetch sizes and large messages: calculate first, change one setting at a time, and measure.Advanced26 min
- 4Hot partitions and backpressureWhy one partition falls behind while the rest are idle, how to find and fix a skewed key, and how a pull-based consumer protects a slow database instead of flooding it.Advanced28 min
- 5Cluster resources and scalingHow to size disk, network, memory and CPU for a Kafka cluster from throughput and retention, and what adding brokers, consumers or tiered storage really changes.Advanced28 min
Related labs
Simulations, not a real cluster.
- Lab 1Producer publishing messagesFoundation20 min
- Lab 2Partition key and orderingFoundation20 min
- Lab 3Consumer group assignmentFoundation20 min
- Lab 4Adding and removing consumersIntermediate25 min
- Lab 9Broker failure and leader electionAdvanced30 min
- Lab 10Consumer lag during high loadIntermediate30 min
- Lab 11Hot partitionAdvanced35 min
- Lab 13Database unavailable while consuming Kafka messagesAdvanced35 min