Skip to content

Path 3 of 10

Kafka Producers

How records are serialised, partitioned, batched, acknowledged and retried, and what each setting costs.

0 / 5 lessons completed ยท about 126 min in total

After this path you can

Outcomes

  • Choose keys that give the ordering you need
  • Configure acks, retries and idempotence deliberately
  • Tune batching and compression for a workload

Lessons

  1. 1Inside the producer: from send() to the partition logWhat happens between KafkaTemplate.send() and a record landing in a partition: serializers, partition choice, the buffer of batches, the sender thread and the result.Intermediate22 min
  2. 2Keys and partitioning: who decides where a record goesHow the producer picks a partition, why the key is what gives you ordering, how to choose a key, and what skew and adding partitions do to both.Intermediate24 min
  3. 3Acks, retries and the idempotent producerWhat an acknowledgement proves, how retries are bounded by the delivery timeout, how they duplicate and reorder records, and exactly what producer idempotence fixes.Intermediate28 min
  4. 4Batching, linger.ms, compression and buffer.memoryHow the producer groups records into per-partition batches, what linger.ms and batch.size trade, what happens when the buffer fills, and where compression is applied and undone.Intermediate22 min
  5. 5A working Spring Boot producer, and what its settings changeBuild a small order service that publishes keyed events with KafkaTemplate and handles the send result, then change its settings and see what moves: throughput or reliability.Intermediate30 min

Related labs

Simulations, not a real cluster.

  1. Lab 1Producer publishing messagesFoundation20 min
  2. Lab 2Partition key and orderingFoundation20 min
  3. Lab 5Duplicate message processingIntermediate25 min
  4. Lab 9Broker failure and leader electionAdvanced30 min
  5. Lab 10Consumer lag during high loadIntermediate30 min
  6. Lab 11Hot partitionAdvanced35 min
  7. Lab 12Transactional outboxAdvanced30 min