Path 5 of 10
Message Delivery Semantics
At-most-once, at-least-once and exactly-once, and where Kafka's guarantees stop and your database's begin.
0 / 5 lessons completed ยท about 124 min in total
After this path you can
Outcomes
- State precisely what each guarantee covers
- Separate producer idempotence from consumer idempotency
- Design effectively-once processing for payments and orders
Lessons
- 1At-most-once, at-least-once and exactly-onceThe three delivery guarantees, defined by what a crash between processing and committing does, and which part of the pipeline each one covers.Intermediate22 min
- 2Duplicates: producer idempotence vs consumer idempotencyEvery place a duplicate can enter a Kafka pipeline, the single one that producer idempotence removes, and how to make a consumer safe against the rest.Intermediate25 min
- 3Kafka transactions and the read-process-write loopHow a transactional producer makes writes to several partitions and the consumer's offset commit atomic, and exactly where that guarantee stops.Advanced30 min
- 4Kafka transactions and your databaseA Kafka transaction covers Kafka partitions and consumer offsets, never your database. See the dual-write failure windows and the designs that close them.Advanced22 min
- 5Effectively-once processing for payments and ordersDelivery stays at-least-once. Design the payment and order flow so each business effect happens once: stable keys, unique constraints, an outbox and safe retries.Advanced25 min
Related labs
Simulations, not a real cluster.
- Lab 5Duplicate message processingIntermediate25 min
- Lab 6Consumer crash before offset commitIntermediate25 min
- Lab 7Poison message and dead-letter queueIntermediate25 min
- Lab 12Transactional outboxAdvanced30 min
- Lab 13Database unavailable while consuming Kafka messagesAdvanced35 min
- Lab 14Kafka transactionAdvanced35 min