Skip to content

Path 5 of 10

Message Delivery Semantics

At-most-once, at-least-once and exactly-once, and where Kafka's guarantees stop and your database's begin.

0 / 5 lessons completed ยท about 124 min in total

After this path you can

Outcomes

  • State precisely what each guarantee covers
  • Separate producer idempotence from consumer idempotency
  • Design effectively-once processing for payments and orders

Lessons

  1. 1At-most-once, at-least-once and exactly-onceThe three delivery guarantees, defined by what a crash between processing and committing does, and which part of the pipeline each one covers.Intermediate22 min
  2. 2Duplicates: producer idempotence vs consumer idempotencyEvery place a duplicate can enter a Kafka pipeline, the single one that producer idempotence removes, and how to make a consumer safe against the rest.Intermediate25 min
  3. 3Kafka transactions and the read-process-write loopHow a transactional producer makes writes to several partitions and the consumer's offset commit atomic, and exactly where that guarantee stops.Advanced30 min
  4. 4Kafka transactions and your databaseA Kafka transaction covers Kafka partitions and consumer offsets, never your database. See the dual-write failure windows and the designs that close them.Advanced22 min
  5. 5Effectively-once processing for payments and ordersDelivery stays at-least-once. Design the payment and order flow so each business effect happens once: stable keys, unique constraints, an outbox and safe retries.Advanced25 min

Related labs

Simulations, not a real cluster.

  1. Lab 5Duplicate message processingIntermediate25 min
  2. Lab 6Consumer crash before offset commitIntermediate25 min
  3. Lab 7Poison message and dead-letter queueIntermediate25 min
  4. Lab 12Transactional outboxAdvanced30 min
  5. Lab 13Database unavailable while consuming Kafka messagesAdvanced35 min
  6. Lab 14Kafka transactionAdvanced35 min