Path 7 of 10
Distributed Systems Design Patterns
Outbox, CDC, sagas, event sourcing, CQRS and the messaging patterns built on Kafka.
0 / 8 lessons completed ยท about 255 min in total
After this path you can
Outcomes
- Publish events reliably alongside a database write
- Coordinate a multi-service workflow with a saga
- Choose a pattern from its trade-offs, not its name
Lessons
- 1Transactional outboxWrite the event to a table in the same database transaction as the business row, then let a relay publish it to Kafka. No event is lost, and none is published for a rolled-back change.Intermediate30 min
- 2Change data capture with DebeziumLog-based CDC reads the database's write-ahead log and turns every committed row change into a Kafka record. How Debezium does it on Kafka Connect, and what to watch in production.Advanced35 min
- 3The saga patternA business transaction across services as a chain of local transactions, each with a compensating action. Choreography vs orchestration, idempotency, timeouts and living without isolation.Advanced35 min
- 4Event sourcingStore every change to an order as an event and derive the current state by replaying them. What Kafka does well as the log, where it falls short as the event store, and what teams do instead.Advanced35 min
- 5CQRS: read models built from eventsKeep one model for changing orders and separate query tables built by Kafka consumers. Learn what the lag costs, how to rebuild a projection, and when the pattern is not worth it.Advanced30 min
- 6Retry topics and dead letter topics as architectureDesign the retry and dead letter topology for a consumer: how many topics, who owns them, what happens to ordering, how big they get and what to watch.Advanced30 min
- 7The idempotent consumer patternMake a consumer safe to run twice on the same event: pick the right idempotency key, record it in the same transaction as the effect, clean the table up, and cover external calls.Advanced30 min
- 8Request-reply and event fan-outTwo messaging shapes on Kafka: asking a question and waiting for the answer on a reply topic, and publishing one event to many independent consumer groups. One of them fits Kafka far better.Intermediate30 min
Related labs
Simulations, not a real cluster.
- Lab 3Consumer group assignmentFoundation20 min
- Lab 4Adding and removing consumersIntermediate25 min
- Lab 5Duplicate message processingIntermediate25 min
- Lab 6Consumer crash before offset commitIntermediate25 min
- Lab 7Poison message and dead-letter queueIntermediate25 min
- Lab 8Retry and backoff strategyIntermediate30 min
- Lab 10Consumer lag during high loadIntermediate30 min
- Lab 12Transactional outboxAdvanced30 min
- Lab 13Database unavailable while consuming Kafka messagesAdvanced35 min
- Lab 15Schema evolutionAdvanced35 min