Skip to content

BuildBreak lab 2 of 15

Partition key and ordering

Order status events sent without a key are consumed out of order. Key them by order id, retest, then see what adding a partition does to a key.

Foundation20 min

Read first: Keys and partitioning: who decides where a record goes · Partitions and offsets

Simulation.This lab is a deterministic simulation. No Kafka broker is running; logs and metrics are simulated.

Brokers
3 (KRaft)
Topics
order-status: 3 partitions, RF 3
Producers
order-service
Consumer groups
order-tracking (1)
Guided run
6 stages: build → break → observe → fix → retest

1

Business scenario

The order service publishes a status event each time an order moves on: created, then paid, then shipped. A tracking service consumes the order-status topic and keeps the latest status of every order in its own table, which is what the customer sees on the "Where is my order?" page. The events are published with no key. In testing, with a trickle of traffic, the page was always right. On the first busy evening customers who already have a dispatch email are looking at "paid", and a few at "created".

2

Starting architecture

Drawing diagram…

The order service producer writes created, paid and shipped events to a topic with three partitions. One tracking service consumer reads all three partitions and upserts the latest status per order into the order_tracking table.

flowchart LR
  OS["Order service producer"] -->|"created, paid, shipped"| T
  subgraph T["Topic order-status"]
    P0["Partition 0"]
    P1["Partition 1"]
    P2["Partition 2"]
  end
  P0 --> TR["Tracking service consumer"]
  P1 --> TR
  P2 --> TR
  TR -->|"upsert latest status"| DB[("order_tracking table")]
Three partitions, one tracking consumer that reads all of them and overwrites the order's row with each event.

3

Expected behaviour

  • For any one order, the tracking service processes created before paid before shipped.
  • The order_tracking row of a shipped order says "shipped" and never goes back to an earlier status.
  • Events of different orders may be processed in any relative order: nothing depends on that.

4 · Build → break → observe → fix → retest

Guided run

buildStage 1 of 6

A trickle of traffic looks fine

Do this

  1. Turn on Order service publishing (2 events per second, no record key) and press play.
  2. Watch the Partition logs panel: events of the same order land on different partitions.

Look for

  • "Out-of-order deliveries" stays at 0 and every row in order_tracking is correct.
  • The consumer is faster than the producer, so it reads each event almost as it arrives. Arrival order is hiding the problem.

Not done yet. The simulator checks this for you:consumed >= 12 && outOfOrder == 0

Simulator

Simulation · not a real cluster
simulated time 0.0s
Speed

Controls for this stage

2 status events per second.

Partition logs

simulated records
  • order-status-0leader b1ISR 1,2,3HW 0end 0

    empty

    order-tracking: committed none, lag 0

  • order-status-1leader b2ISR 2,3,1HW 0end 0

    empty

    order-tracking: committed none, lag 0

  • order-status-2leader b3ISR 3,1,2HW 0end 0

    empty

    order-tracking: committed none, lag 0

dashed = above the high watermark (not replicated, not readable)dim = committed by every groupviolet = same event written twice

Database

simulated · available
order_tracking0 rowsno unique constraint on order_id
Rows of table order_tracking: flagged rows first, then the newest
idorder_idvaluewrittennote
no rows yet

Simulated metrics

computed by the simulation
Events consumed
0
Out-of-order deliveries
0
Orders showing an old status
0
Consumer lag
0records
Keys split across partitions
0
Partitions
3

Consumer group order-tracking

stablegeneration 1lag 0

RangeAssignor (eager)commit after batchmax.poll.records 10

  • tracking-1runningprocessed 0
    order-status-0order-status-1order-status-2
Partitions of group order-tracking: owner, committed offset and lag
PartitionOwnerCommittedLag
order-status-0 tracking-1none0
order-status-1 tracking-1none0
order-status-2 tracking-1none0

Simulated log

newest first · not real broker output
  1. 0.0sINFO [simulation] Simulated cluster ready: 3 brokers, order-status (3 partitions, RF 3). KRaft mode, no ZooKeeper

The root cause, the solution with its code, and what the simulation leaves out appear when you finish the last stage.