Step 1 · The pattern
One controller runs the cluster
Brokers store the data, and one active controller decides which broker leads each partition. Every broker follows the same controller, so every client gets the same map of the cluster.
- write
- confirmed
In short
Controller quorum: how it works and how it breaks
Every Kafka cluster has one active controller. It decides which broker leads each partition, so every client gets the same answer about where to send a write. Two controllers at once is split brain: each hands out its own version of the cluster, and writes accepted on one side are unknown to the other. Kafka prevents this with a quorum. In KRaft mode a small, fixed set of controller nodes, usually three or five, elects a leader by majority vote. Older clusters relied on ZooKeeper for the same job. A group cut off from the majority cannot win an election, and a broker that cannot reach the active controller is fenced and stops serving clients. The price is availability: lose the majority and nothing can be elected. The protection also assumes every node agrees on who the members are, so change the controller set one node at a time.
How this comes up in interviews
"A network partition splits a Kafka cluster in two. How does Kafka make sure both halves do not accept writes as separate clusters?"
Then they ask
- With three controllers, one is isolated along with a few brokers. What does each side do?
- What does it cost you, in availability, to require a majority for every election?
- How would you safely replace or add controller nodes in a running cluster?
A strong answer covers
- One active controller decides which broker leads each partition, so every client gets the same answer.
- Two controllers at once is split brain: writes accepted on one side are unknown to the other.
- Election needs a majority of the full controller set. Any two majorities share a member, so two winners are impossible.
- A broker that cannot reach the active controller is fenced and stops serving clients.
- The price is availability: lose the majority and nothing can be elected. Change membership one node at a time.
Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.
On one page: the track cheat sheet.
Sources: Apache Kafka documentation: KRaft ↗ · KIP-500: Replace ZooKeeper with a self-managed metadata quorum ↗ · Raft: In search of an understandable consensus algorithm ↗ · Report a problem
Diagrams are our own simulations: the timings, rates and counts in them are illustrative, not measurements. Captions are original wording.