Skip to content
All concepts

Concept

Hot partition

A hot partition is one partition or shard that gets far more data or traffic than the others. It is usually caused by the choice of key: everything for one large customer, or one busy channel, hashes to the same place. The rest of the system has spare capacity that the hot partition cannot use.

Watch one tenant's messages all land in one partition while its consumer falls behind. A live step from the lesson Hot partition key (Kafka / streaming). Timings and counts are illustrative.

Where you'll see it

Sources: Apache Kafka documentation: producer partitioning ↗ · Discord: How Discord stores trillions of messages ↗ · Slack: Scaling Slack's job queue ↗ · Report a problem

The definition is in our own words; the sources are those of the lesson Hot partition key.