Step 1 · The pattern
One room, many servers
Each user keeps a connection open to one gateway. The members of a room end up on different gateways, so a message posted on one gateway has to reach users on the others. #big is spread over gateways 1 and 2, #team over 2 and 3.
- message
- delivered to a member
In short
Message fan-out: how it works and how it breaks
A chat service keeps each user's connection open on one of many gateway servers. The members of a room are scattered across those gateways, so a message posted on one must reach users on several others. The simplest design copies every message to every gateway and lets each one discard what its users cannot see. Each gateway's work then follows the traffic of the whole service, not of its own users, and adding gateways adds more copies. Looping over members on the sender's server is no better: one big room holds up everything behind it. The usual design is publish/subscribe per room. A gateway subscribes to a room when its first member connects and unsubscribes when the last one leaves. A message is published once to the room's topic, and only the subscribed gateways receive it and pass it to their users.
How this comes up in interviews
"Users of a chat service are connected to many gateway servers. Someone posts in a room whose members are spread across them. How does the message reach everyone?"
Then they ask
- Why not copy every message to every gateway and let each one filter? What happens as you add gateways?
- A gateway dies. What happens to its subscriptions, and to messages published while its users reconnect?
- How do you keep every member seeing a room's messages in the same order?
A strong answer covers
- Each user's connection is held by one gateway, so a room's members are scattered across gateways.
- Broadcasting every message to every gateway ties each gateway's work to the whole service's traffic, and more gateways means more copies.
- Looping over members on the sender's server lets one big room hold up everything behind it.
- Use publish/subscribe per room: a gateway subscribes when its first member connects and unsubscribes when the last one leaves.
- A message is published once to the room's topic. Only subscribed gateways receive it and pass it to their own users.
Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.
On one page: the track cheat sheet.
Sources: Real-time Messaging (Slack Engineering) ↗ · How Discord Scaled Elixir to 5,000,000 Concurrent Users (Discord) ↗ · Report a problem
Diagrams are our own simulations: the timings, rates and counts in them are illustrative, not measurements. Captions are original wording.