Skip to content

Use case · Social & messaging

News feed celebrity fan-out

Copying every new post into each follower's feed is fast, until one account has ten million followers. Watch one post clog the queue, then pull it at read time.

Used by apps like X, Instagram, Facebook, LinkedIn and Threads

Happy path · break it · fix it

News feed celebrity fan-out: sequence diagram

POST /postPosterappPostsserviceFan-outworkersFeed cacheper readerReaderapp

News feed celebrity fan-out, shown as messages between Poster (The app of someone publishing a post: an ordinary user or a celebrity), Posts (The post service, which stores every post once, by author), Fan-out (The fan-out queue and its workers, which copy a new post's id into followers' feeds), Feed cache (The feed service and its cache: one ready-made list of post ids per reader), Reader (The app of a follower opening their home feed).

Happy path: 8 steps.

  1. Poster → Posts: POST /post. An ordinary user with 300 followers publishes a post.
  2. Posts: post p1 stored. The post itself is stored once, under its author.
  3. Posts → Poster: 201 posted (reply). The poster gets an answer straight away. The rest happens in the background.
  4. Posts → Fan-out: job p1. One fan-out job is queued: add this post to the feed of each of the 300 followers.
  5. Fan-out → Feed cache: add p1 (many at once). Workers write the post's id into every follower's cached feed. It is done in a moment.
  6. Feed cache: feeds ready to read. Each follower's feed is now a finished list, built before anyone asks for it.
  7. Reader → Feed cache: GET /feed. A follower opens the app and asks for their home feed.
  8. Feed cache → Reader: feed + p1 (reply). One read returns the ready-made feed, new post included. Reads stay cheap.

Result: Work is done once at post time, so every read is one lookup.

Break it: 10 steps.

  1. Poster → Posts: POST /post. This time the poster is a celebrity with 10 million followers (an illustrative number).
  2. Posts → Poster: 201 posted (reply). The post is stored once and the celebrity gets the usual quick answer.
  3. Posts → Fan-out: job c1. The same code path queues the same kind of job, but this one means ten million feed writes.
  4. Fan-out → Feed cache: add c1 (many at once). Every worker starts copying the same post into feed after feed. It will take minutes.
  5. Fan-out: queue backs up. Writes arrive far faster than workers finish them, so the fan-out queue grows.
  6. Poster → Posts: POST /post. A moment later an ordinary user posts to their 300 followers, as in the happy path.
  7. Posts → Fan-out: job p1. Their small job joins the back of the same queue. Feeds are meant to be fresh within 5 seconds.
  8. Fan-out → Feed cache: add p1 (never arrives). It waits behind millions of writes for someone else's post, and the 5 seconds run out.
  9. Reader → Feed cache: GET /feed (many at once). The ordinary user's friends open the app and pull to refresh.
  10. Feed cache → Reader: no p1 yet (reply) (many at once). Their cached feeds are stale: the friend's post is stored, yet nobody can see it.

Result: One famous account made everybody's feed late.

Fix it: 10 steps.

  1. Poster → Posts: POST /post. The same celebrity with 10 million followers publishes the same post.
  2. Posts: 10M > limit: no job. Over the follower limit: the post is stored once and marked for pulling at read time.
  3. Posts → Poster: 201 posted (reply). The celebrity gets the same quick answer. Nothing was added to the fan-out queue.
  4. Poster → Posts: POST /post. A moment later the ordinary user posts to their 300 followers.
  5. Posts → Fan-out: job p1. Under the limit, so it fans out as before, and the queue in front of it is empty.
  6. Fan-out → Feed cache: add p1 (many at once). The friends' cached feeds get the post within the 5 seconds. The queue stays short.
  7. Reader → Feed cache: GET /feed. A reader opens the app. Their cached feed holds posts from ordinary accounts only.
  8. Feed cache → Posts: new from my celebs?. The feed service also asks for recent posts by the few celebrities this reader follows.
  9. Posts → Feed cache: c1 (reply). A handful of short lookups, one per celebrity followed, and easy to cache.
  10. Feed cache → Reader: p1 + c1 (reply). The two lists are merged by time: the friend's post and the celebrity's post both appear.

Result: Ten million writes became a few reads, only for readers who show up.

step 1 of 8

An ordinary user with 300 followers publishes a post.

  • request
  • success
  • held / reserved

At scale

Every dot is one feed write. Ordinary users post at a steady pace; three seconds in, the account in the middle posts. Pick how many followers it has. Numbers are illustrative and not to scale.

Numbers are illustrative: a small simulation, not a measurement.

The happy path, step by step

  1. A user publishes a post from the app.
  2. The post service stores the post once, under its author, and answers the poster straight away.
  3. It queues one fan-out job for the post.
  4. Fan-out workers look up the author's followers and add the post's id to each follower's cached feed.
  5. For an account with a few hundred followers that is a few hundred small writes, finished in a moment.
  6. Every follower's feed is now a ready-made list, built before anyone asks for it.
  7. A follower opens the app and asks for their home feed.
  8. The feed service returns the cached list in one read, with the new post in it.

The failure

What breaks and why

Fan-out on write does work in proportion to the author's follower count, and follower counts are wildly uneven. A post from an account with millions of followers goes down the same code path and becomes millions of feed writes, far more than the workers finish per second, so the fan-out queue backs up. The queue is shared, so an ordinary user's small job posted a moment later waits behind someone else's millions of writes, and their friends refresh and see nothing new. Most of that work is also wasted: many of those followers will not open the app before the post is old.

Before / after

The fix

Use both models and choose per author. Posts from ordinary accounts are still pushed into followers' cached feeds, which keeps reads cheap. An author over a follower limit is stored once and never fanned out. When a reader opens the app, the feed service reads the cached feed, fetches recent posts from the few very large accounts that reader follows, and merges the two by time. The cost moves from millions of writes at post time to a few cacheable reads, paid only for readers who turn up. The numbers and the limit here are illustrative.

Before · pseudo-codePush every post to every follower
on_post(post):
  store(post)
  for follower in followers(post.author):
    push(feed[follower], post.id)   // 10M writes

on_read(reader):
  return feed[reader]
After · pseudo-codePush small accounts, pull big ones
on_post(post):
  store(post)
  if follower_count(post.author) > LIMIT:
    return                          // pull at read
  for follower in followers(post.author):
    push(feed[follower], post.id)

on_read(reader):
  pushed = feed[reader]
  pulled = [recent_posts(c)
            for c in big_accounts_followed(reader)]
  return merge_by_time(pushed, pulled)

From their own write-ups

How real companies approach it

How this comes up in interviews

"Design the home feed for a social network. What happens when an account with millions of followers posts?"

Then they ask

  1. Where do you set the follower limit between push and pull, and what happens to an account that crosses it?
  2. A reader follows many very large accounts. How do you keep that reader's feed fast?
  3. Ordinary users' posts arrive late while one large fan-out is running. How do you isolate them?

A strong answer covers

  • Fan-out on write makes a feed read one cached list, but its work is in proportion to the author's follower count.
  • A post from a huge account becomes millions of feed writes, backs up the shared queue and delays everyone else's posts.
  • Choose per author: push for ordinary accounts, store posts from accounts over a follower limit once and never fan them out.
  • At read time, merge the cached feed with recent posts from the few very large accounts the reader follows, by time.
  • The cost moves to a few cacheable reads, paid only for readers who turn up.

Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.

Your turn

What would you do?

Your feed copies each new post into every follower's cached feed through one worker queue. An account with 12 million followers posts, the queue grows for minutes, and ordinary users' posts reach their friends late. Which change removes the cause?

News feed celebrity fan-out questions

What is the difference between fan-out on write and fan-out on read?
Fan-out on write (push) copies each new post into every follower's stored feed when it is published, so reading a feed is a single lookup. Fan-out on read (pull) stores each post once and builds the feed when a reader asks, by fetching recent posts from everyone they follow. Push makes reads cheap and writes costly; pull does the opposite.
What is the celebrity problem in news feed design?
With fan-out on write, one post costs one write per follower. For an account with millions of followers a single post becomes millions of writes, which can back up the fan-out queue and delay everyone else's posts. The usual answer is a hybrid: push for ordinary accounts, and pull the posts of very large accounts when a feed is read.
Is this the same as Instagram's "Justin Bieber problem"?
It is a cousin of it. Wired reported in 2015 that so many likes on one celebrity's photos overloaded Instagram's cache and database, and that Instagram engineers described moving to a like count kept in a single database cell. That is a hot counter, not feed fan-out, but the root is the same: one account far larger than the design assumed.

Sources: LinkedIn Engineering: FollowFeed: LinkedIn's Feed Made Faster and Smarter ↗ · Engineering at Meta: Serving Facebook Multifeed: efficiency, performance gains through redesign ↗ · Engineering at Meta: TAO: The power of the graph ↗ · Wired: How Instagram Solved Its Justin Bieber Problem (press reporting, 2015) ↗ · Report a problem

Last updated: Engineering write-up

A typical design, informed by public write-ups, not any one company's exact system. Diagrams are our own. Numbers in the simulation are illustrative. All wording is original and paraphrases the linked sources.