Step 1 · The pattern
One primary, one replica
Writes go to the primary. It ships each change to a replica, which serves the reads. Splitting the work this way lets the database handle far more traffic.
- request
- change being replicated
- done
In short
Replication lag: how it works and how it breaks
To handle more reads, a database is often run as one primary that accepts writes and several replicas that copy its changes and serve queries. The copying is usually asynchronous: the primary confirms a write first and ships it to the replicas afterwards. The delay, replication lag, is normally milliseconds, but it stretches to seconds under heavy writes, a long transaction or a slow network. During that delay a replica answers with slightly old data. The classic symptom is a user who saves something, is redirected to a page that reads from a replica, and does not see what they just saved. They save again, and now there are two. The usual fix is read-your-writes: for a short time after a user writes, send that user's reads to the primary, or to a replica known to have caught up to the write's position. Everyone else can keep reading from replicas.
How this comes up in interviews
"A user saves a comment, the page reloads, and the comment is missing. A moment later it appears. Your reads go to replicas. What happened and how do you fix it?"
Then they ask
- Why not send every read to the primary? What do you give up?
- You pin a user to the primary for a few seconds after a write. When is that timer wrong, and what is more precise?
- With several replicas, a user sees a comment and then loses it on refresh. What guarantee is missing, and how do you provide it?
A strong answer covers
- Replication is usually asynchronous: the primary confirms a write first and ships it afterwards, so a replica serves slightly old data during the delay.
- Lag is normally milliseconds but stretches to seconds under heavy writes, a long transaction or a slow network.
- The classic symptom is a user who cannot see their own write and saves it again, which creates a duplicate.
- Read-your-writes: for a short time after a user writes, send that user's reads to the primary or to a replica known to have caught up.
- Everyone else keeps reading from replicas, so the primary does not carry all the read load.
Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.
On one page: the track cheat sheet.
Sources: PostgreSQL documentation: high availability, load balancing and replication ↗ · Consistency tradeoffs in modern distributed database system design (PACELC) ↗ · GitHub: October 21 post-incident analysis ↗ · Report a problem
Diagrams are our own simulations: the timings, rates and counts in them are illustrative, not measurements. Captions are original wording.