Step 1 · The pattern
A stage, a gate and a snapshot
Changes go to stage 1 first. After a wait called the bake, a health gate decides whether the other servers get it. The state that passes is saved as last known good, which is what a rollback restores.
- request
- served
In short
Bake time & health gates: how it works and how it breaks
A staged rollout puts a change on a first stage, waits, and then asks a health gate whether to continue. That wait is the bake time, and it decides what the gate is able to see. Some faults are slow: a background job, a cache that has to fill, a leak. If the fault needs longer to appear than the bake lasts, the gate sees a healthy stage and passes. The change is then promoted everywhere, and it is also saved as the last known good snapshot, the state a rollback restores. When the servers start failing, rolling back puts the same fault straight back. The fix is to bake longer than your slowest failure takes to show, and to treat a snapshot as only as trustworthy as the gate that approved it. A longer wait costs speed. It buys a fault that stops at stage one.
How this comes up in interviews
"A change passed its health gate, was promoted everywhere, then failed. The rollback restored the same fault. What happened, and how do you prevent it?"
Then they ask
- How long should a stage bake, and how do you decide without making every release take days?
- Which faults show late, and which signals would you add to the gate so it can see them?
- When should a promoted state become the last known good snapshot that a rollback restores?
A strong answer covers
- A gate only reports on the window and the signals it watched, so a fault that needs longer than the bake passes untouched.
- The state that passes is saved as last known good, so a rollback restores exactly what the gate approved, fault included.
- Bake for longer than the slowest known failure takes to show, and keep watching after promotion.
- Measure the bake in requests served as well as time, and treat missing data as a failed gate.
- Keep an older snapshot within reach, and mark a state as last known good on a slower clock than promotion.
Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.
On one page: the track cheat sheet.
Sources: Azure Front Door post-incident review (October 2025) ↗ · Automating safe, hands-off deployments (Amazon Builders' Library) ↗ · Report a problem
Diagrams are our own simulations: the timings, rates and counts in them are illustrative, not measurements. Captions are original wording.