Practice
SEV-1 simulationDatabases · Real outage · Reconstructed data · 15–20 minutes
Incident #006 — Forty-three seconds
Customer impact
The site is slow and some pages show out-of-date data. Database primaries have moved to the other coast.
| Signal | Current value |
|---|---|
| Web p95 | ≈2.4 seconds |
| MySQL primaries | West Coast |
| Writes only on East | 954, busiest cluster |
| Network link | back after 43 s |
| Webhooks and Pages | running |
| Deploys | locked |
Your task
You are the incident commander. Work out why a 43-second network blip left the site degraded, using the service map, metrics, logs, traces and recent changes. Then choose what to do first, and name the cause and the permanent fix.
23:13Impact continuing · +0 minabout 0 thousand writes at risk lost since you joined
Investigate · each first look costs 2 simulated minutes
What do you want to look at?
Nothing open yet. Pick a view above. You do not have to open them all before you act.
Step 1 · mitigate · you can act at any time