Practice
SEV-1 simulationConfig and deploys · Real outage · Reconstructed data · 15–20 minutes
Incident #003 — The network that keeps blinking
Customer impact
The proxy fleet is returning 5xx on and off, in a five-minute rhythm. Your status page is unreachable.
| Signal | Current value |
|---|---|
| 5xx on HTTP traffic | on and off, 5 min |
| Bot feature file | ≈2× usual size |
| Requests arriving | normal |
| Workers KV errors | elevated |
| Dashboard logins | failing |
| Status page | unreachable |
Your task
You join the incident call. Work out why the network keeps failing and recovering, using the service map, metrics, logs, traces and recent changes. Stop the impact first, then name the cause and the permanent fix.
11:35Impact continuing · +0 minabout 0 million failed requests lost since you joined
Investigate · each first look costs 2 simulated minutes
What do you want to look at?
Nothing open yet. Pick a view above. You do not have to open them all before you act.
Step 1 · mitigate · you can act at any time