Practice
SEV-1 simulationConfig and deploys · Real outage · Reconstructed data · 15–20 minutes
Incident #002 — Every core at 100%
Customer impact
Every edge machine is at 100% CPU. Visitors get 502 errors and traffic served is down 80%.
| Signal | Current value |
|---|---|
| Edge CPU | ≈100%, everywhere |
| Traffic served | down 80% |
| 502 responses | every location |
| Requests arriving | normal |
| Engineer logins | failing |
Your task
You are on call for the edge network. Work out what is using the CPU, using the service map, metrics, logs, traces and recent deployments. Stop the impact first, then name the cause and the permanent fix.
13:46Impact continuing · +0 minabout 0 million failed requests lost since you joined
Investigate · each first look costs 2 simulated minutes
What do you want to look at?
Nothing open yet. Pick a view above. You do not have to open them all before you act.
Step 1 · mitigate · you can act at any time