Practice
SEV-1 simulationCapacity and scaling · Real outage · Reconstructed data · 15–20 minutes
Incident #014 — Overloaded, yet idle
Customer impact
The app is unavailable. The web tier is saturated, and your dashboards and alerting are down.
| Signal | Current value |
|---|---|
| Web tier | saturated |
| Web tier CPU | low |
| Apache workers busy | near 100% |
| Packet loss between VPCs | elevated |
| Autoscaling | requested 1,200 servers |
| Dashboards and alerting | down |
Your task
External monitoring paged you; your own dashboards are not loading. Work out why the web tier is saturated but idle, using the service map, metrics, logs, traces and recent changes. Stop the impact first, then name the cause and the permanent fix.
07:05Impact continuing · +0 minabout 0 thousand failed messages lost since you joined
Investigate · each first look costs 2 simulated minutes
What do you want to look at?
Nothing open yet. Pick a view above. You do not have to open them all before you act.
Step 1 · mitigate · you can act at any time