Skip to content
Practice

SEV-1 simulationConfig and deploys · Real outage · Reconstructed data · 15–20 minutes

Incident #002 — Every core at 100%

Customer impact

Every edge machine is at 100% CPU. Visitors get 502 errors and traffic served is down 80%.

SignalCurrent value
Edge CPU≈100%, everywhere
Traffic serveddown 80%
502 responsesevery location
Requests arrivingnormal
Engineer loginsfailing

Your task

You are on call for the edge network. Work out what is using the CPU, using the service map, metrics, logs, traces and recent deployments. Stop the impact first, then name the cause and the permanent fix.

13:46Impact continuing · +0 minabout 0 million failed requests lost since you joined

Investigate · each first look costs 2 simulated minutes

What do you want to look at?

Nothing open yet. Pick a view above. You do not have to open them all before you act.

Step 1 · mitigate · you can act at any time

Most of your customers' traffic is failing right now. What do you do first?