Practice
SEV-1 simulationPlatform and OS · Real outage · Reconstructed data · 15–20 minutes
Incident #009 — Bigger boxes didn't help
Customer impact
Players online have fallen to half of normal. The Consul cluster everything depends on is slow and unstable.
| Signal | Current value |
|---|---|
| Players online | ≈50% of normal |
| Consul KV write p50 | ≈2 s (normal <300 ms) |
| Consul leader | out of sync with voters |
| Scheduling | failing |
| Vault | degraded |
| Consul server CPU | high on the leader |
Your task
You join the incident on the first evening. Work out why Consul is slow, using the service map, metrics, logs, traces and recent changes. Choose what to do, then name the cause and the permanent fix.
16:40Impact continuing · +0 minabout 0 thousand player sessions lost lost since you joined
Investigate · each first look costs 3 simulated minutes
What do you want to look at?
Nothing open yet. Pick a view above. You do not have to open them all before you act.
Step 1 · mitigate · you can act at any time