Incident · Medium · DNS/HTTP · Databases
The network that keeps blinking
- SEV1
- 11:20 UTC
- GLOBAL
90s
The situation
Your global proxy fleet starts returning 5xx on a huge share of traffic. Then it recovers. Then it fails again. Your status page, hosted somewhere else entirely, is also down.
The evidence
- 5xx %
Logs and signalsReconstructed
10:59bot-features.file size=1.0x11:05db: ClickHouse permissions change applied11:20bot-features.file size=2.0x11:20proxy panic: called unwrap() on Err (features > 200)11:25bot-features.file size=1.0x11:30bot-features.file size=2.0x-- file regenerated every 5 min
Your call
What's actually taking the network down?
Charts and logs are reconstructions. Aim for 90 seconds; going over costs nothing.