Incident · Medium · DNS/HTTP · Infra
Fixing DNS didn't fix it
- SEV1
- 07:30 PST
- INTERNAL NETWORK
90s
The situation
An automated scaling job runs on your internal network at 07:30. Soon internal DNS lookups fail, the console is unreachable and your monitoring goes dark. The team moves internal DNS away from the congested path and lookups recover. Everything else stays broken.
The evidence
- DNS errors
- connection attempts
Logs and signalsReconstructed
07:30automated scaling activity on internal network07:31clients: connection surge to network devices07:33internal DNS: errors09:28DNS moved off congested path → DNS errors gone09:30devices still congested · clients retrying without backing off
Your call
What is keeping the outage alive?
Charts and logs are reconstructions. Aim for 90 seconds; going over costs nothing.