Incident · Hard · DNS/HTTP · Infra
Name servers alive, nobody home
- SEV0
- ~6 HOURS
- EVERYTHING
90s
The situation
Every product you run disappears from the internet at once. Resolvers worldwide return SERVFAIL for your domains. Your name servers are powered on and healthy, but nobody can route to them. Internal tools and chat are failing too. Minutes earlier, a routine maintenance command ran on the backbone that links your data centres.
The evidence
- backbone links up
- DNS routes announced
Logs and signalsReconstructed
15:39backbone: capacity-assessment command issued15:39audit tool: command not blocked15:40backbone: all links down15:40edge DNS: cannot reach data centres → withdrawing own BGP routes15:41world: SERVFAIL15:45remote access tools: cannot resolve
Your call
What is the root cause?
Charts and logs are reconstructions. Aim for 90 seconds; going over costs nothing.