Incident · Medium · DNS/HTTP
The rollback made it worse
- SEV2
- <1% OF USERS
- ~24 HOURS
90s
The situation
You are enabling DNSSEC on your main domain, on the third attempt. The final step publishes a DS record at the parent zone, cached for up to 24 hours. A small share of users report that the app will not resolve. The team reacts fast and rolls back by removing the signatures from the zone. Now far more users cannot resolve it.
The evidence
- SERVFAIL
Logs and signalsReconstructed
t+0 DS record published at parent (TTL 24h)t+30msome resolvers: wildcard records fail validationt+1h rollback: zone signatures removedt+1h validating resolvers: DS cached, zone unsigned → SERVFAIL
Your call
Why did the rollback make it worse?
Charts and logs are reconstructions. Aim for 90 seconds; going over costs nothing.