Skip to content
All outages

Incident · Easy · URL shortener · Databases

2,147,483,647

  • SEV1
  • ~5 HOURS
  • READ-ONLY
90s

The situation

Without warning your app can no longer save anything. People can read everything as usual, but every new comment, upload or to-do fails. Disk has plenty of room. Nothing was deployed. The busiest table in the database, which records every action, has been growing steadily for years.

The evidence

Highest ID in the busiest tableillustrative
  • max id
  • limit
Logs and signalsReconstructed
12:21INSERT INTO events … ERROR: value out of range for column id
12:21SELECT … OK
12:22disk: 41% used · connections: normal
12:23last id issued: 2147483647

Your call

What happened?

Charts and logs are reconstructions. Aim for 90 seconds; going over costs nothing.

Last updated: Official postmortem