Concept
Metastable failure
A metastable failure is an outage that keeps itself going after the thing that triggered it has been fixed. Something in the overloaded state, usually retries or cold caches, creates enough extra work to hold the system there. Recovery needs the loop broken from outside: shed load, stop retries, then let traffic back gradually.
Where you'll see it
Related concepts
Sources: Bronson et al.: Metastable Failures in Distributed Systems (HotOS 2021) ↗ · Huang et al.: Metastable Failures in the Wild (OSDI 2022) ↗ · Report a problem
The definition is in our own words; the sources are those of the lesson Metastable failure.