What's the difference?
OOMKilled vs Evicted
Two ways to lose a pod to memory. Watch which component acts, what it removes, and what comes back.
Scenario 1 · OOMKilled
A container hits its own limit
Pod A's memory climbs to its 512Mi limit. The kernel kills the process with SIGKILL, exit code 137. The pod itself stays on the node, and the kubelet restarts the container in place. The other pods are not involved.
- memory
- kill
- restart
Scenario 1 · OOMKilled
A container hits its own limit
Pod A's memory climbs to its 512Mi limit. The kernel kills the process with SIGKILL, exit code 137. The pod itself stays on the node, and the kubelet restarts the container in place. The other pods are not involved.
- memory
- kill
- restart
- OOMKilledThe kernel lights up. One container dies and the kubelet starts it again in the same pod.
- EvictedThe kubelet lights up. A whole pod is removed and a replacement is created somewhere else.
How to tell them apart
| Question | OOMKilled | Evicted |
|---|---|---|
| Who acts | The Linux kernel's OOM killer | The kubelet on that node |
| What triggers it | One container reaches its own memory limit | Free memory on the node drops below the kubelet's threshold |
| What is removed | The container's process | The whole pod, all containers |
| What comes back | The same pod; the container is restarted in place | A new pod from the controller, possibly on another node |
| What you see | Last State: OOMKilled, exit code 137, RESTARTS rising | STATUS Evicted, phase Failed, “The node was low on resource: memory” |
| Who is chosen | The container that exceeded its limit | Pods using most above their requests, lower priority first |
| First check | kubectl describe pod: Last State and Limits | kubectl describe node: MemoryPressure and events |
Who acts
- OOMKilled
- The Linux kernel's OOM killer
- Evicted
- The kubelet on that node
What triggers it
- OOMKilled
- One container reaches its own memory limit
- Evicted
- Free memory on the node drops below the kubelet's threshold
What is removed
- OOMKilled
- The container's process
- Evicted
- The whole pod, all containers
What comes back
- OOMKilled
- The same pod; the container is restarted in place
- Evicted
- A new pod from the controller, possibly on another node
What you see
- OOMKilled
- Last State: OOMKilled, exit code 137, RESTARTS rising
- Evicted
- STATUS Evicted, phase Failed, “The node was low on resource: memory”
Who is chosen
- OOMKilled
- The container that exceeded its limit
- Evicted
- Pods using most above their requests, lower priority first
First check
- OOMKilled
- kubectl describe pod: Last State and Limits
- Evicted
- kubectl describe node: MemoryPressure and events
In short
OOMKilled vs Evicted: the difference
OOMKilled: one container broke its own limit, and the kernel killed it. Evicted: the node ran short, and the kubelet removed a whole pod.
The two look alike from a distance and are different events. OOMKilled is local to one container: its memory use reached the limit set in its own spec, and the Linux kernel killed the process. The pod stays where it is, the kubelet restarts the container in place, and the restart count goes up. Eviction is about the node: free memory on the machine fell below the kubelet's threshold, so the kubelet chose a pod and removed all of it. An evicted pod is not restarted on that node. It is left in the Failed phase with the reason Evicted, and its controller creates a replacement that the scheduler places wherever there is room. A pod can be evicted while well under its own limit, because the kubelet ranks pods by how far their usage exceeds their requests.
Your turn
Which one is this?
Which one is this? A pod with a 2Gi memory limit is using 600Mi when it disappears. kubectl get pods shows it as Failed, and a new pod with a different name is starting on another node.
Related errors
Questions
- What is the difference between OOMKilled and Evicted?
- OOMKilled means one container exceeded its own memory limit and the kernel killed it; the kubelet restarts it in the same pod. Evicted means the node ran low on a resource and the kubelet removed the whole pod; a controller creates a replacement.
- Can a pod be evicted while under its memory limit?
- Yes. Eviction is decided at node level. The kubelet looks at how much each pod uses above its requests, and at pod priority, not at whether a limit was reached.
- How do I avoid evictions?
- Set memory requests close to real usage so the scheduler does not overfill nodes, and keep limits near requests for important workloads. Pods whose usage stays within their requests are the last to be evicted.
Sources: Kubernetes docs: Node-pressure eviction ↗ · Kubernetes docs: Resource management for pods and containers ↗ · Report a problem
Diagrams are our own simulations. All wording is original and paraphrases the linked Kubernetes documentation.
Practise: diagnose the cluster
Read the terminal, watch the animation, pick the error and the first fix.