Readiness probe failure
The pod is Running but not Ready, so the Service has stopped sending it traffic. The container is not restarted.
$ kubectl get pods -l app=webNAME READY STATUS RESTARTS AGE web-6b7f9c5d8-4hx2m 0/1 Running 0 12m web-6b7f9c5d8-9qk7t 0/1 Running 0 12m web-6b7f9c5d8-tz5wd 0/1 Running 0 12m
$ kubectl describe pod web-6b7f9c5d8-4hx2mConditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Events: Type Reason Age From Message Warning Unhealthy 3s (x41 over 6m) kubelet Readiness probe failed: HTTP probe failed with statuscode: 503
$ kubectl get endpointslices -l kubernetes.io/service-name=webNAME ADDRESSTYPE PORTS ENDPOINTS AGE web-x7k2p IPv4 8080 10.42.1.17,10.42.2.9,10.42.3.21 12m
Watch it happen
How Readiness probe failure unfolds
Step 3 · The failure
Every pod fails at once
All three checks depend on the same thing, and it is down. One by one the pods are pulled out until the Service has no endpoints left. Every request now fails, although all three containers are still Running.
- failed
- NotReady
Step 3 · The failure
Every pod fails at once
All three checks depend on the same thing, and it is down. One by one the pods are pulled out until the Service has no endpoints left. Every request now fails, although all three containers are still Running.
- failed
- NotReady
Where it breaks: Your container fails its own readiness check. The kubelet reports that, and the EndpointSlice controller removes the pod from the Service. Nothing is broken in Kubernetes: it is doing what the probe told it to.
In short
What Readiness probe failure means
A readiness probe asks one question: can this container take requests right now? While the answer is no, the pod stays Running but shows 0/1 in the READY column. The kubelet records the failure, the pod's Ready condition turns False, and the EndpointSlice controller marks the pod as not ready in the Service's endpoints, so the proxy stops routing to it. Nobody restarts the container; restarting is the job of the liveness probe. With one pod out, the others carry the traffic and users notice nothing. The outage comes when every pod fails at once, typically because the readiness endpoint checks a shared dependency or because the probe points at the wrong port or path. Then the Service has no endpoints, and clients get connection errors or a 503 from the ingress in front.
In order
Diagnose in 60 seconds
- 1
kubectl get pods -l app=<name>- Look for
- STATUS Running with READY 0/1, and RESTARTS not rising.
- What it tells you
- Rules in a readiness problem. If RESTARTS climbs, look at liveness or crashes instead.
- 2
kubectl describe pod <pod>- Look for
- The Readiness line of the container (port, path, timings), and Unhealthy events with “Readiness probe failed” and the reason after it.
- What it tells you
- A status code means the app answered “not ready”. “connection refused” means nothing listens on that port. “context deadline exceeded” means it answered too slowly.
- 3
kubectl get endpointslices -l kubernetes.io/service-name=<service> -o yaml- Look for
- Under each endpoint, conditions.ready.
- What it tells you
- ready: false on every endpoint confirms the Service has nowhere to send traffic.
- 4
kubectl exec <pod> -- wget -qO- http://localhost:<port><path>- Look for
- What the readiness endpoint itself returns from inside the container.
- What it tells you
- Separates a wrong port or path in the probe from an app that really reports not ready, and usually names the failing dependency.
- 5
kubectl logs <pod> --tail=50- Look for
- Errors about the dependency the readiness handler checks: database, cache, another service.
- What it tells you
- Rules in the root cause behind the “not ready” answer.
Most common first
Root causes
1A dependency that the readiness endpoint checks is down or slow.
- Confirm
- The probe fails with a 5xx status in every pod at the same moment, and the logs name the dependency.
- Fix now
- Restore the dependency. The pods return by themselves on the next successful check.
- Prevent
- Decide which dependencies really make the pod unable to serve. Serve degraded responses where you can instead of reporting not ready.
2The probe points at the wrong port, path or scheme.
- Confirm
- The failure reads “connection refused” or a 404, and it began with a config or image change.
- Fix now
- Correct the port, path or scheme in the readinessProbe.
- Prevent
- Name the container port and refer to it by name in the probe.
3The app answers, but slower than the probe's timeout, which is one second by default.
- Confirm
- Events say “context deadline exceeded” or mention a timeout, mostly under load.
- Fix now
- Raise timeoutSeconds, and make the readiness handler cheap.
- Prevent
- Keep heavy work out of the readiness endpoint.
4The app is still starting, and the probe gives it too little time.
- Confirm
- New pods stay at 0/1 for a while after each deploy and then recover.
- Fix now
- Wait, or add a startup probe sized for the real boot time.
- Prevent
- Measure start-up time and set the startup probe from it.
5The pod is overloaded and refuses or queues the probe's connection.
- Confirm
- Pods flip between 1/1 and 0/1 during traffic peaks, and CPU is throttled or at its limit.
- Fix now
- Add replicas or CPU. Pods leaving the Service push more load onto the rest.
- Prevent
- Scale on load before saturation, and give the health endpoint its own small worker pool where the framework allows it.
The tempting wrong fix
Delete or restart the pods
Before / after
The most common fix: point the probe at what the app really serves
containers:
- name: web
ports:
- containerPort: 8080
readinessProbe:
httpGet:
path: /health # the app serves /ready
port: 80 # nothing listens on 80
periodSeconds: 10containers:
- name: web
ports:
- name: http
containerPort: 8080
readinessProbe:
httpGet:
path: /ready
port: http # refers to the named port
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3Defaults if you leave them out: a check every 10 seconds, a 1 second timeout, 3 failures to become not Ready and 1 success to become Ready again.
In the real world
Depends on version and provider
Seen in a real outage
The opposite failure is just as real: a check that keeps passing while the server cannot serve. At Val Town in 2024, a fixed group of customers had requests hang for seven hours while the health check stayed green.
Break it: Same customers, same sick serverYour turn
What would you do?
All pods of a Deployment show Running, 0/1 READY and 0 restarts. Clients get 503 from the ingress. Which statement is correct?
Related errors
Readiness probe failure questions
- Why is my pod Running but not Ready?
- Its readiness probe is failing. The container is up, but it is telling the kubelet it cannot take requests, so the pod shows 0/1 READY and is left out of the Service until the probe passes.
- Does a failed readiness probe restart the pod?
- No. A readiness failure only removes the pod from the Service's endpoints. The container keeps running. Restarts come from a failing liveness or startup probe, or from the process exiting.
- What happens when all pods fail their readiness probe?
- The Service has no ready endpoints. Requests to it fail, typically with a refused connection inside the cluster or a 503 from the ingress controller, until at least one pod passes its probe again.
Sources: Kubernetes docs: Readiness probe ↗ · Kubernetes docs: Configure liveness, readiness and startup probes ↗ · Kubernetes docs: EndpointSlices, conditions ↗ · Report a problem
Diagrams are our own simulations. Terminal output is illustrative. All wording is original and paraphrases the linked Kubernetes documentation.
Practise: diagnose the cluster
Read the terminal, watch the animation, pick the error and the first fix.