What's the difference?
Readiness vs liveness vs startup probes
Three pods behind one Service, each with a different probe. The same thing goes wrong in all three at the same moment. Watch what the kubelet does to each.
Scenario 1 · Hang
All three apps hang
At the same moment every app stops answering. Readiness: after three failed checks the pod is pulled from the Service and gets no traffic, but nobody restarts it. Liveness: the kubelet restarts the container and it recovers, after serving errors in between. Startup: nothing happens. That probe finished long ago, so the pod keeps receiving requests it cannot answer.
- probe / request
- failed
- not yet / NotReady
- restart
- ok
Scenario 1 · Hang
All three apps hang
At the same moment every app stops answering. Readiness: after three failed checks the pod is pulled from the Service and gets no traffic, but nobody restarts it. Liveness: the kubelet restarts the container and it recovers, after serving errors in between. Startup: nothing happens. That probe finished long ago, so the pod keeps receiving requests it cannot answer.
- probe / request
- failed
- not yet / NotReady
- restart
- ok
$ kubectl get podsNAME READY STATUS RESTARTS AGE readiness-pod 0/1 Running 0 12m liveness-pod 1/1 Running 1 (20s ago) 12m startup-pod 1/1 Running 0 12m
$ kubectl describe pod readiness-podEvents: Type Reason Age From Message Warning Unhealthy 8s (x6 over 58s) kubelet Readiness probe failed: Get "http://10.244.1.21:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
$ kubectl describe pod liveness-podEvents: Type Reason Age From Message Warning Unhealthy 50s (x3 over 70s) kubelet Liveness probe failed: Get "http://10.244.1.22:8080/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Normal Killing 50s kubelet Container api failed liveness probe, will be restarted
- ReadinessReadiness: the pod is pulled out of the Service. Traffic stops, the container is not restarted.
- LivenessLiveness: the kubelet restarts the container. It heals from a hang, and it kills a slow starter.
- StartupStartup: only matters while booting. It does nothing about a later hang.
How to tell them apart
| Question | Readiness | Liveness | Startup |
|---|---|---|---|
| The question it asks | Can you take requests right now? | Are you stuck beyond recovery? | Have you finished starting? |
| When it fails | Pod removed from the Service's endpoints | Container killed, then restarted | Container killed, then restarted |
| Restarts the container? | No, never | Yes | Yes, but only during start-up |
| Stops traffic? | Yes, until it passes again | No. A restart does that only as a side effect | Yes: the pod is not ready until it has started |
| When it runs | For the whole life of the container | For the whole life of the container | Only until its first success |
| Without one | Ready as soon as the container has started | Never restarted for being stuck | Liveness and readiness begin immediately |
| Defaults | Every 10s, 1s timeout, 3 failures | Every 10s, 1s timeout, 3 failures | Every 10s, 1s timeout, 3 failures |
The question it asks
- Readiness
- Can you take requests right now?
- Liveness
- Are you stuck beyond recovery?
- Startup
- Have you finished starting?
When it fails
- Readiness
- Pod removed from the Service's endpoints
- Liveness
- Container killed, then restarted
- Startup
- Container killed, then restarted
Restarts the container?
- Readiness
- No, never
- Liveness
- Yes
- Startup
- Yes, but only during start-up
Stops traffic?
- Readiness
- Yes, until it passes again
- Liveness
- No. A restart does that only as a side effect
- Startup
- Yes: the pod is not ready until it has started
When it runs
- Readiness
- For the whole life of the container
- Liveness
- For the whole life of the container
- Startup
- Only until its first success
Without one
- Readiness
- Ready as soon as the container has started
- Liveness
- Never restarted for being stuck
- Startup
- Liveness and readiness begin immediately
Defaults
- Readiness
- Every 10s, 1s timeout, 3 failures
- Liveness
- Every 10s, 1s timeout, 3 failures
- Startup
- Every 10s, 1s timeout, 3 failures
In short
Readiness vs liveness vs startup probes: the difference
Readiness decides whether a pod gets traffic. Liveness decides whether its container is restarted. Startup holds the other two back until the app has started.
The three probes ask different questions and trigger different actions. A readiness probe asks whether the container can take requests right now. When it fails, the pod is taken out of the Service's endpoints and stops receiving traffic; the container is left alone and returns when the probe passes again. A liveness probe asks whether the container is beyond recovery. When it fails, the kubelet kills the container and restarts it; traffic is not touched until then. A startup probe asks whether the application has finished starting. Until it succeeds, the other two probes are not run, which protects slow starters from being killed early; after that it never runs again. All three share the same defaults: a check every 10 seconds, a one second timeout, and three failures in a row before acting. Most services want readiness and liveness together, plus startup when boot is slow.
All three together
containers:
- name: api
startupProbe: # about 5 min to boot (30 checks, 10s apart)
httpGet: { path: /healthz, port: 8080 }
periodSeconds: 10
failureThreshold: 30
livenessProbe: # is the process itself stuck?
httpGet: { path: /healthz, port: 8080 }
periodSeconds: 10
failureThreshold: 3
readinessProbe: # can it serve right now?
httpGet: { path: /ready, port: 8080 }
periodSeconds: 5
failureThreshold: 3Keep the liveness endpoint about the process only. If it checks the database, a database outage makes the kubelet restart every pod at once. Put dependency checks in readiness instead.
Interactive
Solve it
Case A
The Spring Boot app that never comes up
A Spring Boot service needs about a minute to start in production. Since a liveness probe was added last week it never becomes Ready, and its restart count keeps climbing.
$ kubectl get podsNAME READY STATUS RESTARTS AGE orders-5d9c7b6f48-t7xkp 0/1 Running 4 (25s ago) 3m10s
$ kubectl describe pod orders-5d9c7b6f48-t7xkpLiveness: http-get http://:8080/actuator/health/liveness delay=0s timeout=1s period=10s #success=1 #failure=3 Events: Warning Unhealthy 25s (x12 over 2m55s) kubelet Liveness probe failed: Get "http://10.244.2.9:8080/actuator/health/liveness": dial tcp 10.244.2.9:8080: connect: connection refused Normal Killing 25s (x4 over 2m35s) kubelet Container orders failed liveness probe, will be restarted
1 · What is wrong?
Try the numbers
How much time does a probe give you?
Move the sliders to see how much time a probe gives your app before something happens. The first check runs after initialDelaySeconds (straight after the container starts when that is 0), then one every periodSeconds. The action comes at the failureThreshold-th failure in a row, so at about initialDelaySeconds + (failureThreshold − 1) × periodSeconds.
Too slow. Killed at about 20s (plus up to the 1s timeout). The app needs 45s, so it never finishes starting, and the same happens after every restart.
Once running, an app that hangs is restarted after about 21–31s: 3 failed checks 10s apart, each waiting up to the 1s timeout.
Tap to answer
Which probe do I need?
Can the app take longer to start than your liveness probe would wait, about (failureThreshold − 1) × periodSeconds?
Can the process get stuck in a state that only a restart fixes, such as a deadlock, while it keeps running?
Does the pod receive traffic through a Service?
How a probe checks
- httpGet
- Passes when: The response status is 200 or higher and below 400. The usual choice for web services. Redirects are followed.
- tcpSocket
- Passes when: The port accepts a connection. Only proves something is listening, not that the app works.
- exec
- Passes when: The command exits with status 0. Runs inside the container on every check, so keep it cheap.
- grpc
- Passes when: The gRPC health service answers SERVING. The app must implement the standard gRPC health checking protocol.
All three probe types can use any mechanism. successThreshold defaults to 1 and must be 1 for liveness and startup probes; only readiness can require several passes in a row before a pod counts as ready again.
Framework note
Spring Boot Actuator
Spring Boot's Actuator provides separate liveness and readiness health groups at /actuator/health/liveness and /actuator/health/readiness. They are enabled automatically when the app detects it runs on Kubernetes, and can be switched on or off with management.endpoint.health.probes.enabled. If Actuator runs on a separate management port, a probe there can pass while the main port is stuck, so Spring Boot can also expose the groups on the main port as /livez and /readyz. Spring's own guidance matches this page: the liveness group should not depend on external systems, and adding one to readiness is a judgement call.
# application.properties
management.endpoint.health.probes.enabled=true
# also serve them on the main port as /livez and /readyz
management.endpoint.health.probes.add-additional-paths=true
# add your own check to readiness (never to liveness)
management.endpoint.health.group.readiness.include=readinessState,customCheckSource: Spring Boot reference: Actuator endpoints, Kubernetes probes ↗ · Report a problem
Your turn
Quiz: 3 questions
Which probe is this? During a database outage, every pod of a service is restarted again and again, although the app processes themselves are healthy.
A pod has shown Running, 0/1 READY and 0 restarts for ten minutes. Which probe is failing?
An app needs 45 seconds to start. Its only probe is liveness with periodSeconds 10, failureThreshold 3 and no initial delay. What happens?
Related errors
Questions
- What is the difference between a readiness probe and a liveness probe?
- A failing readiness probe removes the pod from the Service's endpoints, so it stops receiving traffic, and leaves the container running. A failing liveness probe makes the kubelet kill and restart the container.
- When do I need a startup probe?
- When the application can take longer to start than the liveness probe would tolerate. The startup probe holds liveness and readiness checks back until the app is up, so a slow start is not mistaken for a hang.
- Does a readiness probe restart the pod?
- No. A readiness failure only stops traffic to the pod. The container keeps running and starts receiving traffic again as soon as the probe passes.
Sources: Kubernetes docs: Liveness, readiness and startup probes ↗ · Kubernetes docs: Configure liveness, readiness and startup probes ↗ · Kubernetes docs: Pod lifecycle, container probes ↗ · Report a problem
Diagrams are our own simulations. All wording is original and paraphrases the linked Kubernetes documentation.
Practise: diagnose the cluster
Read the terminal, watch the animation, pick the error and the first fix.