Skip to content

Load balancing & consistent hashing · 2 of 6

Step 1 · The pattern

Spread the load, check the servers

The load balancer shares requests across three servers. Every second or so it sends each a health check and only routes to the ones that pass.

  • request
  • health check
  • OK

In short

LB health checks: how it works and how it breaks

A load balancer spreads requests across several servers and uses a health check to decide which ones may receive traffic. It calls an endpoint on each server every few seconds and removes any that fail. So the balancer's view of the world is only as good as that endpoint. The commonest mistake is a check that proves the process is running and nothing more. A server whose database connection is broken answers 200 OK on /health and returns errors on every real request. The balancer keeps sending it a full share of traffic, and that share of users fails. A better check exercises what real requests depend on. That has its own danger: if every server shares one dependency and it blips, every server fails together and the balancer has nowhere to send anyone. So balancers should fail open when all targets look unhealthy.

In Spring Boot

Readiness and liveness groups with Actuator

application.properties
management.endpoint.health.probes.enabled=true
# what "ready for traffic" depends on: here, the database as well
management.endpoint.health.group.readiness.include=readinessState,db

# point the load balancer or Kubernetes at:
#   /actuator/health/readiness   may this instance receive traffic?
#   /actuator/health/liveness    should this instance be restarted?

Actuator exposes two separate health groups. The readiness group answers the balancer's question, and here it also includes the database check, so an instance that cannot reach its database stops receiving traffic.

Gotcha: By default Spring Boot adds no other checks to either group, so an instance with a broken database still reports ready. Adding one is a choice: when the database is down, every instance leaves rotation together. Keep external systems out of liveness, or one outage restarts the whole fleet.

How this comes up in interviews

"A server behind your load balancer is up but has lost its database connection. Walk me through what users see and how you would design the health check."

Then they ask

  1. Your deep check queries the database, and the database has a short blip. What does the balancer do, and what should it do?
  2. A server sits near the check's timeout and keeps passing and failing. How do you stop it moving traffic back and forth?
  3. Should the same check decide whether an orchestrator restarts the process? Why or why not?

A strong answer covers

  • The balancer only knows what the health endpoint tells it. A check that proves the process runs will pass on a server that fails every real request.
  • A deeper check does a small piece of real work, such as a trivial database query, so a broken dependency takes the server out of rotation.
  • A dependency shared by every server fails every check together, so the balancer should fail open when all targets look unhealthy.
  • Require several failures before removal and several passes before return, and watch real responses as well as probes.
  • Keep the restart check shallow and local to the process, and let dependency checks affect routing only.

Points to cover, not a script. Say them in your own words and tie them to a system you have worked on.

On one page: the track cheat sheet.

Last updated: Engineering write-up

Diagrams are our own simulations: the timings, rates and counts in them are illustrative, not measurements. Captions are original wording.