Load Balancer Health Check: Settings That Matter
A load balancer health check decides which backends get traffic. Here are the four parameters, the shallow-versus-deep trap, and what to return.
A load balancer health check is a probe the balancer runs against each backend to decide whether that backend still gets traffic. Every platform implements the same idea with different field names and different defaults. The interesting decisions are not the field names. They are what the endpoint tests, and what happens when every backend fails at once.
What is a load balancer health check?
It is an automated request sent to each registered target on a fixed schedule, whose answer decides pool membership. A target that fails enough consecutive attempts is taken out of rotation. A target that passes enough consecutive attempts is put back in.
Two jobs hide inside that one mechanism, and a Reddit thread on the topic separates them better than most vendor docs. The first job is readiness, which asks whether a freshly started instance is ready for its first request. The second is liveness, which asks whether a long-running instance has since gone bad. The same endpoint usually serves both, and that is fine as long as you know it is doing two things.
The four parameters, under four names
Every implementation exposes the same quartet. How often to probe, how long to wait, how many failures remove a target, and how many successes restore it. Only the spelling changes.
| Concept | AWS target group | Google Cloud health check |
|---|---|---|
| Probe frequency | HealthCheckIntervalSeconds | checkIntervalSec |
| Per-probe patience | HealthCheckTimeoutSeconds | timeoutSec |
| Failures before removal | UnhealthyThresholdCount | unhealthyThreshold |
| Successes before return | HealthyThresholdCount | healthyThreshold |
| Path to request | HealthCheckPath | requestPath |
| Acceptable status codes | Matcher | implicit 200 |
Multiply interval by the unhealthy threshold and you get your worst-case detection time. An Application Load Balancer probes every 30 seconds by default as of September 2026. A dead target can therefore keep receiving requests for a minute or more before it leaves the pool. Google Cloud's health checks and Azure's load balancer probes expose the same arithmetic with their own defaults.
Set all four explicitly rather than inheriting them. We prefer a 10 second interval with an unhealthy threshold of 2 for stateless HTTP backends, which caps exposure near 20 seconds. Set the healthy threshold higher, so a flapping instance has to earn its way back.
aws elbv2 create-target-group \
--name api-tg --protocol HTTP --port 8080 --vpc-id vpc-0abc123 \
--health-check-path /healthz \
--health-check-interval-seconds 10 \
--health-check-timeout-seconds 3 \
--unhealthy-threshold-count 2 \
--healthy-threshold-count 4 \
--matcher HttpCode=200
Keep the timeout below the interval. When a probe can outlive the gap to the next one, you get overlapping requests against a backend that is already struggling.
Shallow or deep?
A shallow check proves the process is listening and can serve a trivial route. A deep check also verifies the database, the cache and whatever else the service depends on. Deep sounds obviously better, and it is the way most teams get a full outage from a partial one.
Here is the trap. If every target queries the same database and that database has a bad ten seconds, every target fails its load balancer health check simultaneously. The balancer dutifully removes all of them, and a degraded service becomes a dead one. Amazon's own Builders' Library article on implementing health checks describes this failure mode and the fail-open behaviour that large balancers use to escape it.
The practical rule is a split. The route the balancer probes should test only what is local to that instance. A separate, deeper route can test dependencies, and you point alerting at that one instead of pool membership.
| Check depth | Tests | Point it at |
|---|---|---|
| Shallow | Process is up, can allocate, can respond | The load balancer |
| Local | Disk, worker threads, in-process queues | The load balancer, carefully |
| Deep | Database, cache, downstream APIs | Alerting and dashboards, not rotation |
What your health endpoint should return
Return a plain 200 with a tiny body and no authentication. A probe that has to negotiate a login is a probe that fails for the wrong reasons, which is the most common question about this on Stack Overflow.
{ "status": "ok", "checks": { "disk": "ok", "workers": "ok" }, "version": "2026.09.18" }
Three details matter more than the shape. Exclude the route from access logs or it will dominate them. Exclude it from any rate limiter, since the probe hits it constantly by design. Include a build identifier, because it turns a rolling deploy into something you can watch rather than guess at.
Who checks the load balancer itself?
A healthy target group is a statement about the pool behind the listener. It says nothing about the listener, the certificate on it, the DNS record pointing at it, or the security group in front of it.
Each of those fails independently. A certificate expires and every target stays green. A DNS record points at a balancer that was replaced last quarter, and the console still shows five healthy targets. The pool is fine and the site is unreachable, which is exactly the class of incident the console cannot show you.
A check that starts outside the network closes that gap. It has to run from more than one place, since a routing fault often hits one continent and misses the others. Uptime Room probes the public hostname from six regions as of September 2026.
A restrictive security group will need the addresses those probes come from, and they sit with the region reference. Monitor fields are in the API reference. That external check reports whether users can reach the service, not which target dropped out, so keep the console for the second question. The same outside-in stance is argued at length in blackbox monitoring.
Common mistakes
Probing the homepage. It renders templates, queries the database and calls a third-party script. You have built a deep check by accident and pointed your rotation at it.
Matching only on a 200 with no body assertion. A framework's default error page returns 200 more often than anyone expects. Assert on a string from the real response.
Forgetting the grace period at boot. A target that needs 40 seconds to warm caches will fail its first probes and never join the pool. Every platform has a startup or initial-delay setting, so use it instead of loosening the threshold.
Testing the balancer only from inside the VPC. A probe on the private subnet tells you nothing about the public path. Prove the failover path separately with failover testing.
Get the four parameters set explicitly, keep the probed route shallow, and put the dependency checks somewhere that cannot empty your pool. A load balancer health check is a rotation decision rather than an alerting signal, and treating it as both is how a slow database becomes a full outage. Pair it with one external check on the hostname your customers type, which takes about two minutes to set up from the pricing page.
FAQ
It is an automated request the balancer sends to each registered backend on a fixed schedule. A backend that fails a set number of consecutive attempts is removed from rotation, and one that passes a set number is added back. The mechanism decides pool membership, which is a different job from alerting a human.
Expose a route like /healthz that needs no authentication, returns 200 with a small body, and tests only what is local to that instance. Keep database and cache checks on a separate deeper route. Exclude the health route from access logs and rate limiting, and include a build identifier so rolling deploys are visible.
The usual causes are a security group that does not allow the balancer to reach the target port, a path that requires authentication, a matcher expecting 200 while the app returns 302, or a target that needs longer to boot than the first probes allow. AWS publishes a troubleshooting runbook for failing Application Load Balancer checks.
Not the one the balancer probes. If every target shares a database and that database stalls, every target fails at once and the balancer removes the whole fleet. Test dependencies on a separate route and point alerting at that, so a degraded service does not become a dead one.
Worst-case detection is the interval multiplied by the unhealthy threshold, so pick both together. A 10 second interval with a threshold of two caps exposure near 20 seconds for stateless HTTP backends. Keep the per-probe timeout below the interval so probes never overlap against an already slow target.
Related reading
- HAProxy Health Check Config: Examples and Gotchas - An HAProxy health check decides which backends stay in rotation. Here are the directives that matter, sane defaults, and the failure it cannot see.
- Blackbox Monitoring: What the Outside View Sees - Blackbox monitoring tests a service from outside with no access to its internals, which is how you catch DNS, TLS and routing failures.