Somewhere in almost every codebase there is a route that returns 200 {"status": "ok"} from a static handler. It will return 200 while your database is unreachable, your queue is thirty thousand messages deep and every real request is timing out. Your availability dashboard will show a flat green line through the whole incident.
Why this happens
The endpoint was written for a load balancer, and for that purpose it is correct — a load balancer asks "should I send traffic to this instance?" and making that depend on a database would mean a brief database wobble pulls every instance out of rotation at once. The mistake is reusing it for monitoring, where the question is completely different.
Two endpoints, two questions
Liveness: cheap, dependency-free, answers "is this process alive?" This is what your load balancer polls. Readiness: verifies the things a real request needs are working. This is what your uptime monitoring polls, and it is allowed to fail.
GET /health/live -> 200, always, if the process is up
GET /health/ready -> 200 if dependencies are healthy, 503 if not
What belongs in the deep check
Database connectivity, with an actual round trip like SELECT 1, and both primary and replica if you have one. Critical downstream services, using a cached result rather than calling on every poll. Essential local resources. Background processing freshness — the age of the oldest unprocessed queue item is one of the most valuable numbers you can surface.
What does not belong
Non-critical dependencies. Anything slow — the check should complete in well under a second. Anything with side effects.
Returning something useful
{
"status": "degraded",
"version": "2.14.3",
"checks": {
"database": {"status": "ok", "latency_ms": 4},
"payment_provider": {"status": "degraded", "checked_ago_s": 22},
"queue_oldest_message_age_s": 14
}
}
The version field alone is worth including — an enormous proportion of incidents are answered by "which build is actually running?"
The test that proves it works
Once a quarter, in staging, break a dependency on purpose and look at your health endpoint and your monitoring. If your uptime check stayed green while a critical dependency was down, you have learned something important before it mattered.