Health check
GitLab provides liveness and readiness probes to indicate service health and reachability to required services. These probes report on the status of the database connection, Redis connection, and access to the file system. These endpoints can be provided to schedulers like Kubernetes to hold traffic until the system is ready or restart the container as needed.
Health check endpoints are normally used for load balancers and other Kubernetes scheduling systems that need to determine service availability before redirecting traffic.
You should not use these endpoints to determine effective uptime on large Kubernetes deployments. Doing so can show false negatives when pods are removed by autoscaling, node failure, or for other normal and otherwise non-disruptive operational needs.
To determine uptime on large Kubernetes deployments, look at traffic
to the UI. This is properly balanced and scheduled, and therefore is
a better indicator of effective uptime. You can also monitor the sign-in
page /users/sign_in endpoint.
On GitLab.com, tools such as Pingdom and Apdex measurements are used to determine uptime.
IP allowlist
To access monitoring resources, the requesting client IP needs to be included in the allowlist. For details, see how to add IPs to the allowlist for the monitoring endpoints.
Using the endpoints locally
With default allowlist settings, the probes can be accessed from localhost using the following URLs:
GET http://localhost/-/health
GET http://localhost/-/readiness
GET http://localhost/-/liveness
Health
Checks whether the application server is running.
It does not verify the database or other services
are running. This endpoint circumvents Rails Controllers
and is implemented as additional middleware BasicHealthCheck
very early into the request processing lifecycle.
GET /-/health
Example request:
curl "https://gitlab.example.com/-/health"
Example response:
GitLab OK
Readiness
The readiness probe checks whether the GitLab instance is ready to accept traffic via Rails Controllers. The check by default does validate only instance-checks.
If the all=1 parameter is specified, the check also validates
the dependent services (Database, Redis, Gitaly etc.)
and gives a status for each.
GET /-/readiness
GET /-/readiness?all=1
Example request:
curl "https://gitlab.example.com/-/readiness"
Example response:
{
"master_check":[{
"status":"failed",
"message": "unexpected Master check result: false"
}],
...
}
On failure, the endpoint returns a 503 HTTP status code.
This check is being exempt from Rack Attack.
Liveness
Checks whether the application server is running. This probe is used to know if Rails Controllers are not deadlocked due to a multi-threading.
GET /-/liveness
Example request:
curl "https://gitlab.example.com/-/liveness"
Example response:
On success, the endpoint returns a 200 HTTP status code, and a response like below.
{
"status": "ok"
}
On failure, the endpoint returns a 503 HTTP status code.
This check is being exempt from Rack Attack.
Sidekiq
Learn how to configure the Sidekiq health checks.