• Adding new endpoints
• Authentication
• Git Authentication
  • Known consumers
• LFS Authentication
  • Known consumers
• Authorized Keys Check
  • Known consumers
• Get user for user ID or key
  • Known consumers
• Instance information
  • Known consumers
• Get new 2FA recovery codes using an SSH key
  • Known consumers
• Get new personal access-token
  • Known consumers
• Incrementing counter on pre-receive
• PostReceive
• Kubernetes agent endpoints
  • Kubernetes agent information
  • Kubernetes agent project information
  • Kubernetes agent usage metrics
  • Kubernetes agent alert metrics
• Subscriptions
  • Creating a subscription
  • Updating a subscription
  • Retrieving a subscription
  • Known consumers
• CI minute provisioning
  • Creating an additional pack
• Upcoming reconciliations
  • Updating upcoming_reconciliations
  • Known consumers

Internal API

The internal API is used by different GitLab components, it can not be used by other consumers. This documentation is intended for people working on the GitLab codebase.

This documentation does not yet include the internal API used by GitLab Pages.

Adding new endpoints

API endpoints should be externally accessible by default, with proper authentication and authorization. Before adding a new internal endpoint, consider if the API would potentially be useful to the wider GitLab community and can be made externally accessible.

One reason we might favor internal API endpoints sometimes is when using such an endpoint requires internal data that external actors can not have. For example, in the internal Pages API we might use a secret token that identifies a request as internal or sign a request with a public key that is not available to a wider community.

Another reason to separate something into an internal API is when request to such API endpoint should never go through an edge (public) load balancer. This way we can configure different rate limiting rules and policies around how the endpoint is being accessed, because we know that only internal requests can be made to that endpoint going through an internal load balancer.

Authentication

These methods are all authenticated using a shared secret. This secret is stored in a file at the path configured in config/gitlab.yml by default this is in the root of the rails app named .gitlab_shell_secret

To authenticate using that token, clients read the contents of that file, and include the token Base64 encoded in a secret_token parameter or in the Gitlab-Shared-Secret header.

Git Authentication

This is called by Gitaly and GitLab Shell to check access to a repository.

• When called from GitLab Shell: No changes are passed, and the internal API replies with the information needed to pass the request on to Gitaly.
• When called from Gitaly in a pre-receive hook: The changes are passed and validated to determine if the push is allowed.

Calls are limited to 50 seconds each.

POST /internal/allowed

Example request:

curl --request POST --header "Gitlab-Shared-Secret: <Base64 encoded token>" \
     --data "key_id=11&project=gnuwget/wget2&action=git-upload-pack&protocol=ssh" \
     "http://localhost:3001/api/v4/internal/allowed"

Example response:

{
  "status": true,
  "gl_repository": "project-3",
  "gl_project_path": "gnuwget/wget2",
  "gl_id": "user-1",
  "gl_username": "root",
  "git_config_options": [],
  "gitaly": {
    "repository": {
      "storage_name": "default",
      "relative_path": "@hashed/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.git",
      "git_object_directory": "",
      "git_alternate_object_directories": [],
      "gl_repository": "project-3",
      "gl_project_path": "gnuwget/wget2"
    },
    "address": "unix:/Users/bvl/repos/gitlab/gitaly.socket",
    "token": null
  },
  "gl_console_messages": []
}

Known consumers

• Gitaly
• GitLab Shell

LFS Authentication

This is the endpoint that gets called from GitLab Shell to provide information for LFS clients when the repository is accessed over SSH.

Example request:

curl --request POST --header "Gitlab-Shared-Secret: <Base64 encoded token>" \
     --data "key_id=11&project=gnuwget/wget2" "http://localhost:3001/api/v4/internal/lfs_authenticate"

{
  "username": "root",
  "lfs_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJkYXRhIjp7ImFjdG9yIjoicm9vdCJ9LCJqdGkiOiIyYWJhZDcxZC0xNDFlLTQ2NGUtOTZlMi1mODllYWRiMGVmZTYiLCJpYXQiOjE1NzAxMTc2NzYsIm5iZiI6MTU3MDExNzY3MSwiZXhwIjoxNTcwMTE5NDc2fQ.g7atlBw1QMY7QEBVPE0LZ8ZlKtaRzaMRmNn41r2YITM",
  "repository_http_path": "http://localhost:3001/gnuwget/wget2.git",
  "expires_in": 1800
}

Known consumers

• GitLab Shell

Authorized Keys Check

This endpoint is called by the GitLab Shell authorized keys check. Which is called by OpenSSH for fast SSH key lookup.

GET /internal/authorized_keys

Example request:

curl --request GET --header "Gitlab-Shared-Secret: <Base64 encoded secret>" "http://localhost:3001/api/v4/internal/authorized_keys?key=<key as passed by OpenSSH>"

Example response:

{
  "id": 11,
  "title": "admin@example.com",
  "key": "ssh-rsa ...",
  "created_at": "2019-06-27T15:29:02.219Z"
}

Known consumers

• GitLab Shell

Get user for user ID or key

This endpoint is used when a user performs ssh git@gitlab.com. It discovers the user associated with an SSH key.

GET /internal/discover

Example request:

curl --request GET --header "Gitlab-Shared-Secret: <Base64 encoded secret>" "http://localhost:3001/api/v4/internal/discover?key_id=7"

Example response:

{
  "id": 7,
  "name": "Dede Eichmann",
  "username": "rubi"
}

Known consumers

• GitLab Shell

Instance information

This gets some generic information about the instance. This is used by Geo nodes to get information about each other.

GET /internal/check

Example request:

curl --request GET --header "Gitlab-Shared-Secret: <Base64 encoded secret>" "http://localhost:3001/api/v4/internal/check"

Example response:

{
  "api_version": "v4",
  "gitlab_version": "12.3.0-pre",
  "gitlab_rev": "d69c988e6a6",
  "redis": true
}

Known consumers

• GitLab Geo
• GitLab Shell’s bin/check
• Gitaly

Get new 2FA recovery codes using an SSH key

This is called from GitLab Shell and allows users to get new 2FA recovery codes based on their SSH key.

GET /internal/two_factor_recovery_codes

Example request:

curl --request POST --header "Gitlab-Shared-Secret: <Base64 encoded secret>" \
     --data "key_id=7" "http://localhost:3001/api/v4/internal/two_factor_recovery_codes"

Example response:

{
  "success": true,
  "recovery_codes": [
    "d93ee7037944afd5",
    "19d7b84862de93dd",
    "1e8c52169195bf71",
    "be50444dddb7ca84",
    "26048c77d161d5b7",
    "482d5c03d1628c47",
    "d2c695e309ce7679",
    "dfb4748afc4f12a7",
    "0e5f53d1399d7979",
    "af04d5622153b020"
  ]
}

Known consumers

• GitLab Shell

Get new personal access-token

This is called from GitLab Shell and allows users to generate a new personal access token.

POST /internal/personal_access_token

Example request:

curl --request POST --header "Gitlab-Shared-Secret: <Base64 encoded secret>" \
     --data "user_id=29&name=mytokenname&scopes[]=read_user&scopes[]=read_repository&expires_at=2020-07-24" \
     "http://localhost:3001/api/v4/internal/personal_access_token"

Example response:

{
  "success": true,
  "token": "Hf_79B288hRv_3-TSD1R",
  "scopes": ["read_user","read_repository"],
  "expires_at": "2020-07-24"
}

Known consumers

• GitLab Shell

Incrementing counter on pre-receive

This is called from the Gitaly hooks increasing the reference counter for a push that might be accepted.

POST /internal/pre_receive

Example request:

curl --request POST --header "Gitlab-Shared-Secret: <Base64 encoded secret>" \
     --data "gl_repository=project-7" "http://localhost:3001/api/v4/internal/pre_receive"

Example response:

{
  "reference_counter_increased": true
}

PostReceive

Called from Gitaly after a receiving a push. This triggers the PostReceive-worker in Sidekiq, processes the passed push options and builds the response including messages that need to be displayed to the user.

POST /internal/post_receive

Example Request:

curl --request POST --header "Gitlab-Shared-Secret: <Base64 encoded secret>" \
     --data "gl_repository=project-7" --data "identifier=user-1" \
     --data "changes=0000000000000000000000000000000000000000 fd9e76b9136bdd9fe217061b497745792fe5a5ee gh-pages\n" \
     "http://localhost:3001/api/v4/internal/post_receive"

Example response:

{
  "messages": [
    {
      "message": "Hello from post-receive",
      "type": "alert"
    }
  ],
  "reference_counter_decreased": true
}

Kubernetes agent endpoints

The following endpoints are used by the GitLab Kubernetes Agent Server (kas) for various purposes.

These endpoints are all authenticated using JWT. The JWT secret is stored in a file specified in config/gitlab.yml. By default, the location is in the root of the GitLab Rails app in a file called .gitlab_kas_secret.

Kubernetes agent information

Called from GitLab Kubernetes Agent Server (kas) to retrieve agent information for the given agent token. This returns the Gitaly connection information for the agent’s project in order for kas to fetch and update the agent’s configuration.

GET /internal/kubernetes/agent_info

Example Request:

curl --request GET --header "Gitlab-Kas-Api-Request: <JWT token>" \
     --header "Authorization: Bearer <agent token>" "http://localhost:3000/api/v4/internal/kubernetes/agent_info"

Kubernetes agent project information

Called from GitLab Kubernetes Agent Server (kas) to retrieve project information for the given agent token. This returns the Gitaly connection for the requested project. GitLab kas uses this to configure the agent to fetch Kubernetes resources from the project repository to sync.

Only public projects are supported. For private projects, the ability for the agent to be authorized is not yet implemented.

GET /internal/kubernetes/project_info

Example Request:

curl --request GET --header "Gitlab-Kas-Api-Request: <JWT token>" \
     --header "Authorization: Bearer <agent token>" "http://localhost:3000/api/v4/internal/kubernetes/project_info?id=7"

Kubernetes agent usage metrics

Called from GitLab Kubernetes Agent Server (kas) to increase the usage metric counters.

POST /internal/kubernetes/usage_metrics

Example Request:

curl --request POST --header "Gitlab-Kas-Api-Request: <JWT token>" --header "Content-Type: application/json" \
     --data '{"gitops_sync_count":1}' "http://localhost:3000/api/v4/internal/kubernetes/usage_metrics"

Kubernetes agent alert metrics

Called from GitLab Kubernetes Agent Server (KAS) to save alerts derived from Cilium on Kubernetes Cluster.

POST internal/kubernetes/modules/cilium_alert

Example Request:

curl --request POST --header "Gitlab-Kas-Api-Request: <JWT token>" \
     --header "Authorization: Bearer <agent token>" --header "Content-Type: application/json" \
     --data '"{\"alert\":{\"title\":\"minimal\",\"message\":\"network problem\",\"evalMatches\":[{\"value\":1,\"metric\":\"Count\",\"tags\":{}}]}}"' \
     "http://localhost:3000/api/v4/internal/kubernetes/modules/cilium_alert"

Subscriptions

The subscriptions endpoint is used by CustomersDot (customers.gitlab.com) in order to apply subscriptions including trials, and add-on purchases, for personal namespaces or top-level groups within GitLab.com.

Creating a subscription

Use a POST to create a subscription.

POST /namespaces/:id/gitlab_subscription

Example request:

curl --request POST --header "TOKEN: <admin_access_token>" "https://gitlab.com/api/v4/namespaces/1234/gitlab_subscription?start_date="2020-07-15"&plan="silver"&seats=10"

Example response:

{
  "plan": {
    "code":"silver",
    "name":"Silver",
    "trial":false,
    "auto_renew":null,
    "upgradable":false
  },
  "usage": {
    "seats_in_subscription":10,
    "seats_in_use":1,
    "max_seats_used":0,
    "seats_owed":0
  },
  "billing": {
    "subscription_start_date":"2020-07-15",
    "subscription_end_date":null,
    "trial_ends_on":null
  }
}

Updating a subscription

Use a PUT command to update an existing subscription.

PUT /namespaces/:id/gitlab_subscription

Example request:

curl --request PUT --header "TOKEN: <admin_access_token>" "https://gitlab.com/api/v4/namespaces/1234/gitlab_subscription?max_seats_used=0"

Example response:

{
  "plan": {
    "code":"silver",
    "name":"Silver",
    "trial":false,
    "auto_renew":null,
    "upgradable":false
  },
  "usage": {
    "seats_in_subscription":80,
    "seats_in_use":82,
    "max_seats_used":0,
    "seats_owed":2
  },
  "billing": {
    "subscription_start_date":"2020-07-15",
    "subscription_end_date":"2021-07-15",
    "trial_ends_on":null
  }
}

Retrieving a subscription

Use a GET command to view an existing subscription.

GET /namespaces/:id/gitlab_subscription

Example request:

curl --header "TOKEN: <admin_access_token>" "https://gitlab.com/api/v4/namespaces/1234/gitlab_subscription"

Example response:

{
  "plan": {
    "code":"silver",
    "name":"Silver",
    "trial":false,
    "auto_renew":null,
    "upgradable":false
  },
  "usage": {
    "seats_in_subscription":80,
    "seats_in_use":82,
    "max_seats_used":82,
    "seats_owed":2
  },
  "billing": {
    "subscription_start_date":"2020-07-15",
    "subscription_end_date":"2021-07-15",
    "trial_ends_on":null
  }
}

Known consumers

• CustomersDot

CI minute provisioning

The CI Minute endpoint is used by CustomersDot (customers.gitlab.com) to apply additional packs of CI minutes, for personal namespaces or top-level groups within GitLab.com.

Creating an additional pack

Use a POST to create an additional pack.

POST /namespaces/:id/minutes

Example request:

curl --request POST \
  --url http://localhost:3000/api/v4/namespaces/123/minutes \
  --header 'Content-Type: application/json' \
  --header 'PRIVATE-TOKEN: <admin access token>' \
  --data '{
    "number_of_minutes": 10000,
    "expires_at": "2022-01-01",
    "purchase_xid": "46952fe69bebc1a4de10b2b4ff439d0c" }'

Example response:

{
  "namespace_id": 123,
  "expires_at": "2022-01-01",
  "number_of_minutes": 10000,
  "purchase_xid": "46952fe69bebc1a4de10b2b4ff439d0c"
}

Upcoming reconciliations

The upcoming_reconciliations endpoint is used by CustomersDot (customers.gitlab.com) to update upcoming reconciliations for namespaces.

Updating upcoming_reconciliations

Use a PUT command to update upcoming_reconciliations.

PUT /internal/upcoming_reconciliations

Each array element contains:

Example request:

curl --request PUT --header "PRIVATE-TOKEN: <admin_access_token>" --header "Content-Type: application/json" \
     --data '{"upcoming_reconciliations": [{"namespace_id": 127, "next_reconciliation_date": "13 Jun 2021", "display_alert_from": "06 Jun 2021"}, {"namespace_id": 129, "next_reconciliation_date": "12 Jun 2021", "display_alert_from": "05 Jun 2021"}]}' \
     "https://gitlab.com/api/v4/internal/upcoming_reconciliations"

Example response:

200

Known consumers

• CustomersDot