Publishing a modular feature's OpenAPI specification

Overview

This page lists the steps to put a modular feature’s OpenAPI specification in the public API reference at https://api.gitlab.com/rest/. The rules are in ADR 008.

Out of scope:

• The lint ruleset. See the ruleset documentation.
• Breaking-change detection. See epic 432.
• Contract testing. See epic 434.

Before you start

To list a document as public, its repository must be public. The public reference commits every listed document to the public gitlab-org/api-docs repository.

You need two approvals from outside your team:

1. An API Platform code owner approves your change to gitlab-org/api-docs (step 5). A Maintainer merges it.
2. An infra-mgmt approver from gitlab-com/gl-infra, gitlab-org/delivery or gitlab-org/production-engineering approves your change to gitlab-com/gl-infra/infra-mgmt (step 6). Fleet Management owns the file.

Do step 5 before step 7. An unlisted spec refreshes nothing. Do step 6 before step 7. Without the token, the job fails on your default branch. Until steps 6 and 7 are done, the daily api-docs refresh at 05:00 UTC picks up your changes within a day.

The worked example is Artifact Registry. It is only an example. Its repository is gitlab-org/ops/artifact-registry.

Step 1. Write one document per API surface

Put each document in api/openapi/ in your component repository. No document spans assemblies. A component repository can hold several assemblies, so their documents share api/openapi/. Name each document after the assembly that serves it, the API surface it describes, and its major version:

• <assembly>-v<major>.yaml for the assembly’s main API surface.
• <assembly>-<surface>-v<major>.yaml for each further surface, such as <assembly>-internal-v1.yaml.

<assembly> is the assembly’s name in bench.yaml. You choose how you produce each document.

Step 2. Give the public document an absolute servers URL

An absolute URL makes “Try it” on api.gitlab.com call your service, not api.gitlab.com. The host is a variable because self-managed installations run on their own domain. The relative / entry keeps same-origin callers working. This is the block from Artifact Registry’s public document:

# api/openapi/<assembly>-v1.yaml
servers:
  - url: https://{hostname}
    description: The Artifact Registry service.
    variables:
      hostname:
        default: artifact-registry.gitlab.com
        description: The Artifact Registry host serving this API.
  - url: /
    description: The artifact-registry service root.

Step 3. Add an apis entry per document

bench scaffold creates api/openapi/redocly.yaml. It extends the shared ruleset at a full release tag. Add one apis entry for each document in api/openapi/. This is the file after adding Artifact Registry’s two documents:

# api/openapi/redocly.yaml
# The shared ruleset is a floor: add rules here, never switch off or weaken one.
# Docs: https://gitlab.com/gitlab-com/gl-infra/common-ci-tasks/-/blob/main/openapi-ruleset.md
extends:
  - https://gitlab.com/gitlab-com/gl-infra/common-ci-tasks/-/raw/v5.37.0/openapi-ruleset.yaml
apis:
  management@v1:
    root: v1.yaml
  gitlab@v1:
    root: gitlab-v1.yaml

If you move the ruleset pin later, use a full release tag, never a minor tag. See the ruleset documentation. The lint CI component is not shipped yet.

Step 4. Mark internal operations

Mark an internal operation with x-gitlab-internal: true on the operation object:

paths:
  /example:
    get:
      x-gitlab-internal: true
      # ... rest of the operation

Interim rule: this marker hides nothing yet, because api-docs publishes whole documents. Until the api-docs filter ships, keep internal operations in a separate document, and do not list that document in specs.js. Artifact Registry does this with gitlab-v1.yaml. See Internal operations and documents.

Step 5. List each public document

Open an MR in gitlab-org/api-docs that adds an entry to src/config/specs.js. Add one entry for each public document. Do not list an internal document, or a document your component keeps out of the reference. An API Platform code owner approves each entry. This is Artifact Registry’s entry:

  {
    title: 'GitLab Artifact Central API',
    slug: 'artifact-central',
    file: 'artifact_registry_v1.json',
    published: true,
    // Not translated, so the Japanese reference serves it in English.
    upstream: {
      en: gitlabFile('gitlab-org/ops/artifact-registry', 'api/openapi/v1.yaml'),
    },
  },

This is the generic form:

  {
    title: '<display name>',
    slug: '<component name>',
    file: '<service>_v1.json',
    published: true,
    upstream: {
      en: gitlabFile('<project path>', 'api/openapi/v1.yaml'),
    },
  },

The fields:

1. title is the display name.
2. slug is a permanent URL. Use one path segment named after the component, never the protocol. Scalar prefixes every link hash with it, such as #gitlab/tag/projects. Changing it breaks every shared link.
3. file is the served filename, at https://api.gitlab.com/rest/<file>.
4. published is required on every entry. true renders the document. false fetches, lints and keeps it fresh but does not render it. false is not privacy. The document is still committed to the public repository and deployed to /rest/<file>.
5. upstream maps a locale to a URL. en is required. ja is optional. gitlabFile(project, path, ref = 'main') builds the URL. Pass the third argument if your default branch is not main.
6. Do not copy default: true. Only the gitlab entry has it.

The specs.js file says “never list a spec whose upstream is private”. api-docs sends a JOB-TOKEN header when it fetches, so a private upstream could be fetched and then published.

Step 6. Create the trigger token

Open an MR in gitlab-com/gl-infra/infra-mgmt that adds a token and a CI/CD variable to environments/gitlab-com/project_api_docs_trigger_tokens.tf. There is one token for each consuming project. One approval from the infra-mgmt Infrastructure rule is required. Fleet Management is the code owner. Atlantis applies the change after merge. The token reaches your project as the masked, protected API_DOCS_TRIGGER_TOKEN variable.

This is Artifact Registry’s pair:

resource "gitlab_pipeline_trigger" "api-docs-trigger-artifact-registry" {
  project     = module.project_api_docs.id
  description = "Triggered by gitlab-org/ops/artifact-registry to refresh its OpenAPI spec (IaC managed)"
}

resource "gitlab_project_variable" "api-docs-trigger-artifact-registry" {
  project = 79865069 # gitlab-org/ops/artifact-registry, not managed in this environment

  key   = "API_DOCS_TRIGGER_TOKEN"
  value = gitlab_pipeline_trigger.api-docs-trigger-artifact-registry.token

  protected         = true
  masked            = true
  environment_scope = "*"
}

This is the generic copy:

resource "gitlab_pipeline_trigger" "api-docs-trigger-<service>" {
  project     = module.project_api_docs.id
  description = "Triggered by <project path> to refresh its OpenAPI spec (IaC managed)"
}

resource "gitlab_project_variable" "api-docs-trigger-<service>" {
  project = <project ID> # <project path>, not managed in this environment

  key   = "API_DOCS_TRIGGER_TOKEN"
  value = gitlab_pipeline_trigger.api-docs-trigger-<service>.token

  protected         = true
  masked            = true
  environment_scope = "*"
}

Step 7. Include api-docs-refresh

Add the api-docs-refresh component to your .gitlab-ci.yml. It runs on a push to your default branch when a path in spec_paths changed. It runs on gitlab.com only, and not on */security/* mirrors. It uses the trigger token, so it does not matter who merged. Your default branch must be protected.

include:
  - component: $CI_SERVER_FQDN/gitlab-com/gl-infra/common-ci-tasks/api-docs-refresh@v5.38
    inputs:
      spec_paths:
        - api/openapi/v1.yaml

Use the same version as your other common-ci-tasks components. They share includes, and a mismatch silently changes which one wins. stage defaults to validate. Set stage only if you need a different one. Artifact Registry sets publish. List public documents only in spec_paths. Artifact Registry lists v1.yaml and not gitlab-v1.yaml.

Do not copy the trigger-api-docs-pipeline job from gitlab-org/gitlab. It runs as the merging user and needs Maintainer on api-docs. See gitlab-org/gitlab#627956.

Step 8. Check the result

A green api-docs-refresh job only means a refresh was requested. Open the api-docs pipeline URL that the job prints. Then open https://api.gitlab.com/rest/#<slug>. For Artifact Registry, that is https://api.gitlab.com/rest/#artifact-central.

Internal operations and documents

Internal does not mean secret. The upstream repository is public, so anyone can read the operations you mark. The marker only keeps them out of the public API reference.

An internal operation is marked with x-gitlab-internal: true (step 4). Once the api-docs filter ships, api-docs strips marked operations from a public document before it publishes it.

A whole internal document is not listed in specs.js and not in spec_paths. Artifact Registry’s gitlab-v1.yaml is an example.

Opting out

Do not list the component’s documents in specs.js. Skip steps 5 to 8 for them. With no public document, there is nothing for spec_paths, so no token and no include. An unlisted document is never fetched or rendered.

Planned automation

These items are tracked elsewhere. They do not change the steps above yet.

1. Registration (step 5): api-docs#43 tracks opening the MR for you. Approval is still needed.
2. The trigger in the scaffold (step 7): common-template-copier#18.
3. Stub generation in bench build: common-template-copier#19.
4. Existing services: common-template-copier#21 and bench#48.
5. The lint CI component: gitlab-org/gitlab#624044, #633601 and #633603.

View page source - Edit this page - please contribute.