Handbook Development

The handbook site uses Hugo for static page generation from Markdown.

The handbook uses a base theme called Docsy. A custom theme override is located in the Docsy GitLab Theme project and automatically included in the handbook setup.

Both handbooks use the same shared theme and development documentation. Each has its own content repository and builds its site with Hugo.

flowchart TD
    Docsy["Docsy upstream theme"] --> Theme["docsy-gitlab: shared theme and docs"]
    Theme --> Public["handbook: public content"]
    Theme --> Internal["internal-handbook: internal content"]
    Public -->|Hugo build and deploy| PublicSite["handbook.gitlab.com"]
    Internal -->|Hugo build and deploy| InternalSite["internal.gitlab.com"]

Support

For help and support with the development environment, please reach out in the public #handbook Slack channel.

If you encounter a problem or bug, please open an issue or MR in the respective handbook projects.

Edit the handbook in your browser

Follow the editing the handbook documentation.

Run the handbook locally for edits

You can edit a local checkout and use GitLab CI to check your changes without installing Hugo or Docker. A local preview is optional.

Requirements

• Install Git and use an editor you are comfortable with.
• For a container preview, follow Running the Handbook in Docker.
• For a native preview, follow Running the Handbook Locally.

For VS Code as editor, use the GitLab for VS Code extension and follow its setup guide to sign in to GitLab.com and connect your project. This lets you work with issues, merge requests, and pipeline status from your editor.

Clone the handbook Git repository

Choose the repository for the content you want to change: public handbook, internal handbook, or Docsy GitLab for shared development documentation and theme changes. Sign in to GitLab.com and confirm you can open that project. If you cannot access the internal handbook, ask in #handbook Slack.

The commands below use SSH. First, follow GitLab’s SSH setup to add a key to your account and verify the connection. If SSH is already configured, reuse it. You can also copy the HTTPS clone URL from the project’s Code menu and follow the HTTPS cloning instructions.

Clone only the repository you need:

Public handbook:

git clone git@gitlab.com:gitlab-com/content-sites/handbook.git

Internal handbook:

git clone git@gitlab.com:gitlab-com/content-sites/internal-handbook.git

Docsy GitLab theme:

git clone git@gitlab.com:gitlab-com/content-sites/docsy-gitlab.git

Set up the repository

Once you have cloned the repository, create a branch and start editing. You can use CI/CD to check your changes, or set up a local preview if you want to see them as you work.

Create a branch

Open the cloned folder in your editor, then create a branch for your changes. For example, from your terminal after cloning the public handbook:

cd handbook
git switch -c docs/my-handbook-update

You can now edit, commit, and push changes to an MR for CI feedback. No data sync or build tools are needed for editing files alone. See the handbook editing guide for the contribution and review steps.

Running Hugo

A local preview lets you check how your edits look in the handbook before opening a merge request. Choose one setup:

• Docker: follow the Docker setup, then run docker compose up. Dependencies and data synchronization are handled inside the container.
• Native tools: follow Running Hugo locally to complete setup, then run make view. It installs dependencies and synchronizes data automatically. The guide also explains how to run each command individually.

Linting content

CI/CD checks your changes when you push to a merge request. Open a failed job to find the affected files and errors, fix them, and push again. You can also run Markdown and writing checks locally before pushing.

Errors fail the relevant job. Warnings and suggestions are available in the job logs and code quality report.

Run the commands below from the repository root. Replace path/to/page.md with a file you edited. For Docker commands, keep your container app running.

markdownlint

Refer to the markdown guide for style guide information, and guidance on rules.

We use markdownlint-cli2 in our pipelines with a slightly customized set of rules. Before pushing any changes, you can run markdownlint-cli2 and fix any suggested changes to avoid pipeline failures.

To run markdownlint-cli2 using Docker, run:

docker run --rm -v "$PWD:$PWD" -w "$PWD" davidanson/markdownlint-cli2 path/to/page.md

If you have markdownlint-cli2 installed locally, run:

markdownlint-cli2 path/to/page.md

To automatically fix supported formatting errors, add --fix, then review the diff:

markdownlint-cli2 --fix path/to/page.md

Vale

We use Vale to verify some of the rules in the Handbook Markdown Guide.

Vale errors will fail the pipeline and are included in the MR comment. Warnings are included in the job log, code quality, and MR diff.

To run Vale using Docker, run:

docker run --rm -v "$PWD:$PWD" -w "$PWD" jdkato/vale --no-wrap --minAlertLevel error path/to/page.md

If you have Vale installed locally, run the same check without Docker:

vale --no-wrap --minAlertLevel error path/to/page.md

Handbook lint

Open the Handbook lint job log and fix the reported file or ownership errors. The detailed checks below explain what the job looks for.

Hugo lint

We use Hugo lint to check for the validity of internal links. It supports relative linking to the markdown file, and the production path.

Each job also outputs the top 30 incorrect links to assist in fixing them.

For local usage and configuration, see the Hugo lint README.

Steps to use nested branches

Advanced development

Use these options to debug Hugo, inspect generated files, or work on the theme.

Parameters for Hugo

The handbook is large. Hugo serves generated files from disk by default.

The following options for the hugo command can be helpful for debugging or otherwise running locally:

• --environment=production: generate a production build (asset minification, checksums, etc)
  • Note: hugo v0.x.x+extended version must be installed, use hugo version to check
  • Note: npm ci must be run as a prerequisite to install postcss and autoprefixer dependencies
• --renderToMemory: serves generated pages from memory, which can be faster but uses more RAM
• --logLevel debug: enables debug logging output
• --templateMetrics and --templateMetricsHints: prints metrics related to how frequently templates are invoked and how much time is being spent evaluating them
• --printMemoryUsage: periodically prints memory usage while the site is building

Build static files

To render the entire site to disk (and inspect the output in ${PWD}/public), use the Makefile after completing the local setup, or build in a container.

make build
# Or build in Docker:
make docker-build

Permissions

For some pages to render you may need a personal access token. Generate one and export it as an environment variable prior to running hugo commands:

export HUGOxPARAMSxGITLAB_API_KEY="<token>"
make view

If you need to make changes to the underlying theme you’ll need to make changes in the Gitlab-Docsy module.

Architecture Information

Please see the handbook architecture page.

Navigation and Menus

For the public Handbook, the top level menu (top navigation bar) uses Hugo’s menu settings to define menu entries. Edits to menu items can be made in the default /config/_default/config.yaml file in the main: menu parameters in the handbook repo. These config settings take precedence over docsy config settings.

Docsy builds upon Hugo’s menu system and offers additional features, such as the ability to add top level menu items directly from a page’s frontmatter. For example, the Job Families section landing page was added to the main menu by adding main: menu parameters in the frontmatter.

Docsy GitLab Theme Development

Docsy is used as a base for the internal and public handbooks. Modifications and extensions have been added on top to ensure efficient handbook usage.

All Hugo partials and shortcodes should be added to this repository, instead of duplicating them locally into the handbook projects.

The Docsy GitLab theme is integrated into the public/internal handbook repositories using a Go module in go.mod.

Local Development

To develop on the Docsy GitLab theme locally build the handbook (or internal-handbook) project with the updated local copy of the docsy-gitlab theme following the steps below:

1. Add the following line to the bottom of the go.mod file in the handbook project:

   replace gitlab.com/gitlab-com/content-sites/docsy-gitlab => /YOUR/LOCAL/PATH/TO/docsy-gitlab
   

2. Run the Hugo server:

   npm ci
   make view
   

3. Load the site locally:

   open http://localhost:1313/
   

Any changes made to files in local docsy-gitlab project will be rendered in the local instance.

Testing changes

When making changes that may have a user impact (not content only changes):

1. Test the changes locally first by performing the upgrades to gitlab-docsy, handbook, and internal-handbook and running the builds locally. Manual testing locally will make it easy to spot any major issues.
2. If there are no concerns, create a MR in the Docsy project with the changes, and have a maintainer review. Ensure that only the relevant changes you’re testing are in the MR.
   • Note: At the moment, there is no review app for the docsy project. When relevant, include local testing notes in the MR.
3. With the new version of the theme available, create new MRs in handbook and internal-handbook with the changes and upgrade to the theme if needed. Use review apps for both MRs to test the dependency changes in a production-like environment.
4. Apply the changes to the internal handbook first, and ensure changes are okay.
5. Next, apply the changes to the public handbook.
6. If there are any issues, revert the changes to the handbook projects.

Cross-project tests

In some cases, a page or shortcode may pull data directly from the www-gitlab-com project. (Example). In these cases, you can test the change by exporting a variable with the branch name of a www-gitlab-com MR before running hugo locally.

Example:

1. You make a change to ux_department.yml.
2. You push your change in a git branch named ux-dept-yml-update and create an MR.
3. The www-gitlab-com MR review app can’t show your change, because the data is rendered in the handbook project.
4. Before running the hugo server locally, execute export PUBLIC_HANDBOOK_REF=ux-dept-yml-update
5. Run the server locally: make view
6. Test your changes - the version of the files from your branch should now be used.

Releases

Changes made to docsy-gitlab will be made available to the handbook and internal-handbook after the version is bumped and deployed to each project. This includes all theme updates with partials, shortcodes, and any changes to /docs. Much of this process has been automated, but a few manual steps are still required. To release a docsy-gitlab change follow these steps:

1. After the merge request is merged, a pipeline on the main branch will kick off. After the pipeline succeeds, the release job can be triggered manually. For example: https://gitlab.com/gitlab-com/content-sites/docsy-gitlab/-/pipelines/1304430830 By default, the job increments the patch version. To choose a minor or major release, select the release job name, set the release_version input to a value such as v0.4.0, then select Run job. See job inputs. The value must be a stable vMAJOR.MINOR.PATCH tag, newer than all existing stable tags and not already published. Leave the input empty for the default patch increment. The job passes this input to the release script as RELEASE_VERSION. Use a pipeline created after this input was added; older jobs retain their original configuration.
2. The release job will do three things:
   1. Create a release in the docsy-gitlab project at the pipeline commit. Example: https://gitlab.com/gitlab-com/content-sites/docsy-gitlab/-/releases/v0.3.25
   2. Create a merge request to handbook. Example: https://gitlab.com/gitlab-com/content-sites/handbook/-/merge_requests/6086
   3. Create a merge request to internal-handbook. Example: https://gitlab.com/gitlab-com/content-sites/internal-handbook/-/merge_requests/4771
3. Assign a handbook backend maintainer or codeowner to each MR for review.
4. Once the merge request to handbook and internal-handbook have been merged, the theme changes will be deployed to the sites.

The release job stops on a failed command. If the release was already created before a downstream update failed, inspect the release and existing branches before retrying; published versions are not overwritten. Git authentication is supplied temporarily to clone and push commands, and downstream remotes use plain HTTPS URLs.

For more information on the release process, see the release script.

Dependencies

Updating dependencies

Note: Before updating dependencies, review the go.mod file in the target project for additional notes and requirements.

Always update dependencies individually to isolate potential bugs and regressions. Never pull latest but pin the dependencies to a specific released version.

hugo mod get server.com/project-name@version

Google Docsy

hugo mod get github.com/google/docsy@v0.9.1

Do not bump Docsy versions without testing them extensively in MR review apps.

Redirects

You can add redirects to the layouts/index.redirects file in the relevant repository. Refer to the GitLab Pages redirect documentation for how these are formatted.

Please add your redirect in chronological order with a removal date of:

• 3 months for within a single handbook, from one relative path to another relative path,
• 3-6 months when linking from one handbook to another (such as linking from handbook.gitlab.com to internal.gitlab.com), or
• 3-12 months for external links, meaning not part of the handbook.gitlab.com or internal.gitlab.com domains.

If a redirect needs to be kept permanently for legal and compliance reasons, please add the redirect to the designated section.

CI/CD Pipelines

Handbook & Internal Handbook

The CI/CD pipelines for the handbook and internal-handbook projects are roughly the same, and should remain that way.

build stage

The build stage performs a few linter and security jobs (handbooklint, hugolint, markdownlint, and secret_detection), and a build job which builds the whole site and saves it as CI artifacts.

deploy stage

The deploy stage uses the artifacts generated in the build stage to deploy the production site for changes to the main branch or review apps which can be deployed for MRs.

notify stage

The notify stage has two jobs that are triggered on specific events:

1. post_comment_on_failure posts a comment to the MR when the markdownlint or handbooklint jobs fail.
2. notify_slack_on_build_failures post a message to the #handbook-escalation Slack channel when there is a build failure on the main branch. This type of failure would indicate a broken CI pipeline.

$BUILD_AND_TEST_ONLY mode

The pipelines can be triggered to run in $BUILD_AND_TEST_ONLY mode which means they will run all the test and build jobs, but none of the deploy or notify jobs. A pipeline can be triggered in $BUILD_AND_TEST_ONLY mode by setting the $BUILD_AND_TEST_ONLY CI variable to true. This is used primarily by the downstream pipelines triggered from the www-gitlab-com project.

$GITLAB_TOKEN

Tools or scripts that need to interact with the repositories to perform handbook tasks should use the GITLAB_TOKEN CI variable. This is group level access token called Docsy GitLab Release with a developer role and has api and write_repository scopes. This token will expire after a year and will need to be refreshed. Follow the steps below to refresh the token:

1. Create a new developer group level access token in <https://gitlab.com/groups/gitlab-com/content-sites/-/settings/access_tokens with api> and write_repository scopes. Give it the name GITLAB_TOKEN_YYYY_MM_DD (replace the YYYY_MM_DD portion with the current date).
2. Update the value of GITLAB_TOKEN in docsy-gitlab, handbook, and internal-handbook with the new token value.
3. Create a new release of the Docsy GitLab Theme and ensure the pipeline works correctly.
4. Delete the old token.

For more information, see the general management of access tokens section.

GitLab Pages Deployment

The deploy and review apps are configured in the pages job in the .gitlab-ci.yml configuration file.

• Review app deploys can be run for merge requests by running the manual pages job, or by adding the ~"deploy-review-app-always" label to the MR.

• Production deploys are only run from the main branch.

• Conditional CI variables are used to configure the environment for review apps using GitLab Pages Multiple Deployments.

  variables:
    PAGES_PREFIX: 'mr$CI_MERGE_REQUEST_IID'
    ENVIRONMENT: 'mr$CI_MERGE_REQUEST_IID'
    HUGO_BASEURL: "${CI_PAGES_URL}/${PAGES_PREFIX}"
  

The route map is configured in the .gitlab/route-map.yml.

Docsy GitLab

The CI/CD pipeline for docsy-gitlab is responsible for building, testing, and releasing the Docsy GitLab theme used by both handbook and internal-handbook.

test stage

The test stage performs the markdownlint, and secret_detection jobs.

build stage

The build stage performs runs the hugo command to ensure the theme builds successfully.

release stage

The release stage performs one manual job to create a new release of the theme and merge requests to both handbook and internal-handbook. See the Releases section for more details.

www-gitlab-com

The CI/CD pipeline for www-gitlab-com is responsible for triggering downstream builds of both handbook and internal-handbook in $BUILD_AND_TEST_ONLY mode. This ensures that any changes to dependent data files in www-gitlab-com will build successfully for both handbook and internal-handbook.

Danger review and reviewer roulette

Right now the handbook projects doesn’t use the Reviewer Roulette feature of the danger-review CI/CD component. See issue.

We use the architecture plugin to provide guidance on how to review MRs that contain changes to architecture evolution design documents.

In order to make Danger post message to MRs we need to setup a project token (api scope with Developer role) and a CI/CD variable called DANGER_GITLAB_API_TOKEN to contain this project token (Masked but not Protected). For more information, see the general management of access tokens section.

For more details, follow the Danger documentation and inspect the MR adding the reviewer roulette to the public handbook.

Code quality report MR commenter

The CI/CD linting jobs generate code quality report artifacts for MR widget integration. If there are errors detected, a custom CI/CD job posts a Markdown table summary as MR comment, linking to helpful handbook editing resources.

The parse-codequality-report.sh script in the handbook and internal-handbook generates the code quality report posted in the MR. Changes to these reports need to be applied to both projects.

The code quality report relies on the GITLAB_TOKEN to perform the reporting actions on any MRs.

CI/CD maintenance tasks

See maintenance tasks and agent skills for maintenance.

Triage bot

The triage-bot project is meant to help triage issues in the handbook, internal-handbook, and docsy-theme projects. It has two main goals:

1. Ensure issues are triaged.
2. Close stale issues and merge requests.

Please see the .triage-policies.yml for rules. You can find more about how to create policies in the GitLab triage gem project readme.

The triage bot is scheduled to run 3 times a week. See the project’s pipeline schedule for details.

Triage uses the TRIAGE_BOT access token to perform tasks. This must have the Developer role to take actions on merge requests. For more information, see the general management of access tokens section.

View page source - Edit this page - please contribute.