Handbook Development

Development environment and processes for maintaining the handbook

Note: You can edit this page in the docsy-gitlab theme project.

The handbook site uses Hugo for static page generation from Markdown.

The handbook uses a base theme called Docsy. A custom theme override is located in the Docsy GitLab Theme project and automatically included in the handbook setup.

Support

For help and support with the development environment, please reach out in the public #handbook Slack channel.

If you encounter a problem or bug, please open an issue or MR in the respective handbook projects.

Edit the handbook in your browser

Follow the editing the handbook documentation.

Run the handbook locally for edits

Supported methods:

  1. Docker or compatible runtime (all software preinstalled in a container image)
  2. Source installation (experienced users, more software dependencies, can break)

Requirements

  • Command line: git, wget (for syncing data). Additional requirements for source installation, see below.
  • Docker or compatible runtime (for running the Hugo environment in a container)
    • On macOS: Docker Desktop, Rancher Desktop, etc.
    • On Linux: Docker engine, Podman, etc.

Clone the handbook Git repository

Cloning the repository allows you to manually edit the handbook locally. If you prefer to use the Web IDE, please continue reading the editing the handbook documentation.

We recommend using git to clone the repository and then editing the handbook with a text editor such as Visual Studio Code, Typora, Nova or Sublime to name a few.

Clone the repo with HTTPS or SSH.

Public handbook:

1
2
3
4
5

# HTTPS
git clone https://gitlab.com/gitlab-com/content-sites/handbook.git

# SSH
git clone git@gitlab.com:gitlab-com/content-sites/handbook.git

Internal handbook:

1
2
3
4
5

# HTTPS
git clone https://gitlab.com/gitlab-com/content-sites/internal-handbook.git

# SSH
git clone git@gitlab.com:gitlab-com/content-sites/internal-handbook.git

Docsy GitLab theme:

1
2
3
4
5

# HTTPS
git clone https://gitlab.com/gitlab-com/content-sites/docsy-gitlab.git

# SSH
git clone git@gitlab.com:gitlab-com/content-sites/docsy-gitlab.git

Set up the repository

After cloning the repository, sync the required data files from the data file location (currently the www-gitlab-com repository). Without this step, the handbook cannot be run locally.

Open a terminal, navigate into the cloned handbook repository path, and run the sync-data.sh script.

Example for the public handbook:

1
2
3

cd handbook

./scripts/sync-data.sh

Running Hugo

Hugo needs to be run to generate the static files from the handbook markdown content.

Running Hugo in Docker

You can use the Hugo container to start a locally running instance of the handbook, and verify how your changes look.

The following command starts the Hugo server, using the hugomods/hugo container image. The exts container image tag is important, as it provides the required SASS/CSS conversion tools.

1

docker run --rm -v $(pwd):$(pwd) -w $(pwd) --network host hugomods/hugo:exts hugo server

This will start the Hugo server listening on http://localhost:1313. If that doesn’t work, try http://127.0.0.1:1313. It may take a couple of minutes to load the first time.

You can also start a new container, and run the commands with Hugo manually.

1
2
3
4
5

docker run --rm -it -v $(pwd):$(pwd) -w $(pwd) --network host hugomods/hugo:exts sh

hugo server

ctrl+d # to quit

Running Hugo from source

Prerequisites

The handbook development environment requires the extended version of Hugo. The extended version is required for the Docsy GitLab theme as we use SASS for generating the CSS stylesheets for the site. This will also require the installation of Node/NPM.

  1. Install asdf. The handbook repositories provide a .tools-version configuration file for asdf.
  2. Install hugo using Homebrew (Note: It is important to install Hugo extended), by following their documentation.
  3. Install the asdf packages: NodeJS (SASS/CSS), Go (theme development), Vale (linting)
  4. Install Git, curl, wget for sync scripts.
  5. Install the GitLab CLI for linting jobs.
1
2
3
4
5

asdf install

brew install hugo git curl wget glab

npm install
Running the Hugo binary
1
2
3
4
5
6
7

./scripts/sync-data.sh

# Build the static website, default output into public/
hugo

# Start a local webserver, listening on localhost:1313
hugo serve

Parameters for Hugo

The handbook is huge, and by default, the hugo server command loads everything to memory.

The following options for the hugo command can be helpful for debugging or otherwise running locally:

  • --environment=production: generate a production build (asset minification, checksums, etc)
    • Note: hugo v0.x.x+extended version must be installed, use hugo version to check
    • Note: npm i must be run as a prerequisite to install postcss and autoprefixer dependencies
  • --renderToDisk: slower but requires less memory to run. Useful if you have <16GB allocated to docker machine
  • --verbose: enables verbose logging output
  • --templateMetrics and --templateMetricsHints: prints metrics related to how frequently templates are invoked and how much time is being spent evaluating them
  • --printMemoryUsage: periodically prints memory usage while the site is building

Build static files

To render the entire site to disk (and inspect the output in ${PWD}/public), purge the generated files first, and then run Hugo.

1
2
3
4
5

make clean

docker run --rm -v $(pwd):$(pwd) -w $(pwd) hugomods/hugo:exts hugo
# or
hugo

Permissions

For some pages to render you may need a personal access token. Generate one and export it as an environment variable prior to running hugo commands:

1
2

export HUGOxPARAMSxGITLAB_API_KEY="<token>"
hugo ...

If you need to make changes to the underlying theme you’ll need to make changes in the Gitlab-Docsy module.

Linting content

We use markdownlint-cli2 and Vale to enforce rules in handbook content.

Markdownlint

We use markdownlint-cli2 in our pipelines with a slightly customized set of rules. Before pushing any changes, you should run markdownlint-cli2 and fix any suggested changes to avoid pipeline failures.

To run markdownlint-cli2 using Docker, run:

1

docker run --rm -v $(pwd):$(pwd) -w $(pwd) davidanson/markdownlint-cli2 content/**/*.md

Vale

We use Vale to warn when some rules from the Handbook Markdown Guide. are broken. Vale is not run in pipelines.

To run Vale using Docker, run:

1

docker run --rm -v $(pwd):$(pwd) -w $(pwd) jdkato/vale content

Docsy GitLab Theme Development

Docsy is used as a base for the internal and public handbooks. Modifications and extensions have been added on top to ensure efficient handbook usage.

All Hugo partials and shortcodes should be added to this repository, instead of duplicating them locally into the handbook projects.

The Docsy GitLab theme is integrated into the public/internal handbook repositories using a Go module in go.mod.

Releases

Documentation changes to /docs, theme updates with partials, and more require a new theme module release.

  1. Navigate into Code > Tags and create a new Git tag.
  2. Bump the semantic version (bugfixes: minor version, features: feature version, breaking changes: major version).
  3. Collect the recently merged MR titles into the description.
  4. Publish the Git tag

The new Git tag release needs to be pulled into the public/internal handbook projects as updated dependency, following the next steps.

Dependencies

Updating dependencies

Note: Before updating dependencies, review the go.mod file in the target project for additional notes and requirements.

Always update dependencies individually to isolate potential bugs and regressions. Never pull latest but pin the depedencies to a specific released version.

1

hugo mod get server.com/project-name@version

Google Docsy

1

hugo mod get github.com/google/docsy@v0.9.1

Do not bump Docsy versions without testing them extensively in MR review apps.

GitLab Docsy Theme update in handbook projects

  1. Ensure that the latest release is tagged in the Docsy GitLab Theme.
  2. Bump the version in the handbook projects.
  3. Navigate into the public/internal handbook project.
  4. Update the dependency version to the desired release.
1

hugo mod get gitlab.com/gitlab-com/content-sites/docsy-gitlab@v0.3.15
  1. Create an MR and assign a handbook backend maintainer or codeowner for review.

Redirects

Edit layouts/index.redirects and add the source and target.

CI/CD Pipelines

The CI/CD pipelines do the following:

  1. Run linting scripts to check for content accuracy, codeowners, etc. Add an MR comment in case of linting errors.
  2. Use GitLab Pages for Review Apps to preview the changes in the same environment.
  3. Deploy to GitLab Pages when MRs are merged to main.

GitLab Pages Review Apps

The review apps are configured in the pages-review job in the .gitlab-ci.yml configuration file.

  1. Extends the Hugo build template
  2. Rules: Only run in MRs
  3. Environment: Use $CI_COMMIT_REF_SLUG as unique name
  4. Access job artifacts using $CI_JOB_ID in the URL
  5. Set the HUGO_ENVIRONMENT variable value to review.

Example for the public handbook:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

pages-review:
  extends: [.hugo]
  rules:
    - if: $CI_MERGE_REQUEST_IID
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    # There is no predefined CI variable that is formatted quite right
    # neither $CI_PAGES_URL nor $CI_JOB_URL have /-/ directly after the hostname
    url: https://gitlab-com.gitlab.io/-/content-sites/handbook/-/jobs/$CI_JOB_ID/artifacts/public/
    on_stop: stop-pages-review
  variables:
    HUGO_ENVIRONMENT: review
    RUN_TYPE: MR

The route map is configured in the .gitlab/route-map.yml.

Reviewer Roulette

The handbook projects use Reviewer Roulette feature of the danger-review CI/CD component.

In order to make Danger post message to MRs we need to setup a project token (api scope with Developer role) and a CI/CD variable called DANGER_GITLAB_API_TOKEN to contain this project token (Masked but not Protected).

For more details, follow the Danger documentation and inspect the MR adding the reviewer roulette to the public handbook.

Code quality report MR commenter

The CI/CD linting jobs generate code quality report artifacts for MR widget integration. If there are errors detected, a custom CI/CD job posts a Markdown table summary as MR comment, linking to helpful handbook editing resources.

Requires a project token (api scope with Developer role) and a CI/CD variable called MR_UPDATE_TOKEN to contain this project token (Masked but not Protected).

CI/CD maintenance tasks

  1. Test and bump the Hugo image versions, aligned with the config/_default/config.yml mimimum version.
  2. Test and bump the davidanson/markdownlint-cli2 Docker image tags in the markdownlint job. We are not using the latest tag to avoid production failures pulling different latest versions.

Architecture

The Markdown files get parsed by Hugo, and converted into static HTML files, including all assets (JS, CSS, images, etc.). The static files are uploaded to GitLab Pages.

Templates and Partials

Review the Hugo development documentation to learn more.

View page source - Edit this page - please contribute. Creative Commons License