Handbook Development
Note: You can edit this page in the docsy-gitlab theme project.
The handbook site uses Hugo for static page generation from Markdown.
The handbook uses a base theme called Docsy. A custom theme override is located in the Docsy GitLab Theme project and automatically included in the handbook setup.
Support
For help and support with the development environment, please reach out in the public #handbook Slack channel.
If you encounter a problem or bug, please open an issue or MR in the respective handbook projects.
Edit the handbook in your browser
Follow the editing the handbook documentation.
Run the handbook locally for edits
Supported methods:
- Docker or compatible runtime (all software preinstalled in a container image)
- Source installation (experienced users, more software dependencies, can break)
Requirements
- Command line: git, wget (for syncing data). Additional requirements for source installation, see below.
- Docker or compatible runtime (for running the Hugo environment in a container)
Clone the handbook Git repository
Cloning the repository allows you to manually edit the handbook locally. If you prefer to use the Web IDE, please continue reading the editing the handbook documentation.
We recommend using git to clone the repository and then editing the handbook with a text editor such as Visual Studio Code, Typora, Nova or Sublime to name a few.
Clone the repo with HTTPS or SSH.
Public handbook:
|
|
Internal handbook:
|
|
Docsy GitLab theme:
|
|
Set up the repository
After cloning the repository, sync the required data files from the
data file location (currently the www-gitlab-com
repository). Without this
step, the handbook cannot be run locally.
Open a terminal, navigate into the cloned handbook repository path, and run the sync-data.sh
script.
Example for the public handbook:
|
|
Running Hugo
Hugo needs to be run to generate the static files from the handbook markdown content.
Running Hugo in Docker
You can use the Hugo container to start a locally running instance of the handbook, and verify how your changes look.
The following command starts the Hugo server, using the hugomods/hugo
container image.
The exts
container image tag is important, as it provides the required SASS/CSS conversion tools.
|
|
This will start the Hugo server listening on http://localhost:1313
. If that doesn’t work, try http://127.0.0.1:1313
. It may take a couple of minutes to load the first time.
You can also start a new container, and run the commands with Hugo manually.
|
|
Running Hugo from source
Prerequisites
The handbook development environment requires the extended version of Hugo. The extended version is required for the Docsy GitLab theme as we use SASS for generating the CSS stylesheets for the site. This will also require the installation of Node/NPM.
- Install asdf. The handbook repositories provide a
.tools-version
configuration file forasdf
. - Install
hugo
using Homebrew (Note: It is important to install Hugo extended), by following their documentation. - Install the asdf packages: NodeJS (SASS/CSS), Go (theme development), Vale (linting)
- Install Git, curl, wget for sync scripts.
- Install the GitLab CLI for linting jobs.
|
|
Running the Hugo binary
|
|
Parameters for Hugo
The handbook is huge, and by default, the hugo server
command loads everything to memory.
The following options for the hugo
command can be helpful for debugging or otherwise running locally:
--environment=production
: generate a production build (asset minification, checksums, etc)- Note:
hugo v0.x.x+extended
version must be installed, usehugo version
to check - Note:
npm i
must be run as a prerequisite to installpostcss
andautoprefixer
dependencies
- Note:
--renderToDisk
: slower but requires less memory to run. Useful if you have <16GB allocated to docker machine--verbose
: enables verbose logging output--templateMetrics
and--templateMetricsHints
: prints metrics related to how frequently templates are invoked and how much time is being spent evaluating them--printMemoryUsage
: periodically prints memory usage while the site is building
Build static files
To render the entire site to disk (and inspect the output in ${PWD}/public
),
purge the generated files first, and then run Hugo.
|
|
Permissions
For some pages to render you may need a personal access token.
Generate one and export it as an environment variable prior to running hugo
commands:
|
|
If you need to make changes to the underlying theme you’ll need to make changes in the Gitlab-Docsy module.
Linting content
We use markdownlint-cli2
and Vale to enforce
rules in handbook content.
Markdownlint
We use markdownlint-cli2
in our pipelines with a slightly customized set of rules. Before pushing any changes, you
should run markdownlint-cli2
and fix any suggested changes to avoid pipeline failures.
To run markdownlint-cli2
using Docker, run:
|
|
Vale
We use Vale to warn when some rules from the Handbook Markdown Guide. are broken. Vale is not run in pipelines.
To run Vale using Docker, run:
|
|
Docsy GitLab Theme Development
Docsy is used as a base for the internal and public handbooks. Modifications and extensions have been added on top to ensure efficient handbook usage.
All Hugo partials and shortcodes should be added to this repository, instead of duplicating them locally into the handbook projects.
The Docsy GitLab theme is integrated into the public/internal handbook repositories using a Go module in go.mod.
Releases
Documentation changes to /docs
, theme updates with partials, and more require a new theme module release.
- Navigate into
Code > Tags
and create a new Git tag. - Bump the semantic version (bugfixes: minor version, features: feature version, breaking changes: major version).
- Collect the recently merged MR titles into the description.
- Publish the Git tag
The new Git tag release needs to be pulled into the public/internal handbook projects as updated dependency, following the next steps.
Dependencies
Updating dependencies
Note: Before updating dependencies, review the
go.mod
file in the target project for additional notes and requirements.
Always update dependencies individually to isolate potential bugs and regressions. Never pull latest
but pin the depedencies to a specific released version.
|
|
Google Docsy
|
|
Do not bump Docsy versions without testing them extensively in MR review apps.
GitLab Docsy Theme update in handbook projects
- Ensure that the latest release is tagged in the Docsy GitLab Theme.
- Bump the version in the handbook projects.
- Navigate into the public/internal handbook project.
- Update the dependency version to the desired release.
|
|
- Create an MR and assign a handbook backend maintainer or codeowner for review.
Redirects
Edit layouts/index.redirects
and add the source and target.
CI/CD Pipelines
The CI/CD pipelines do the following:
- Run linting scripts to check for content accuracy, codeowners, etc. Add an MR comment in case of linting errors.
- Use GitLab Pages for Review Apps to preview the changes in the same environment.
- Deploy to GitLab Pages when MRs are merged to main.
GitLab Pages Review Apps
The review apps are configured in the pages-review
job in the .gitlab-ci.yml
configuration file.
- Extends the Hugo build template
- Rules: Only run in MRs
- Environment: Use
$CI_COMMIT_REF_SLUG
as unique name - Access job artifacts using
$CI_JOB_ID
in the URL - Set the
HUGO_ENVIRONMENT
variable value toreview
.
Example for the public handbook:
|
|
The route map is configured in the .gitlab/route-map.yml
.
Reviewer Roulette
The handbook projects use Reviewer Roulette
feature of the danger-review
CI/CD component.
In order to make Danger post message to MRs we need to setup a project token (api
scope with Developer
role) and
a CI/CD variable called DANGER_GITLAB_API_TOKEN
to contain this project token (Masked
but not Protected
).
For more details, follow the Danger documentation and inspect the MR adding the reviewer roulette to the public handbook.
Code quality report MR commenter
The CI/CD linting jobs generate code quality report artifacts for MR widget integration. If there are errors detected, a custom CI/CD job posts a Markdown table summary as MR comment, linking to helpful handbook editing resources.
Requires a project token (api
scope with Developer
role) and a CI/CD variable called MR_UPDATE_TOKEN
to contain this project
token (Masked
but not Protected
).
CI/CD maintenance tasks
- Test and bump the Hugo image versions, aligned with the
config/_default/config.yml
mimimum version. - Test and bump the davidanson/markdownlint-cli2
Docker image tags in the
markdownlint
job. We are not using thelatest
tag to avoid production failures pulling different latest versions.
Architecture
The Markdown files get parsed by Hugo, and converted into static HTML files, including all assets (JS, CSS, images, etc.). The static files are uploaded to GitLab Pages.
Templates and Partials
Review the Hugo development documentation to learn more.