Handbook Maintenance

Documentation on how to carry out maintenance tasks.

On a regular basis (currently monthly), an issue with a list of maintenance tasks is created and assigned to the relevant DRIs.

The issues, pipeline schedule, and all relevant code reside in the maintenance tasks project.

Note: The maintenance project uses the TRIAGE_BOT group access token. See access tokens for more on general management of tokens.

Content tasks

Content tasks are assigned to the content DRI. They should consider a designated backup from their team if needed.

Image compressing

  1. Ensure you have each image compression tool installed locally, pngquant and jpegoptim.

  2. Run the commands below in the root folder for the relevant project.

    find . -name '*.png' -size +100k -print0 | xargs -0 -P8 -L1 pngquant --ext .png --force --skip-if-larger

## Run the following a second time with the `.jpeg` extension
find . -name '*.jpg' -size +50k -print0 | xargs -0 -P8 -L1 jpegoptim -t -T5

  3. Create a MR with the compressed images.

Image use checking

  1. Download the unlinked image script if you do not already have it, ensuring it’s the latest version.
  2. Run the script in for each handbook repository: /path/to/unlinked_images.sh /path/to/handbook/repository.
  3. Make a MR to remove all unlinked files. If there are a lot, consider breaking it up into smaller MRs.
  4. Ask the person who added or recently edited the images to review the MR.

To check links, ensure you have lychee installed locally.

  1. Download the lychee configuration file if you do not already have it, ensuring it’s the latest version.
  2. Ensure each handbook repository is up to date. You will need to complete the task for each repository. The rest of the instructions refer to a single repository.
  3. In the root folder of the repository, run lychee -c ~tasks/lychee.toml --include-fragments **/*.md, replacing the configuration file location to wherever you have downloaded it to.
    • If the report comes back with very few links, decrease the max_redirects by 1, and run it again. Ideally, we get max_redirects down to 0. If you change it, remember to update the file in the maintenance project. If max_redirects is 0, then remove this note.
    • While --base is an option, we currently use hugolint for checking relative links.
  4. Edit the repository files to fix all links, as much as possible.
    • If a (sub)domain should be ignored, update the configuration file to exclude it.
    • Ignore files not part of the repository that you may have locally and that we do not maintain, such as those in the node_modules folder.
    • Consider leaving broken links on pages from a specific past timeframe, such as updates or reports for a specific quarter.
    • If it redirects, replace the link with the new URL.
    • If a page is under maintenance or unavailable at this time, but it is likely to be available again in the future, leave it as is.
    • If the site still exists, but the page doesn’t exist, try searching for a new version.
    • If the site no longer exists, remove the link and make a note (link no longer available).
    • Some sites (such as LinkedIn) may be rate limited (throws a 429), check these manually or leave as is.
  5. Optionally, run lychee again to verify there are no changes you missed.
  6. Once you have fixed the links, create an MR.
  7. If needed, create an issue with a report on broken links that you could not fix, and tag the maintainers (or post in a relevant Slack channel).

Each site has hugolint running as part of the pipeline to check for broken relative links.

For each site:

  1. Look for the latest hugolint job.
  2. Browse or download the artifacts.
  3. The missinglinks.txt lists the top 20 broken links, while the linkcheck file lists all of them.
  4. Fix at least the top 20 from the missinglinks. The number before the path indicates the number of occurrences.
    • Most of the time, there is a redirect in place, so visiting the “broken” path will take you to the new one. Otherwise, search the same handbook, then the other handbook for a similar page.
  5. Create a MR with the fixed links.

Check for configuration updates

Check for changes to the product documentation markdownlint and vale configuration changes.

Update the handbook projects as needed. Not all rules are the same as the product documentation.

The markdown guide explains a lot of the usage for markdownlint. Vale usage refers to product documentation as the reasons are the same.

Check for and fix errors

To run the linters locally, ensure that you have markdownlint and vale installed. You can reference the product documentation local testing information on how to install them. More information how they are used in the handbook can also be found in the development docs.

Fix trailing spaces
  1. Ensure you have the task markdownlint config file.
  2. Run markdownlint-cli2 --config tasks/task.markdownlint-cli2.jsonc **/*.md --fix, replacing the file path with where you have downloaded the configuration file.
  3. Commit the changes, and create an MR. You can combine the MR with other linter tasks.
Check and fix errors

The linter jobs do not run on main due to how slow they can be. Part of maintenance then is to double check that errors are not somehow being merged.

The instructions are for a single repository. Do the following for each repository:

  1. In the root folder, run markdownlint-cli2 "content/**/*.md" "assets/**/*.md" "layouts/**/*.md".
  2. In the root folder, run vale --no-wrap --minAlertLevel error --glob='{content,assets,layouts}/**/*.md' ..
  3. For each report, note the number of errors in the maintenance issue.
  4. Fix all errors, and create a MR.

Once completed, look back at the previous maintenance issues. If in the last 3 months, we consistently have errors, discuss with the handbook DRIs about adding linting to main.

If there are no errors in the last 3 months, we can likely safely remove this task from maintenance.

Redirects removal

  1. In each repository, look for the index.redirects file.
  2. Remove any redirects past the removal date as noted in a comment.
  3. Create a MR.

Backend tasks

Backend tasks are assigned to the technical DRI. They should consider a designated backup from the “Code Maintainer” group if needed.

Dependencies

The dependencies should be reviewed and bumped as appropriate.

docsy

Whenever the upstream Docsy theme releases a new version, open a new issue and select the Docsy-upgrade template. Follow the steps in the issue to upgrade the theme.

golang / hugo / nodejs

It’s important to keep the language versions up-to-date and in sync across the projects. When upgrading versions be sure to make those changes to docsy-gitlab, handbook, and internal-handbook.

Versions are defined in both the .tool-versions file and the .gitlab-ci.yml file. Make sure those versions stay in sync or you might see inconsistent behavior between local and CI builds.

Review changelogs for each tool to be aware of any potential breaking changes. Changelogs can be found:

When upgrading dependences:

  • Test the changes locally first by performing the upgrades to gitlab-docsy, handbook, and internal-handbook and running the builds locally. Manual testing locally will make it easy to spot any major issues.
  • If there are no concerns, next, release a new version of the docsy theme with the changes.
  • With the new version of the theme available, create new MRs in handbook and internal-handbook to apply the version changes and upgrade to the newly released versions of the docsy-gitlab. Use review apps for both MRs to test the dependency changes in a production-like environment.

markdownlint and vale

The linters shouldn’t need to be updated often, but may include fixes or new useful features.

Review the release log for each and determine if a version bump is appropriate. If so:

  1. Update your local linter to the appropriate version.
  2. Test the linters locally to ensure there are no errors.
  3. Bump the version in .tool-versions and .gitlab-ci.yml.
  4. Create MR.

Other tasks

Some other tasks that may be occasionally done, but are not part of monthly maintenance.

Delete old branches

One way to delete old branches is to use git to sort by date and delete a specified number of branches.

git fetch --all --prune
git branch -r --sort=authordate:iso8601 | head -n 10 | sed 's/  origin\///' | grep -v '^master$' | xargs git push origin -d
View page source - Edit this page - please contribute. Creative Commons License