Handbook Maintenance
On a regular basis (currently monthly), an issue with a list of maintenance tasks is created and assigned to the relevant DRIs.
The issues, pipeline schedule, and all relevant code reside in the maintenance tasks project.
Note: The maintenance project uses the TRIAGE_BOT
group access token.
See access tokens for more on general management of tokens.
Content tasks
Content tasks are assigned to the content DRI. They should consider a designated backup from their team if needed.
Image compressing
-
Ensure you have each image compression tool installed locally,
pngquant
andjpegoptim
. -
Run the commands below in the root folder for the relevant project.
find . -name '*.png' -size +100k -print0 | xargs -0 -P8 -L1 pngquant --ext .png --force --skip-if-larger ## Run the following a second time with the `.jpeg` extension find . -name '*.jpg' -size +50k -print0 | xargs -0 -P8 -L1 jpegoptim -t -T5
-
Create a MR with the compressed images.
Image use checking
- Download the unlinked image script if you do not already have it, ensuring it’s the latest version.
- Run the script in for each handbook repository:
/path/to/unlinked_images.sh /path/to/handbook/repository
. - Make a MR to remove all unlinked files. If there are a lot, consider breaking it up into smaller MRs.
- Ask the person who added or recently edited the images to review the MR.
Link checking (external)
To check links, ensure you have lychee installed locally.
- Download the lychee configuration file if you do not already have it, ensuring it’s the latest version.
- Ensure each handbook repository is up to date. You will need to complete the task for each repository. The rest of the instructions refer to a single repository.
- In the root folder of the repository, run
lychee -c ~tasks/lychee.toml --include-fragments **/*.md
, replacing the configuration file location to wherever you have downloaded it to.- If the report comes back with very few links, decrease the
max_redirects
by 1, and run it again. Ideally, we getmax_redirects
down to 0. If you change it, remember to update the file in the maintenance project. Ifmax_redirects
is 0, then remove this note. - While
--base
is an option, we currently usehugolint
for checking relative links.
- If the report comes back with very few links, decrease the
- Edit the repository files to fix all links, as much as possible.
- If a (sub)domain should be ignored, update the configuration file to exclude it.
- Ignore files not part of the repository that you may have locally and that we do not maintain, such as those in the
node_modules
folder. - Consider leaving broken links on pages from a specific past timeframe, such as updates or reports for a specific quarter.
- If it redirects, replace the link with the new URL.
- If a page is under maintenance or unavailable at this time, but it is likely to be available again in the future, leave it as is.
- If the site still exists, but the page doesn’t exist, try searching for a new version.
- If the site no longer exists, remove the link and make a note
(link no longer available)
. - Some sites (such as LinkedIn) may be rate limited (throws a 429), check these manually or leave as is.
- Optionally, run lychee again to verify there are no changes you missed.
- Once you have fixed the links, create an MR.
- If needed, create an issue with a report on broken links that you could not fix, and tag the maintainers (or post in a relevant Slack channel).
Link checking (internal)
Each site has hugolint
running as part of the pipeline to check for broken relative links.
For each site:
- Look for the latest
hugolint
job. - Browse or download the artifacts.
- The
missinglinks.txt
lists the top 20 broken links, while thelinkcheck
file lists all of them. - Fix at least the top 20 from the
missinglinks
. The number before the path indicates the number of occurrences.- Most of the time, there is a redirect in place, so visiting the “broken” path will take you to the new one. Otherwise, search the same handbook, then the other handbook for a similar page.
- Create a MR with the fixed links.
Linter related tasks
Check for configuration updates
Check for changes to the product documentation markdownlint and vale configuration changes.
Update the handbook projects as needed. Not all rules are the same as the product documentation.
The markdown guide explains a lot of the usage for markdownlint
.
Vale usage refers to product documentation as the reasons are the same.
Check for and fix errors
To run the linters locally, ensure that you have markdownlint
and vale
installed.
You can reference the product documentation local testing information on how to install them.
More information how they are used in the handbook can also be found in the development docs.
Fix trailing spaces
- Ensure you have the task markdownlint config file.
- Run
markdownlint-cli2 --config tasks/task.markdownlint-cli2.jsonc **/*.md --fix
, replacing the file path with where you have downloaded the configuration file. - Commit the changes, and create an MR. You can combine the MR with other linter tasks.
Check and fix errors
The linter jobs do not run on main
due to how slow they can be.
Part of maintenance then is to double check that errors are not somehow being merged.
The instructions are for a single repository. Do the following for each repository:
- In the root folder, run
markdownlint-cli2 "content/**/*.md" "assets/**/*.md" "layouts/**/*.md"
. - In the root folder, run
vale --no-wrap --minAlertLevel error --glob='{content,assets,layouts}/**/*.md' .
. - For each report, note the number of errors in the maintenance issue.
- Fix all errors, and create a MR.
Once completed, look back at the previous maintenance issues.
If in the last 3 months, we consistently have errors, discuss with the handbook DRIs about adding linting to main
.
If there are no errors in the last 3 months, we can likely safely remove this task from maintenance.
Redirects removal
- In each repository, look for the
index.redirects
file. - Remove any redirects past the removal date as noted in a comment.
- Create a MR.
Backend tasks
Backend tasks are assigned to the technical DRI. They should consider a designated backup from the “Code Maintainer” group if needed.
Dependencies
The dependencies should be reviewed and bumped as appropriate.
docsy
Whenever the upstream Docsy theme releases a new version, open a new issue and select the Docsy-upgrade template. Follow the steps in the issue to upgrade the theme.
golang / hugo / nodejs
It’s important to keep the language versions up-to-date and in sync across the projects. When upgrading versions be sure to make those changes to docsy-gitlab
, handbook
, and internal-handbook
.
Versions are defined in both the .tool-versions
file and the .gitlab-ci.yml
file. Make sure those versions stay in sync or you might see inconsistent behavior between local and CI builds.
Review changelogs for each tool to be aware of any potential breaking changes. Changelogs can be found:
When upgrading dependences:
- Test the changes locally first by performing the upgrades to
gitlab-docsy
,handbook
, andinternal-handbook
and running the builds locally. Manual testing locally will make it easy to spot any major issues. - If there are no concerns, next, release a new version of the docsy theme with the changes.
- With the new version of the theme available, create new MRs in
handbook
andinternal-handbook
to apply the version changes and upgrade to the newly released versions of thedocsy-gitlab
. Use review apps for both MRs to test the dependency changes in a production-like environment.
markdownlint and vale
The linters shouldn’t need to be updated often, but may include fixes or new useful features.
- markdownlint changelog, refer to markdownlint project tags for dates
- Vale releases
Review the release log for each and determine if a version bump is appropriate. If so:
- Update your local linter to the appropriate version.
- Test the linters locally to ensure there are no errors.
- Bump the version in
.tool-versions
and.gitlab-ci.yml
. - Create MR.
Other tasks
Some other tasks that may be occasionally done, but are not part of monthly maintenance.
Delete old branches
One way to delete old branches is to use git
to sort by date and delete a specified number of branches.
git fetch --all --prune
git branch -r --sort=authordate:iso8601 | head -n 10 | sed 's/ origin\///' | grep -v '^master$' | xargs git push origin -d