Log Aggregation Working Group

Attributes

Business Goal

Increase the quality, value, and accessibility of our GitLab.com logs.

Intent is to:

1. Analyze and document the locations, retention, and variety of production logs as they exist today, including analysis of the data classification and access controls
2. Develop troubleshooting and investigation guides to make best use of GitLab.com logs
3. Perform gap analysis of log quality and completeness, and where improvements can be made to streamline investigations
4. Work with Infra and Development teams to integrate the proposed improvements into GitLab.com

Exit Criteria

1. Completed audit of existing log sources, storage locations, and means of accessing the aggregated logs along with a quality and accuracy evaluation of these logs
2. Creation and delivery of a standardized logging lifecycle providing:
   1. Contribution: The how, when, and why Developers should add logged events to their code
   2. Production: Ensuring each service as part of the GitLab package is writing logs in a consistent and configurable fashion
   3. Aggregation: The consistent, reliable, and configurable collection and shipping of logs to defined location(s)
   4. Accessibility: Tooling and functionality to provide timely access and manipulation of the log data
   5. Retention: Define, provide means to enforce, and verify the necessary retention policies on log data
3. Publish documentation and training material on the GitLab logging lifecycle for use by the GitLab.com service and self-managed GitLab

Roles and Responsibilities

View page source - Edit this page - please contribute.