Observation Creation Procedure

This procedure details the creation process for observations.

Purpose

The Observation Management Program at GitLab is used to identify, track, remediate and provide a risk ratings of identified findings, exceptions or deficiencies for any Tier 3 information system risks that are identified as an output of compliance operations or other mechanisms by team members, such as self-identification of a system specific risk.

This procedure details the creation process for observations.

Introduction to Observation Management at GitLab

Scope

Tier 3 risks or observations identified at the information system or business process levels

Roles and Responsibilities

Observation Source Responsible Team
Security control testing activities (CCM) Security Compliance
Third Party Risk Management (TPRM) activities Security Risk
Customer Assurance Activities (CAA) Field Security Team
External audit activities Security Compliance
Third party application scanning (BitSight) Field Security Team
Gap Assessment Activities Security Compliance
Ad-hoc observations Security Assurance

Procedure

Lifecycle Overview

graph TD;
  A[Identified] --> B[Assigned];
  B --> C[Remediation in progress];
  B --> D[Ignored or Invalid];
  C --> F[Resolved];

Identifying Observations

Observations can be identified through the following channels:

  1. Security control testing activities (CCM)
  2. Third Party Risk Management (TPRM) activities
  3. Customer Assurance activities (CAA)
  4. External audit activities
  5. Third party application scanning (BitSight)
  6. Gap assessment activities
  7. Ad-hoc issues

Assigning Observations

The observation identifier is responsible for opening an observation in the GitLab Observation Project. The observation identifier fills out all necessary observation information, remediation recommendations and submits the observation to the Remediation Owner for validation. The Observation Manager is responsible for managing the observation through the observation lifecycle. This includes linking the observation to the associated control in Hyperproof, validating the observation with the Remediation Owner, tracking all remediation progress and updating the GitLab issue with current information and status updates. Each observation has both a GitLab Issue (for Remediation Owners) and a Hyperproof Issue (for Observation Managers). Each observation will be assigned a risk rating, which should drive the priority of remediation. Observation Managers are responsible for updating the Hyperproof Issue status once the observation has been confirmed remediated/closed/ignored/invalid.

See the associated runbooks below for detailed instructions on how to open and assign a new observation based on observation type:

Drafting Observation Description Guidance

The Observation description should include the who, what, when, where, why, and how related to the observation. As a review step, if you knew nothing about this observation could you understand the finding, how it was identified, and the effect it has on objectives? Consider leveraging the 4C’s model:

  1. Condition - current state
  2. Criteria - desired state based on policy, requirement, control, regulation, etc.
  3. Cause - root cause of the observation
  4. Consequence - actual or potential effect on objectives/assets

Risk Ratings

Tier 3 information system risk ratings are based off the formula below.

Risk Rating = Likelihood x Impact

Determining Likelihood

Step 1: Define likelihood based on the below criteria and likelihood table

At GitLab, observations will be rated based on the likelihood the observation has recurring and/or the frequency that the control has seen observations. The criteria used to assess this likelihood can be found in the Likelihood Table below. There are two different definitions for each likelihood rating level, “control observation” and “information system risk (tier 3)”. These definitions delineate the factors that may be considered when thinking of an individual observation from a control perspective and an information system risk perspective. Each should be considered when determining the likelihood of an observation occurring.

  • Control Observation: This criteria is utilized to rate observations identified as an output of control testing (e.g. where control testing performed internally by Security Compliance has failed). The assumption of the Likelihood Table is to consider observations individually rather than in aggregate (i.e if 2 similar observations occur against a single test of a sample of 25, the failure rate is 8% and would be scored a 3. The control does not need to be tested multiple times in the current year or prior 9 months with an observation each time to meet the requirement for a score of 3).
  • Information System Risk (Tier 3): This criteria is utilized to score the likelihood of an information system being exploited (e.g. insufficient encryption mechanisms for the storage of data within [System Name] result in the unintentional exposure/leakage of this information to the public).
Likelihood Table
Qualitative Score Description
1 Control Observation: The observation noted is considered to be a one off occurrence for the control as a result of extenuating circumstances. It is unlikely to occur again once remediated.

Information System Risk: Theoretically impossible and/or requires significant technical expertise for the risk to be exploited.
2 Control Observation: The observation was identified as a result of management's oversight on the control and may potentially occur again. This is the only observation associated with the control in the current fiscal year or prior 9 months, whichever is longer.

Information System Risk: Even with technical expertise, it is somewhat difficult to exploit the risk.
3 Control Observation: The control has had multiple observations in the current fiscal year or prior 9 months, whichever is longer.

Information System Risk: Minimal expertise is required to exploit the risk.
4 Control Observation: The control has observations that have persisted and continue to occur year to year AND/OR the observation noted is associated with the design of the control.

Information System Risk: The risk can be easily exploited and does not require any technical expertise.
Determining Impact

Step 2: Define impact based on the below criteria and impact table

In addition to applying a qualitative scoring factor for likelihood, all observations need to be evaluated for the impact they would have on GitLab at the organization level and/or the compliance impact (if applicable). The criteria and qualitative scores for assessing the impact of an observation can be found in the Impact Scoring Table below. The highest rating in any field is the final impact score of the observation so as to approach observations in a more conservative manner (i.e if all fields are rated at a value of 2 except Remediation Effort which is scored a 3, the final impact score would be a 3).

Important Note: Team members who are leveraging the impact scoring criteria below may judgmentally select the impact factors most relevant to them. Internal Audit and Security Compliance utilize all columns when scoring observations identified as part of controls testing because there may be specific impacts to external compliance audit requirements as a result of these findings. Any information system risk identified outside of control testing may utilize the columns that are most relevant.

Impact Scoring Table
Qualitative Score External Audit Impact Remediation Effort Financial Impact Legal & Regulatory Impact Stakeholder/ ICOFR (Internal Controls Over Financial Reporting)  Impact
1 The observation would not lead to an adverse audit opinion. The observation was related to extenuating circumstances and requires simple reinforcement of policy/process, no additional management oversight required. Observation has potential financial impact which results in loss / misstatement of up to $25K. The observation would not lead to major action by a regulator. The observation has minimal impact on all stakeholders (internal and external)
2 The observation would likely not lead to an adverse audit opinion because it is an isolated occurrence. The observation has remediation effort that requires oversight/support at the management level. Observation has potential financial impact which results in loss / misstatement between $25K to $250K. The observation could lead to minor regulatory action. The observation impacts internal stakeholders and or could lead to financial misstatements, if not addressed on time.
3 The observation is likely to result in an adverse audit opinion if a full sample for remediation testing cannot be provided. The observation has remediation effort that requires oversight/support at the director level. Observation has potential financial impact which results in loss / misstatement between $250K to $500K. The observation could lead to an investigation or regulatory action. The observation impacts internal and external stakeholders. It requires attention of Executives and Board.
4 The observation will result in an adverse audit opinion. The observation has remediation effort that requires oversight/support at the Executive level. Observation has potential financial impact which results in loss / misstatement above $ 500K The observation could directly result in major regulatory action against GitLab. The observation impacts internal and external stakeholders. It requires attention the of Executives and Board and could impact management assertion in 10Q / 10-K.
Determining Risk Rating

Step 3: Determine risk rating based on likelihood and impact scores defined in steps 1 and 2

In order to arrive at a final observation risk rating, the likelihood and impact scores of an observation are multiplied together. The final score determined will determine whether or not the observation is a LOW, MODERATE, or HIGH risk observation using the Observation Risk Rating Table

Observation Risk Rating Table
Observation Risk Matrix
Impact Score
Likelihood 1 2 3 4
1 1 2 3 4
2 2 4 6 8
3 3 6 9 12
4 4 8 12 16
Observation Risk Thresholds
LOW 1 - 3
MODERATE 4 - 9
HIGH 12 - 16

Observation risk ratings play a key role in determining the Control Health and Effectiveness rating of a controls health. See the Control Health and Effectiveness Rating (CHER) handbook page for details on CHER and how to establish a CHER rating.

Observation Risk Ratings

Tier 3 information system risk ratings are based off the STORM risk rating methodology.

Risk Rating = Likelihood x Impact

An observation’s risk rating is based on a judgmental assessment of both the Likelihood and Impact of the identified observation by the team member that identified and opened the observation. Judgmental elements considered may include:

  • frequency of the same control observation
  • if the observation will lead to a material weakness or external audit finding
  • pervasiveness of the observation across multiple systems
  • impact to internal and/or external stakeholders
  • expertise to exploit the observation

See the Observation Management Procedure Handbook page for a full list of elements that determine both Likelihood and Impact scores

Observation Risk Rating Adjustments

As observations are identified, assigned and then move into remediation, there may be cases in which observations do not appropriately map to current GCF controls or are deemed stale. These observations could qualify for an upgraded or downgraded risk level depending on a subset of resolution criteria. Procedures for this process can be found in the Observation Risk Rating Adjustment Runbook

Observation Remediation

For detailed procedures relating to observation remediation, refer to the observation remediation procedure.

Metrics and Reporting

The observation management program dogfoods GitLab features for metric and reporting. Please refer to the observation management project’s insight charts and issue boards for metrics for all observations by risk, status, and department.

Exceptions

Exceptions will be created for observations that breach a mutually agreed upon remediation date, breach in SLA or if the Remediation Owner confirms the observation will not be remediated.

Exceptions to this procedure will be tracked as per the Information Security Policy Exception Management Process.

References

Contact & Feedback

If you have any questions or feedback about the observation management process please contact the GitLab Security Assurance Team, or comment in this feedback issue.

Last modified October 4, 2024: Fix GitLab capitalization (7104f09a)
View page source - Edit this page - please contribute. Creative Commons License