Observation Creation Procedure
This is a Controlled DocumentInline with GitLab’s regulatory obligations, changes to controlled documents must be approved or merged by a code owner. All contributions are welcome and encouraged.
The Observation Management Program at GitLab is used to identify, track, remediate and provide a risk ratings of identified findings, exceptions or deficiencies for any Tier 3 information system risks that are identified as an output of compliance operations or other mechanisms by team members, such as self-identification of a system specific risk.
This procedure details the creation process for observations.
Introduction to Observation Management at GitLab
Tier 3 risks or observations identified at the information system or business process levels
Roles and Responsibilities
|Observation Source||Responsible Team|
|Security control testing activities (CCM)||Security Compliance|
|Third Party Risk Management (TPRM) activities||Security Risk|
|Customer Assurance Activities (CAA)||Field Security Team|
|External audit activities||Security Compliance|
|Third party application scanning (BitSight)||Field Security Team|
|Gap Assessment Activities||Security Compliance|
|Ad-hoc observations||Security Assurance|
graph TD; A[Identified] --> B[Assigned]; B --> C[Remediation in progress]; B --> D[Ignored or Invalid]; C --> F[Resolved];
Observations can be identified through the following channels:
- Security control testing activities (CCM)
- Third Party Risk Management (TPRM) activities
- Customer Assurance activities (CAA)
- External audit activities
- Third party application scanning (BitSight)
- Gap assessment activities
- Ad-hoc issues
The observation identifier is responsible for opening an observation in ZenGRC. The observation identifier fills out all necessary observation information, remediation recommendations and submits the observation to the Remediation Owner for validation. The Observation Manager is responsible for managing the observation through the observation lifecycle. This includes validating the observation with the Remediation Owner, tracking all remediation progress and updating the associated ZenGRC issue with current information and status updates. Each observation has both a GitLab Issue (for Remediation Owners) and a mirrored ZenGRC Issue (for Observation Managers). Each observation will be assigned a risk rating, which should drive the priority of remediation.
See the associated runbooks below for detailed instructions on how to open and assign a new observation based on observation type:
Drafting Observation Description Guidance
The Observation description should include the who, what, when, where, why, and how related to the observation. As a review step, if you knew nothing about this observation could you understand the finding, how it was identified, and the effect it has on objectives? Consider leveraging the 4C’s model:
- Condition - current state
- Criteria - desired state based on policy, requirement, control, regulation, etc.
- Cause - root cause of the observation
- Consequence - actual or potential effect on objectives/assets
Tier 3 information system risk ratings are based off the formula below.
Risk Rating = Likelihood x Impact
Step 1: Define likelihood based on the below criteria and likelihood table
At GitLab, observations will be rated based on the likelihood the observation has recurring and/or the frequency that the control has seen observations. The criteria used to assess this likelihood can be found in the Likelihood Table below. There are two different definitions for each likelihood rating level, “control observation” and “information system risk (tier 3)”. These definitions delineate the factors that may be considered when thinking of an individual observation from a control perspective and an information system risk perspective. Each should be considered when determining the likelihood of an observation occurring.
- Control Observation: This criteria is utilized to rate observations identified as an output of control testing (e.g. where control testing performed internally by Security Compliance has failed). The assumption of the Likelihood Table is to consider observations individually rather than in aggregate (i.e if 2 similar observations occur against a single test of a sample of 25, the failure rate is 8% and would be scored a 3. The control does not need to be tested multiple times in the current year or prior 9 months with an observation each time to meet the requirement for a score of 3).
- Information System Risk (Tier 3): This criteria is utilized to score the likelihood of an information system being exploited (e.g. insufficient encryption mechanisms for the storage of data within
[System Name]result in the unintentional exposure/leakage of this information to the public).
|1||Control Observation: The observation noted is considered to be a one off occurrence for the control as a result of extenuating circumstances. It is unlikely to occur again once remediated.
Information System Risk: Theoretically impossible and/or requires significant technical expertise for the risk to be exploited.
|2||Control Observation: The observation was identified as a result of management's oversight on the control and may potentially occur again. This is the only observation associated with the control in the current fiscal year or prior 9 months, whichever is longer.
Information System Risk: Even with technical expertise, it is somewhat difficult to exploit the risk.
|3||Control Observation: The control has had multiple observations in the current fiscal year or prior 9 months, whichever is longer.
Information System Risk: Minimal expertise is required to exploit the risk.
|4||Control Observation: The control has observations that have persisted and continue to occur year to year AND/OR the observation noted is associated with the design of the control.
Information System Risk: The risk can be easily exploited and does not require any technical expertise.
Step 2: Define impact based on the below criteria and impact table
In addition to applying a qualitative scoring factor for likelihood, all observations need to be evaluated for the impact they would have on GitLab at the organization level and/or the compliance impact (if applicable). The criteria and qualitative scores for assessing the impact of an observation can be found in the Impact Scoring Table below. The highest rating in any field is the final impact score of the observation so as to approach observations in a more conservative manner (i.e if all fields are rated at a value of 2 except Remediation Effort which is scored a 3, the final impact score would be a 3).
Important Note: Team members who are leveraging the impact scoring criteria below may judgmentally select the impact factors most relevant to them. Internal Audit and Security Compliance utilize all columns when scoring observations identified as part of controls testing because there may be specific impacts to external compliance audit requirements as a result of these findings. Any information system risk identified outside of control testing may utilize the columns that are most relevant.
Impact Scoring Table
|Qualitative Score||External Audit Impact||Remediation Effort||Financial Impact||Legal & Regulatory Impact||Stakeholder/ ICOFR (Internal Controls Over Financial Reporting) Impact|
|1||The observation would not lead to an adverse audit opinion.||The observation was related to extenuating circumstances and requires simple reinforcement of policy/process, no additional management oversight required.||Observation has potential financial impact which results in loss / misstatement of up to $25K.||The observation would not lead to major action by a regulator.||The observation has minimal impact on all stakeholders (internal and external)|
|2||The observation would likely not lead to an adverse audit opinion because it is an isolated occurrence.||The observation has remediation effort that requires oversight/support at the management level.||Observation has potential financial impact which results in loss / misstatement between $25K to $250K.||The observation could lead to minor regulatory action.||The observation impacts internal stakeholders and or could lead to financial misstatements, if not addressed on time.|
|3||The observation is likely to result in an adverse audit opinion if a full sample for remediation testing cannot be provided.||The observation has remediation effort that requires oversight/support at the director level.||Observation has potential financial impact which results in loss / misstatement between $250K to $500K.||The observation could lead to an investigation or regulatory action.||The observation impacts internal and external stakeholders. It requires attention of Executives and Board.|
|4||The observation will result in an adverse audit opinion.||The observation has remediation effort that requires oversight/support at the Executive level.||Observation has potential financial impact which results in loss / misstatement above $ 500K||The observation could directly result in major regulatory action against GitLab.||The observation impacts internal and external stakeholders. It requires attention the of Executives and Board and could impact management assertion in 10Q / 10-K.|
Determining Risk Rating
Step 3: Determine risk rating based on likelihood and impact scores defined in steps 1 and 2
In order to arrive at a final observation risk rating, the likelihood and impact scores of an observation are multiplied together. The final score determined will determine whether or not the observation is a LOW, MODERATE, or HIGH risk observation using the Observation Risk Rating Table
Observation Risk Rating Table
|Observation Risk Matrix|
|Observation Risk Thresholds|
|LOW||1 - 3|
|MODERATE||4 - 9|
|HIGH||12 - 16|
Observation risk ratings play a key role in determining the Control Health and Effectiveness rating of a controls health. See the Control Health and Effectiveness Rating (CHER) handbook page for details on CHER and how to establish a CHER rating.
Observation Risk Ratings
Tier 3 information system risk ratings are based off the STORM risk rating methodology.
Risk Rating = Likelihood x Impact
An observation’s risk rating is based on a judgmental assessment of both the Likelihood and Impact of the identified observation by the team member that identified and opened the observation. Judgmental elements considered may include:
- frequency of the same control observation
- if the observation will lead to a material weakness or external audit finding
- pervasiveness of the observation across multiple systems
- impact to internal and/or external stakeholders
- expertise to exploit the observation
See the Observation Management Procedure Handbook page for a full list of elements that determine both Likelihood and Impact scores
Observation Risk Rating Adjustments
As observations are identified, assigned and then move into remediation, there may be cases in which observations do not appropriately map to current GCF controls or are deemed stale. These observations could qualify for an upgraded or downgraded risk level depending on a subset of resolution criteria. Procedures for this process can be found in the Observation Risk Rating Adjustment Runbook
For detailed procedures relating to observation remediation, refer to the observation remediation procedure.
Metrics and Reporting
The observation management program dogfoods GitLab features for metric and reporting. Please refer to the observation management project’s insight charts and issue boards for metrics for all observations by risk, status, and department.
Exceptions will be created for observations that breach a mutually agreed upon remediation date, breach in SLA or if the Remediation Owner confirms the observation will not be remediated.
Exceptions to this procedure will be tracked as per the Information Security Policy Exception Management Process.
- Parent Policy: Information Security Policy
- GCF Contol Lifecycle
- Observation remediation Procedure
- Observation Management Project
- Insight Charts
- Sarbanes-Oxley (SOX) Compliance
Contact & Feedback