Compliance Adherence Reporting

Summary

This blueprint serves as living documentation of the technical considerations in the implementation of Compliance Adherence Reporting. This includes functionality in Compliance Frameworks, the Compliance Center and the relationship with Security Policies.

Proposal

Outline how compliance enforcement and visibility will be handled through Compliance Frameworks as an evolution from Compliance Standards Adherence.

Motivation

There are three main parts to compliance posture of a customer Enforcement, Visibility, and Audit History.

Enforcement

Enforcement within GitLab is currently done through Security Policies and Compliance Pipelines. This gives users two very different ways of implementing the same functionality, which has been described as confusing for users. Compliance pipelines also have some inherent technical limitation, see epic.

Visibility

Currently the Standards are hard coded in the Adherence report (renamed to Status report) and separate from Compliance Frameworks. This makes the system inflexible as we look to:

1. Use Compliance Frameworks to include certain projects in the Adherence Report and distinguish which requirements those projects are complaint with.
2. Add a Requirements level to the Adherence report
3. Add more Standards and Checks
4. Allow users to customise Standards
5. Allow users to create their own Standards
6. Allow users to create customisable Checks

Audit History

This is currently achieved through compliance events (Audit events and Violations within MRs).

Background

Deprecate compliance pipelines

We deprecated compliance pipelines in favour of pipeline execution policy in GitLab 17.3. This decision was taken to align with the future state of enforcing compliance through security policies.

Scope policies through compliance frameworks

We introduced the capability of scoping security policies allowing us to enforce policies against a specific set of projects or against projects applied a given set of compliance frameworks. This helped us move towards the goal of enforcing compliance through frameworks.

Multiple compliance frameworks

Before GitLab 17.3 it was not possible to apply more than one compliance framework to a project. To work towards the future state of allowing users to customise the adherence dashboard, we created the ability to apply multiple compliance frameworks in GitLab 17.3.

Goals

1. Provide support need for multiple frameworks
2. Incongruence with compliance pipelines and security policies
3. Use Compliance Frameworks to include certain projects in the Adherence Report and distinguish which requirements those projects are complaint with.
4. Add a Requirements level to the Adherence report
5. Add more Standards and Controls
6. Allow users to customise Standards
7. Allow users to create their own Standards

Non-Goals

1. Allow users to create customisable Controls
2. Compliance events
   1. Violations within MRs
   2. Audit events

Terminology/Glossary

1. Framework
   1. A Compliance Framework is a user-modifiable capability within GitLab to identify projects that have certain compliance requirements or need additional oversight. Compliance Frameworks generally align with established industry compliance frameworks such as SOC2 or ISO 27001.
2. Standard - deprecated in favor of framework
   1. A standard groups together compliance checks in the Adherence report. Compliance standards align with established compliance frameworks/standards such as SOC2 or ISO 27001
3. Adherence
   1. Reports the percentage of a projects compliance posture against a compliance framework. For example if Project A is 50% complaint towards Framework A.
4. Policy
5. Requirement
   1. A particular requirement from an industry standard compliance frameworks/standards such as SOC2 or ISO 27001. Usually a statement of intent for a particular part of the compliance framework. These are broken down into specific Controls.
6. Check
   1. A Check is a review of a project’s settings, to confirm that it is in a particular position. Checks compose a percentage of a project’s compliance posture against a Control.
7. Control
   1. A control is a specific compliance rule that needs to be met to meet a compliance requirement. Enforcement of this is achieved in GitLab through settings, Security Policies or Compliance Pipelines.

Decisions

• 001: Triggering Checks (changed, see ADR 004)
• 002: Custom Adherence Report
• 003: Custom Controls
• 004: Use Time-based Triggers for Checks

Design Details

See Scalability review document for further details.

We decided to use Sidekiq workers for creating checks and storing the adherence configuration in database as relational data. It was decided to combine compliance_checks and compliance_requirements tables to reduce redundancy.

The compliance requirements would be stored in a separate table with the following schema:

    classDiagram
    class namespaces {
        id: bigint
        name: text
        path: text
        ...(more columns)
    }
    class projects {
        id: bigint,
        name: text
        path: text
        description: text
        ...(more columns)
    }
    class compliance_management_frameworks {
        id: bigint,
        name: text,
        description: text,
        ...(more columns)
    }

    class compliance_requirements {
        id: bigint
        created_at: timestamp
        updated_at: timestamp
        namespace_id: bigint
        framework_id: bigint
        name: text
        description: text
        requirement_type: smallint
        external_url: text
        expression: text
    }

    class project_compliance_status {
        id: bigint
        created_at: timestamp
        updated_at: timestamp
        project_id: bigint
        namespace_id: bigint
        compliance_requirement_id: bigint
        status: smallint
    }

    class compliance_framework_security_policies {
        id: bigint
        created_at: timestamp
        updated_at: timestamp
        framework_id: bigint
        policy_configuration_id: bigint
        policy_index: smallint
        project_id: bigint
        namespace_id: bigint
    }

    compliance_management_frameworks --> compliance_requirements : has_many
    compliance_management_frameworks <-- compliance_requirements : belongs_to
    compliance_management_frameworks <--> projects : many_to_many
    compliance_requirements <--> compliance_framework_security_policies : has_and_belongs_to_many
    projects <-- namespaces : has_many
    projects --> namespaces : belongs_to
    namespaces --> compliance_management_frameworks : has_many
    namespaces <-- compliance_management_frameworks : belongs_to
    projects --> project_compliance_status : has_many
    projects <-- project_compliance_status : belongs_to
    compliance_requirements --> project_compliance_status : has_one
    compliance_requirements <-- project_compliance_status : belongs_to

We created a new table project_compliance_configuration_status for storing the results of compliance requirements and plan on dropping the existing project_compliance_standards_adherence table. We no longer have a standard column as we don’t want to associate requirements directly with a standard, allowing the users to customise and group requirements as per their need.

Unlike the current implementation we would only store results for the projects that have compliance requirements configured. Instead of an enum we would store the compliance_requirement_id in the project_compliance_configuration_status table and would display these results at the compliance dashboard.

In the next iteration we would also allow importing and exporting the compliance requirement configurations.

Implementation Details

FAQ

View page source - Edit this page - please contribute.