Security Analyzer Configuration Profiles

Summary

We need a scalable way of storing and injecting analyzer configuration between security analyzers and Gitlab Rails. By introducing persisted configuration profiles we can provide an explicit and injectable list of configuration options to be used when executing a security scan.

Motivation

Configuration is a comprehensive categorical definition of all user-modifiable behavior around execution of GitLab’s security scans. This includes log levels, file path exclusions, number of cores utilized, and the rules under test.

Analyzer configuration was first designed for CI-based scans, but increasingly, GitLab security analyzers are moving outside of pipelines. For CI-based analyzers, we have traditionally relied on exposing CI variables in the corresponding components and templates which are passed to each analyzer for configuring the execution of a scan.

There are several challenges with this approach:

• Difficulty in configuring scans performed outside of CI contexts (e.g. Secret Push Protection and Continuous Vulnerability Scanning)
• Inconsistent configuration profiles between different scan contexts
• Inability to store user-defined configurations uniformally
• Inability to scale shared configuration to organization-wide levels

There’s also a growing need to standardize configuration and store user-defined configurations in a consistent and scalable manner.

Goals

Provide GitLab users with a way to:

• Securely store user-defined analyzer configurations within GitLab
• Manage multiple configuration profiles
• Inject configuration profiles into all possible scan contexts
• Validate all possible configuration values for each supported analyzer
• Manage configuration profiles at a group and project level
  • Excludes organization profiles
• Get visibility into what has been configured, and to which projects that configuration has been applied

Use cases

Sequence for profile scan context configuration

sequenceDiagram
    autonumber
    participant ProfilesUI
    participant Rails
    participant DB
    participant SEP

    ProfilesUI->>+Rails: Create SD profile
    Rails->>DB: insert
    Rails-->>-ProfilesUI: OK

    ProfilesUI->>+Rails: Enable SPP for profile

    note over Rails: Create execution hook for SPP
    Rails->>DB: insert security_scan_profile_contexts
    Rails->>DB: update <br>project_security_settings.secret_push_protection_enabled <br>(enable setting)
    Rails-->>-ProfilesUI: OK

    ProfilesUI->>+Rails: Disable gitlab_personal_access_token rule for SPP
    Rails->>DB: insert security_scan_profile_rule_overrides
    Rails->>DB: insert security_scan_profile_rule_override_contexts<br>(associate rule to SPP context)

    Rails-->>-ProfilesUI: Enabled!

    ProfilesUI->>+Rails: <br><br><br><br>Enable Pipeline SD for profile

    note over Rails: Create execution hook for PSD
    Rails->>DB: insert security_scan_profile_contexts
    Rails->>SEP: create Security Policy Project w/ SEP
    Rails-->>-ProfilesUI: OK

    ProfilesUI->>+Rails: Disable gitlab_personal_access_token rule for PSD
    Rails->>DB: insert security_scan_profile_rule_overrides
    Rails->>DB: insert security_scan_profile_rule_override_contexts<br>(associate rule to PSD context)

    Rails-->>-ProfilesUI: Enabled!

    ProfilesUI->>+Rails: Add path exclusion
    Rails->>DB: insert security_scan_profile_exclusions
    Rails-->>-ProfilesUI: OK

Sequence for CI-based scans utilizing custom rule exclusions

sequenceDiagram
    autonumber
    Analyzer->>+Rails: GET /api/projects/:id/security_profiles/:scan_type via ci_job_token
    Rails-->>+Analyzer: {excluded_paths: ["vendor"], rule_overrides: {"gitlab_pat": {status: "disabled"} } }
    Analyzer->>Analyzer: Execute scan with configuration profile
    Analyzer->>-Analyzer: Generate report with configuration used
    Analyzer->>+Rails: Store security report
    Rails-->>-Rails: Associate profile to security_scan

Sequence for CI-based scans utilizing custom configuration parameters

sequenceDiagram
    autonumber
    Analyzer->>+Rails: Provide list of configuration options via versioned configuration schema
    Rails-->>+Analyzer: Use user-defined overrides + defaults
    Analyzer->>Analyzer: Execute scan with configuration profile
    Analyzer->>-Analyzer: Generate report with configuration used
    Analyzer->>+Rails: Store security report
    Rails-->>-Rails: Store scan configuration

Non-Goals

• Configuration Profiles are focused on the way in which an analyzer executes its scan. The configuration and triage of the results is out of scope of initial design
• Configuration profiles are not enforced for third-party scans, but are made available if a compliant scan wishes to consume them

Terminology

• Scan - Execution of software that scans for vulnerabilities (see Analyzer and Scanner glossary)

• Rule modification - changes to a single predefined rule used by a analyzer; i.e. severity overrides or rule pattern augmentation

• Ruleset - the collection of rules a single analyzer uses; for example “GitLab Advanced SAST’s ruleset is composed of 100 rules”

• Rule - A collection of patterns associated with an identifiable vulnerability. A rule has a unique identifier and metadata (i.e. severity).

• Rule augmentation - A rule whose pattern has been modified to increase specificity or scope of detection

• Rule exclusion - A rule that has been disabled for a given scan profile configuration

• Rule override - A rule whose metadata properties have been overridden; i.e. a change in severity

• Enablement - When an analyzer has been configured to run by default for a given project and detection context

• Enforcement - When an analyzer has been “enabled” and can only be disabled by elevated permissions

• Execution - When a scan has been triggered for a given analyzer

Terms to Avoid

The following terms are broad, multifacted, and can lead to confusion in overlapping intent. They should be avoided without qualifiers.

• Configuration
• Customization

For example:

• Prefer: “In Q1, we will support Scan Configuration Profiles. In Q2, we will support Scan Configuration Enforcement”
  • Not: “In Q1, we will support Scan Configuration”
• Prefer: “In Q1, we will support Rule Exclusions. In Q2, we will support Rule Overrides”
  • Not: “In Q1, we will support Rule Customization”

Proposal

1. Persistent Configuration Profiles
   1. Define data model for persisting customizable group-level configuration profiles
   2. Support exclusions
      1. Associate path and value-based exclusions with configuration profiles
      2. Associate rule customization with configuration profiles
2. Scan ruleset management
   1. Define syncing mechanism for persisting scan rules to GitLab Rails
3. Scan configuration settings
   1. Define JSON schema encompassing all supported scan configuration options
   2. Develop API for persisting and managing user preferences as configuration profiles (validated against schema)
   3. Develop UI for persisting and managing user preferences as configuration profiles (validated against schema)
   4. Inject chosen** configuration profile into scan contexts (starting with CI)
   5. Update analyzers to prefer injected configuration settings, prioritizing over ENV and defaults
   6. Update analyzers to provide configuration used to execute each scan
   7. Generate audit events on configuration profile changes
   8. Create both a new default and new custom role for scan configuration profile modification

Requirements

1. Core data model
2. Workload orchestration of scan triggers
3. Migration strategy - from existing enablement to profiles
4. Migration strategy - from profiles to advanced configuration
5. Persistence of configuration parameter overrides - Flow is largely defined, but data model needs further definition. This is a blocker for DAST and DS auto-remediation, but nice to have for general behavioral config

Phase 1 - enablement-only profiles

For this phase, we provide read-only profiles meant to facilitate rapid onboarding.

1. Predefined instance-wide configuration profiles, defined per scanner
2. Profiles are associated to individual projects
3. Profiles have predefined scan triggers; i.e. “Secret Detection Push Protection Profile” and “Secret Detection Pipeline Protection Profile”
4. Profiles cannot customize exclusions, rules under test, and scanner parameters
5. When a profile is applied, an on-demand scan is immediately triggered. Subsequently, the trigger condition is used to inform future scans.

Phase 2 - customizable profiles

Customizable configuration profiles, per scanner that can be enabled for each project.

1. Profiles are defined at the group-level, inheritable by subgroups
2. Profile can have many scan triggers
3. Profiles can customize exclusions, rules under test, and scanner parameters
4. When a profile is applied, an on-demand scan is immediately triggered. Subsequently, the trigger condition is used to inform future scans.

Workload orchestration of scan triggers

Profiles include a definition of the triggers by which a scan must run. Alongside persistence of trigger conditions, we need an orchestration layer capable of scheduling and initiating scan using several criteria:

1. Event-based executions, such as git push events, advisory updates, or deployments
2. Time-based executions, such as scheduled pipelines
3. Pipeline-based executions, such as branch and MR pipelines

These strategies will be further explored in [Spike] Profile Workload Orchestration

flowchart LR
  A1[Assign Profile]
  subgraph "Enabled triggers"
    B1[Branch Pipelines]
    C1[MR Pipelines]
    D1[Scheduled scans]
    E1[Push Events]
    F1[IDE Scanning]
    G1[Advisory Updates]
  end

  subgraph "Classification"
    B2[Pipeline-based]
    D2[Time-based]
    E2[Event-based]
  end

  subgraph "Workload Orchestration"
    H1[Scan Execution Policy]
    I1[Pipeline Schedules]
    I2[Scheduled SEP]
    K1[On-Demand]
  end

  A1-->B1
  A1-->C1
  A1-->D1
  A1-->E1
  A1-->F1
  A1-->G1

  B1-->B2
  C1-->B2
  B2-->H1

  D1-->D2
  D2-->I1
  D2-->I2

  E1-->E2

  F1-->E2
  G1-->E2
  E2-->K1

Configuration JSON schema

Example schema:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "",
  "title": "Security Scan Configuration Options",
  "description": "Mapping of all possible configuration options for Gitlab security scans",
  "type": "object",
  "properties": {
    "additional_ca_cert_bundle": {
        "description": "Bundle of CA certs that you want to trust.",
        "type": "string"
    },
    "log_level": {
        "description": "Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: fatal, error, warn, info, debug",
        "type": "string",
        "enum": ["fatal, error, warn, info, debug"],
        "default": "info"
    },
    "secret_detection_historic_scan": {
        "description": "Flag to enable a historic Gitleaks scan",
        "type": "boolean",
        "default": false
    },
  }
}

Profiles DB Schema

classDiagram
    class namespaces {
        id: bigint
        path: text
        ...
    }

    class projects {
        id: bigint
        path: text
        ...
    }

    class security_scan_profile_projects {
        project_id: bigint
        profile_id: bigint
    }

    class security_scan_profiles {
        id: bigint,
        namespace_id: bigint,
        traversal_ids: array,
        ...
    }

    class security_scan_profile_contexts {
        id: bigint,
        context_type: enum, secret_push|secret_pipeline|...
        ...
    }

    note for security_scan_profile_exclusions "migration from project_security_exclusions"
    class security_scan_profile_exclusions {
        id: bigint
        profile_id: bigint
    }

    %% These are rule exclusions/modifications
    %% Store only disablements initially
    note for security_scan_profile_rule_overrides "migration from project_security_exclusions"
    class security_scan_profile_rule_overrides {
        id: bigint
        profile_id: bigint
        %% replaceable with an enum once synchronization infrastructure is in place
        identifier: text
        context_types: array
    }

    class security_scans {
        id: bigint
        project_id: bigint
        scan_type: text
    }

    projects <-- namespaces : has_many
    projects --> security_scans : has_many

    security_scan_profiles --> security_scans : has_many
    security_scan_profiles --> security_scan_profile_rule_overrides : has_many
    security_scan_profile_rule_overrides --> security_scan_profile_contexts
    security_scan_profiles --> security_scan_profile_contexts : has_many
    security_scan_profiles --> security_scan_profile_exclusions : has_many
    security_scan_profile_projects <-- projects : has_many
    security_scan_profile_projects <-- security_scan_profiles : has_many
    namespaces --> security_scan_profiles : has_many

View page source - Edit this page - please contribute.