Workforce hierarchy integration

Import the customer’s HR workforce units and their members into GitLab.
This page contains information related to upcoming products, features, and functionality. It is important to note that the information presented is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. The development, release, and timing of any products, features, or functionality may be subject to change or delay and remain at the sole discretion of GitLab Inc.
Status Authors Coach DRIs Owning Stage Created
accepted pshutsin ahegyi pshutsin devops analytics 2026-11-12

Summary

Customers organize people in an HR system, such as Workday. GitLab groups do not match that structure, so GitLab cannot tell which users belong to which part of the customer’s organization.

This integration imports the customer’s workforce units and their members into GitLab. A daily sync stores the current state in PostgreSQL. Siphon replicates it to ClickHouse, which keeps the history. A GraphQL API reads current and past state from ClickHouse. Without ClickHouse, it reads the current state from PostgreSQL.

The hierarchy is read-only reference data. It grants no access.

This document describes the target state for general availability. Later changes get their own decision records.

Status

A planned internal release (V1) on GitLab.com implements a subset: the gitlab-org root group, the Workday RaaS API, the supervisory unit type, email matching, PostgreSQL only, and the GraphQL API.

Tracking: epic 23251.

Goals

  • Sync a tenant’s hierarchy from an HR source with no manual work after setup.
  • Keep the model vendor-neutral: a new source changes no table and no API.
  • Link HR people to GitLab users, with administrator overrides.
  • Expose the hierarchy through GraphQL and reusable services.
  • Support GitLab.com, Self-Managed, Dedicated and air-gapped instances.

Non-Goals

  • Metrics, dashboards, aggregation engines, GLQL. They consume the hierarchy.
  • Access control. No permission check reads the hierarchy.
  • Write-back to HR systems, or HR attributes such as title, location or compensation.
  • Replacing groups or Teams.

Proposal

flowchart LR
    subgraph SOURCES["Data sources"]
        WAPI(["Workday RaaS API"])
        WFILE(["Workday file import"])
        OTHER(["..."])
    end
    subgraph MONOLITH["Monolith"]
        SYNC["Sync + identity matching"]
        PG[("PostgreSQL<br/>current state")]
    end
    subgraph DIP["DIP"]
        SIPHON["Siphon"]
        CH[("ClickHouse<br/>current state + history")]
    end
    subgraph MONOLITH_READ["Monolith"]
        API["GraphQL"]
    end
    CONSUMERS(["Consumers"])
    WAPI --> SYNC
    WFILE --> SYNC
    OTHER --> SYNC
    SYNC --> PG
    PG --> SIPHON --> CH
    CH --> API
    PG -. "no ClickHouse" .-> API
    API --> CONSUMERS
    style SOURCES fill:none,stroke-dasharray: 5 5
    style MONOLITH fill:none,stroke-dasharray: 5 5
    style DIP fill:none,stroke-dasharray: 5 5
    style MONOLITH_READ fill:none,stroke-dasharray: 5 5
Decision Choice
Purpose The workforce hierarchy should be read-only reference data. It is not a principal or a container, and it stays separate from groups and Teams
Tenant The tenant should be the root group on GitLab.com and the organization on Self-Managed and Dedicated. GitLab.com later moves to the organization (work item 634509). Each tenant has one source and at most one hierarchy per unit type
Model Each hierarchy should be one tree of units of one unit type. A person belongs to at most one unit per type. A unit has at most one manager
Sources All sources should implement one vendor-neutral contract. The first source is the Workday RaaS API. The second source is the Workday RaaS file import
Sync The sync should fetch a daily full snapshot and validate it before any write. It writes only the rows that changed
Identity Identity matching should use pluggable strategies, with email matching as the first strategy. Administrator overrides persist across syncs
Storage PostgreSQL should hold the current state. Siphon replicates it to ClickHouse, which keeps the history
Read API The GraphQL API should sit on top of reusable services and read from ClickHouse. Without ClickHouse, it falls back to the PostgreSQL current state
Naming Code should use the Workforce namespace. It never uses the name “organization”, to avoid confusion with Organizations

Design details

Data model

Tables use the workforce_ prefix in gitlab_main_org, with the tenant as sharding key.

Entity Attributes
Hierarchy Tenant, integration, unit type, last sync status and time
Unit External ID, name, type, parent, ancestry path, manager external ID, manager user
Person External ID, full name, matched user
Unit membership Person, unit, type
Defect Subject kind, reason, locator. Latest run only
Identity override External person ID, user
  • Unit type is free text. A new type, such as cost center, needs no schema change.
  • A hierarchy holds the units of one unit type. A tenant has at most one hierarchy per unit type.
  • A sync run updates all hierarchies of the tenant, because the tenant has one source.
  • A unit’s parent has the same type. The store never contains a cycle, not even between batches.
  • The ancestry path works like traversal_ids: a subtree is one range scan.
  • A person’s manager is not stored. Only units have managers.
  • Deleting a user clears the link and removes the user’s identity overrides. The person and memberships stay.
  • An identity override references the external person ID, not a person row. It stays when the person is absent from a snapshot and applies again when the person comes back.

Sources

A source returns units (ID, name, parent ID, manager ID) and people (ID, name, work email, unit ID per type).

  • Workday RaaS API. Two customer-built reports. OAuth 2.0 refresh token of an Integration System User. The customer controls which columns the reports expose.
  • Workday RaaS file. Upload of the same reports, for air-gapped instances. File formats differ by provider, so each file import is tied to a provider.
  • The integration is configured at tenant level only. It does not propagate to subgroups or projects.

Sync

  • Daily, plus a manual trigger. One run per tenant at a time. 30-minute manual cooldown on GitLab.com only.
  • Fetch and validate the full snapshot first. A failure changes nothing.
  • The source is the source of truth. A complete, valid snapshot is applied as received, even if it removes most people or units. To recover from a wrong snapshot, fix the source and start a manual sync.
  • Apply in batches of at most 5,000 rows, units root to leaf, then people, then memberships. Removals last. No transaction spans the snapshot. A restarted run converges to the same state.
  • Unchanged rows are not written, so Siphon receives no event for them.
  • Invalid rows become defects, as the table below shows. The run succeeds.
  • Logs hold identifiers of tenant and run, and counts. Never personal data.
  • Target: 50,000 people and 10,000 units per tenant.
Invalid row Result
Person or unit with a missing or duplicate ID The row is left out
Unit with an unknown parent or a parent cycle The unit stays, with no parent. It becomes a root
Unit with a manager who is not in the people data The unit stays, with no manager
Person with an empty or unknown unit The person stays, with no membership of that type
Person with a work email that another person also has The person stays unmatched

Defects never appear in read results. A run clears the defects when it starts and writes them when it finishes.

Identity matching

  • Candidates are users of the tenant only: members of the root group hierarchy on GitLab.com, active users of the organization elsewhere.
  • First strategy: work email against confirmed GitLab emails. Email is never persisted or logged.
  • Overrides are stored by external person ID and win over strategies.
  • Unmatched is a normal state, not a defect.

Storage and history

  • PostgreSQL holds the current state. Ingestion stays in Rails.
  • Siphon replicates the tables to ClickHouse. No second export path.
  • ClickHouse builds the history. The method is not defined yet. A separate decision can define it later, for example with a slowly changing dimensions (SCD) solution.

Read API

For each unit, GraphQL returns: name, type, parent, manager user (null if unmatched), direct members, and members including descendants. Members are GitLab users.

  • An optional point in time selects a past state. The default is the current state.
  • With ClickHouse, all reads use ClickHouse. The current state lags PostgreSQL by the Siphon replication delay.
  • Without ClickHouse, reads use the PostgreSQL current state. A point-in-time read returns an error.
  • Resolvers call reusable services. Other features can use the services directly.
  • Every read is authorized.
  • No HR identifiers, HR names of people, emails or unmatched people.
  • Read-only. Subtree with members under 100 ms per page at target scale.

Data protection

  • Only the HR attributes in Data model are stored.
  • The integration stores the source settings and credentials, such as the OAuth 2.0 refresh token. Credentials are encrypted at rest.
  • Deleting the integration removes all tenant hierarchy data in batches. Deactivating it keeps the data.
  • Integration and override changes are audited.
  • The separate history decision defines history retention and erasure.

Alternative Solutions

  • Tree of people linked by manager (proof of concept, MR 261107). Rejected: reporting lines change often. Units are stable and support several hierarchy types.
  • Ingest into ClickHouse first. Rejected: Rails needs current state in PostgreSQL, and a second path duplicates Siphon.
  • History in PostgreSQL. Rejected: ClickHouse serves history reads, and Siphon delivers every change.
  • Delta sync. Rejected: HR reports are full snapshots, and full snapshots make each run self-healing.
  • Teams or groups as storage. Rejected: both carry access semantics.
  • Integration Factory, a shared framework of declarative extractors for third-party systems. Not adopted now: it is an experiment, and no production integration runs on it. We can re-evaluate it for the workforce hierarchy later.

Decisions

Decision record In one line
ADR-001: Workforce hierarchy architecture Read-only units imported by a daily full snapshot into PostgreSQL, with history in ClickHouse through Siphon

References


ADR-001: Workforce Hierarchy Architecture
The workforce hierarchy is read-only reference data of units. A daily full snapshot from a vendor-neutral source updates PostgreSQL, and Siphon replicates it to ClickHouse for history.