Event Stream Working Group

Attributes

Context

There are currnetly 3+ methods for publishing and listening to events withint the GitLab product. There is no single standard for events and there is no standard way of publishing and subscribing to events from other GitLab systems and services.

There is a growing need to standardize events and how to publish and consume them for both GitLab.com and self-managed installs.

Working Group Goals

This Working Group has the following goals:

1. Document the current event systems
2. Document the currently identified use cases for an event stream
3. Define and socialize an architecture for a standardized event stream and events
   1. Architecture Blueprint for Event Stream usage within GitLab and self-managed
   2. Define sene and secure defaults. Provide security recommendations and guidelines.
4. Define a rollout strategy and plan
   1. SaaS rollout strategy for current use cases
   2. Prepare the groundwork for adding the event stream to our reference architectures
5. Develop a communication plan to socialize the outcome of the working group

Exit Criteria Progress

Definitions

What is an Event Stream?

An event stream is simply a stream of events that occur within the product. More specifically, for us at GitLab, it will involve the software changes we need to make in the product, event schemas and protocols, and infrastructure.

Potential Tools/Services

There are some tools/services that other companies are using for this

1. Kafka
2. NSQD
3. RabbitMQ
4. GCP PubSub

Related GitLab Documentation or Issues

• https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34042
• https://gitlab.com/gitlab-org/gitlab/-/issues/338454
• Streaming Audit Events

Exit Criteria

If it is decided that an Event Stream should be implemented, our exit criteria should be:

1. Define the tool(s)/service(s) we should use for SaaS and on-prem (they might be different)
2. Define system architecture
3. Define an implementation plan and philosophy
4. Define event structure

Roles and Responsibilities

View page source - Edit this page - please contribute.