Security Logging Overview
Security Logging supports and develops GitLab’s security log ingestion platform.
Our Vision
Guarantee that GitLab has the logging data coverage required to:
- Perform the threat analysis, alerting and threat detections necessary to protect the company and its customers
- Ensure compliance with internal policies, standards, internal and external audits, and regulatory requirements.
Our Mission Statement
The team achieves its vision by planing, executing and supporting initiatives
that improve the coverage and usability of security logging data on GitLab.
We manage, maintain, design, configure, and document the necessary tools,
systems and processes to make that happen.
Further details can be found in the job family description.
The Team
Team Members
The Security Logging Team is part of the Product Security sub-department. See GitLab’s organizational chart and meet our team members.
Team Responsibilities
The Security Logging Team is responsible for security focused logging, monitoring, and alerting.
What we are responsible for
The Security Logging Team is responsible for managing, maintaining, designing, configuring, and documenting the necessary tools, systems and processes to support all security logging, monitoring, and alerting needs. This includes but is not limited to the following examples:
- Designing, managing, and maintaining the security logging and monitoring infrastructure
- Ensuring reliability and availability of log data for security purposes
- Owning and maintaining the security logging standard that defines important aspects and requirements of security logging, monitoring, and alerting at GitLab
- Working with our internal GitLab customers to ensure they have the logging data, and access to this data, needed to successfully accomplish the responsibilities of their roles
- Working closely with the Infrastructure team ensuring that log aggregation infrastructure is reliable and available to meet the needs of both Infrastructure and Security
- Building an maintaining a library of logging profiles for various technologies that can used to gather, format, and send logging data to the appropriate log aggregation systems
What we are not responsible for
The Security Logging Team is not responsible for the logging, monitoring, and alerting data or infrastructure supporting non-security focused needs. This includes but is not limited to the following examples:
- Infrastructure or application logging supporting reliability or availability of GitLab systems
- Responding to or taking action on the security alerts that are generated by the infrastructure and tools owned and maintained by the Security Logging Team
- Remediation of security vulnerabilities found using the infrastructure and tools owned and maintained by the Security Logging Team
- The Security Logging Team does not own the logging data aggregated, stored, or contained within the infrastructure and tools owned and maintained by the Security Logging Team
- The Security Logging Team is not responsible for the endpoint systems from which logging data is gathered
Working With Us
- Create an issue in our issue tracker dedicated to Business as Usual (BAU) activities and general inquiries.
- It is not necessary to
@mention anyone. In case you want to mention the whole team, use the @gitlab-com/gl-security/engineering-and-research/security-logging handle on GitLab.com.
- You can also chat with us on Slack in the dedicated
#security-logging channel or by tagging us @security-logging-team.
The Security Logging Team can be contacted in Slack using the #security-logging channel, the #security channel, or the #security-division channel. You can also contribute, comment, view, or interact with us in our team repo.
How We Work
We are an internal customer focused and customer driven team. Our customers drive our priorities and help us define our responsibilities. We work to balance this with a risk based approach aimed at reducing and minimizing security risk at GitLab. Additionally, we embrace the DevOps model, software defined infrastructures, a cloud first approach, modular decoupled architectures, self-serviceability, and automate when and wherever possible.
Meetings and Scheduled Calls
Our preference is to work asynchronously, within our project issue tracker as described in the project management section below.
The team does have set of regular synchronous calls:
- A weekly team sync to discuss progress, blockers, and anything related to the InfraSec team.
- A quarterly team retrospective to reflect on what went well in the previous quarter, and discuss what can be improved going forward.
- 1-1s between Individual Contributors and the Engineering Manager.
Team Pages
Project Management
We use Epics, Issues, and Issue Boards to organize our work, as they complement each other:
- The single source of truth for engineering work is the Security-Logging Sub-Group in GitLab. All Epics will be collected at this level.
- Having all projects at this level allows us to use a single list for prioritization and enables us to prioritize
work for different services alongside each other.
- Projects are prioritized in line with Security Logging Board.
Team Planning
- For the long term strategy of the InfraSec Team, you can refer to:
- From a tactical point of view, you can refer to:
Project Ownership
Each project has an owner who is responsible for delivering the project.
The owner needs to:
- Regularly update the status in the Epic description and milestones.
- Work with others to move project issues through the boards.
Labels
Please use the following labels for general work only:
| Label |
Use Case |
~"☁️ SecLog" |
Team Label (to be included in every project-related issue) |
~"SecLog::Incoming-Requests" |
For new issues which need to be triaged |
Design Documents
Before starting a new project, the team is encouraged
to define software designs through design docs.
These design doc documents the high level implementation strategy and key design decisions with emphasis on the trade-offs that were considered during those decisions.
To start discussing a new design:
- Create a new issue in the InfraSec Team Charter repo
- Select the
design_doc template
- Fill the data as requested
Security Logging Program Roles and Responsibilities
The following roles and responsibilities are specific to the management and execution of the Security Logging Program which is overall the responsibility of the Product Security sub-department.
Security Logging is responsible for
- Ownership of, management, and maintenance of our SIEM
- Coordinating and executing log source ingestion
- Overall management and coordination of the security logging program
- Ownership of the to be developed Security Logging Standard
- Subject matter experts with regards to the operation and management of the SIEM
- The data movement and archive of security logging data currently held in Panther
- Capacity planning and forecasting of licensing and infrastructure costs, as a shared responsibility with InfraSec
AppSec is responsible for
- Working closely with Development to ensure the Security Logging Standard is adopted and followed
- Advising SecLogging when log formats will change, prior to the change occurring, if known
- Advising SIRT on any log sources from the application that they may be interested in and recommend new high-value detection considerations to SIRT for new product features or capabilities
- Helping to bridge the gap between the Security Logging Team and the software engineering teams
- Helping to identify logging gaps in the application and proposing solutions to close the gaps
- Aiding and guiding engineering teams that design, develop, or deploy something new on what to log and how to get the logs into the SIEM. This includes logs that SIRT needs to complete investigations.
InfraSec is responsible for
- Working closely with Infrastructure to ensure the Security Logging Standard is adopted and followed
- Advising SecLogging when log formats will change, prior to the change occurring, if known
- Advising SIRT on any log sources from the infrastructure that they may be interested in and recommending new high-value detection considerations to SIRT for new product features or capabilities
- Helping to bridge the gap between the Security Logging Team and the Infrastructure teams
- Helping to identify logging gaps in the infrastructure and proposing solutions to close the gaps
- Aiding and guiding engineering teams that design, develop, or deploy something new on what to log and how to get the logs into the SIEM. This includes logs that SIRT needs to complete investigations.
Capacity planning and forecasting of licensing and infrastructure costs, as a shared responsibility with Security Logging
SIRT is responsible for
- The management and ownership of Panther until it is decommissioned
- Ensuring that they have the logging data needed to effectively and efficiently execute their responsibilities as incident responders
- Documenting gaps in logs and using the prioritization procedure to ensure prioritization meets the standards set by the procedure
- Configuring automation, alerting, and monitoring specific to their needs for incident response
- Guiding and informing the InfraSec and AppSec teams on logging data critical to SIRT through collaboration and maintenance of the development of the Security Logging Standard
- Reporting to the Security Logging Team when logs:
- Are no longer needed
- Are incomplete
- Need format improvements or corrections
- Require more verbosity
- Have efficiency opportunities (e.g. we only use a small portion of sizable logs)
Everyone else across Security is responsible for
- Making requests of the Security Logging Team for logs that you need to execute your responsibilities, we will have a documented process for this soon
- Submitting issues to SIRT when you have detection suggestions where we have a high return on investment
Additional Resources
Onboarding
The purpose of the Critical Logging Tiering Methodology is to support GitLab in identifying and understanding the criticality of logs in systems utilized across the organization that are considered essential in order to maintain operations.
