Demo Systems Infrastructure - Networking

Global CIDR Ranges

Each of the region/zone CIDR ranges exist in the 10.128.0.0/9 CIDR range. We have partitioned the /9 CIDR range into /12 CIDR ranges for each of the regions/zones.

How We Allocated Subnets

The standard subnet range for GCP VPC auto subnet mode is 10.128.0.0/9. This leaves the 10.0.0.0/9 range unused which is normally used for corporate networks (does not apply to GitLab). Our instincts would be to avoid conflicts with auto-created subnets and stick to using networks in the 10.0.0.0/9 range.

Since each VPC network has its own subnets, we can use the entire 10.0.0.0/8 range without issues as long as we don’t use “auto subnet mode”. In the spirit of an open source design that applies to many companies, we will stick to the 10.128.0.0/9 subnet range to avoid potential conflicts with corporate networks.

Global vs Regional Routing Tables

When configuring your VPC networks and routing tables, you can choose to create a global routing table with a 10.128.0.0/9 and subnets for each region/zone that can communicate with each other, or configure each region/zone to operate independently with a /12 routing table. Keep in mind that usage of terminology varies by cloud provider. For example, a GCP VPC is a global resource and an AWS VPC has a lot of options that are different in regards to cross-region network design.

High-Availability with Multiple Availability Zones per Region

If you are designing a high-availability/fault-tolerant environment within a region and want to use multiple availability zones, you can choose to use a /12 CIDR range per zone (recommended for simplicity), or partition a /12 CIDR range into a /13 (2 zones) or /14 (3-4 zones). The documentation assumes that you’re using a /12 per zone.

Region/Zone Subnet Ranges

The /12 CIDR for each region/zone includes 16 /16 CIDR ranges. In other words, there are 16 available values for the second octet of the IP address which provides 65,536 IPs per subnet. We have assigned an alpha character a through p to represent the 16 available values.

Demo Systems Management

demosys-mgmt-vpc

Subnets

Terraform Configuration

Learn more in our Terraform US region configuration and our Terraform region VPC module.

cd ~/Sites/gitlab-com/customer-success/demo-systems/infrastructure/demosys-terraform
cd environments/demosys-mgmt/us
terraform init
terraform plan
terraform apply

US Demo Resources

Subnets

Terraform Configuration

Learn more in our Terraform US region configuration and our Terraform region VPC module.

cd ~/Sites/gitlab-com/customer-success/demo-systems/infrastructure/demosys-terraform
cd environments/demosys-saas/us
terraform init
terraform plan
terraform apply

EU Demo Resources

demosys-eu-vpc

TODO

Asia Demo Resources

demosys-asia-vpc

TODO

Future Planning Considerations

We should…

1. have IP range capacity for 10x-100x our current requirements if it’s a scalable service.
2. have fewer subnets with larger CIDR ranges to reduce complexity and plan for growth.
3. have separate subnets for each service that poses security or sandbox risk. For example, we should have a separate subnet for Kubernetes pods that users create or are using for sandbox purposes.
4. avoid re-using subnets across multiple VPCs to prevent human confusion and internal IP address overlapping. Keep in mind that Google Cloud uses internal DNS to avoid the issue with overlapping subnets, however this is a vendor-specific technology and is not something we should depend on with our vendor agnostic design.

View page source - Edit this page - please contribute.