This topic describes the preparation steps required to install VMware Tanzu Operations Manager (Ops Manager) on Google Cloud Platform (GCP).
Before you prepare your Ops Manager installation, do the following depending on the runtime you intend to deploy:
If you are deploying VMware Tanzu Application Service for VMs (TAS for VMs), see Ops Manager on GCP Requirements.
If you are deploying Enterprise VMware Tanzu Kubernetes Grid Integrated Edition (TKGI), see GCP Prerequisites and Resource Requirements.
This section outlines high-level infrastructure options for Ops Manager on GCP. A Ops Manager deployment includes Ops Manager and your chosen runtime. For example, both Ops Manager with TAS for VMs and Ops Manager with TKGI are Ops Manager deployments. For more information, review the deployment options and recommendations in Reference Architecture for Ops Manager on GCP.
You can deploy Ops Manager using one of two main configurations on a GCP virtual private cloud (VPC):
See Shared vs Single-Project VPCs in Reference Architecture for Ops Manager on GCP for a full discussion and recommendations.
When deploying Ops Manager on GCP, VMware recommends using the following GCP components:
Ops Manager uses IAM service accounts to access GCP resources.
For a single-project installation: Complete the following steps to create a service account for Ops Manager.
For a shared-VPC installation: Complete the following steps twice, to create a host account and service account for Ops Manager.
From the GCP console, select IAM & Admin, then Service accounts.
Click Create Service Account:
bosh
.Role: Use the drop-down menu, select the following roles:
Note: You must scroll down in the pop-up windows to select all required roles.
The Service Account User role is required only if you plan to use The Ops Manager VM Service Account to deploy Ops Manager. For more information about The Ops Manager VM Service Account, see Step 2: Google Cloud Platform Config in Configuring BOSH Director on GCP.
Service account ID: The field automatically generates a unique ID based on the username.
Furnish a new private key: Select this checkbox and JSON as the Key type.
Click Create. Your browser automatically downloads a JSON file with a private key for this account. Save this file in a secure location.
Note: You can use this service account to configure file storage for TAS for VMs. For more information, see GCP in Configuring File Storage for TAS for VMs.
Ops Manager manages GCP resources using the Google Compute Engine and Cloud Resource Manager APIs. To enable these APIs:
Log in to the Google Developers Console at https://console.developers.google.com.
In the console, navigate to the GCP projects where you want to install Ops Manager.
Select API Manager > Library.
Under Google Cloud APIs, select Compute Engine API.
On the Google Compute Engine API page, click Enable.
In the search field, enter Google Cloud Resource Manager API
.
On the Google Cloud Resource Manager API page, click Enable.
To verify that the APIs have been enabled, perform the following steps:
Log in to GCP using the IAM service account you created in Set up IAM Service Accounts:
$ gcloud auth activate-service-account --key-file JSON_KEY_FILENAME
List your projects:
$ gcloud projects list PROJECT_ID NAME PROJECT_NUMBER my-host-project-id my-host-project-name ############## my-service-project-id my-service-project-name ##############This command lists the projects where you enabled Google Cloud APIs.
Log in to the GCP console.
Navigate to the GCP project where you want to install Ops Manager. For a shared-VPC installation, navigate to the host project.
Select VPC network, then CREATE VPC NETWORK.
In the Name field, enter a name of your choice for the VPC network. This name helps you identify resources for this deployment in the GCP console. Network names must be lowercase. For example, pcf-virt-net
.
Under Subnets, complete the form as follows to create an infrastructure subnet for Ops Manager and NAT instances:
Name | pcf-infrastructure-subnet-GCP-REGION Example: pcf-infrastructure-subnet-us-west1 |
---|---|
Region | A region that supports three availability zones. For help selecting the correct region for your deployment, see the Google documentation about regions and zones. |
IP address range | A CIDR ending in /26 Example: 192.168.101.0/26 |
See the following image for an example:
Note: For deployments that do not use external IP addresses, enable Private Google access to allow your runtime to make API calls to Google services.
Click Add subnet to add a second subnet for the BOSH Director and components specific to your runtime. Complete the form as follows:
Name | pcf-RUNTIME-subnet-GCP-REGION Example: pcf-pas-subnet-us-west1 |
---|---|
Region | The same region you selected for the infrastructure subnet |
IP address range | A CIDR ending in /22 Example: 192.168.16.0/22 |
Click Add subnet to add a third Subnet with the following details:
Name | pcf-services-subnet-GCP-REGION Example: pcf-services-subnet-us-west1 |
---|---|
Region | The same region you selected for the previous subnets |
IP address range | A CIDR in /22 Example: 192.168.20.0/22 |
See the following image for an example:
Under Dynamic routing mode, leave Regional selected.
Click Create.
Use NAT instances when you want to expose only a minimal number of public IP addresses.
Creating NAT instances permits internet access from cluster VMs. You might, for example, need this internet access for pulling Docker images or enabling internet access for your workloads.
For more information, see Reference Architecture for Ops Manager on GCP and the GCP documentation.
In the GCP console, with your single project or shared-VPC host project selected, navigate to Compute Engine > VM instances.
Click CREATE INSTANCE.
Complete the following fields:
pcf-nat-gateway-pri
. us-west1
, select zone us-west1-a
.n1-standard-4
.Ubuntu 14.04 LTS
.Expand the additional configuration fields by clicking Management, disks, networking, SSH keys.
In the Startup script field under Automation, enter the following text:
#! /bin/bash
sudo sysctl -w net.ipv4.ip_forward=1
sudo sh -c 'echo net.ipv4.ip_forward=1 >> /etc/sysctl.conf'
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
Click Networking to open additional network configuration fields:
nat-traverse
and pcf-nat-instance
.pcf-virt-net
. You created this network in Step 1: Create a GCP Network with Subnets.pcf-infrastructure-subnet-GCP-REGION
.For Primary internal IP, select Ephemeral (Custom)
. Enter an IP address, for example, 192.168.101.2
, in the Custom ephemeral IP address field. The IP address must meet the following requirements:
pcf-infrastructure-subnet-GCP-REGION
subnet..1
through .9
addresses in the CIDR range you set for the pcf-infrastructure-subnet-GCP-REGION
subnet..1
address in the CIDR range you set for the pcf-infrastructure-subnet-GCP-REGION
subnet.For External IP, select Ephemeral
.
Note: If you select a static external IP address for the NAT instance, then you can use the static IP to further secure access to your CloudSQL instances.
Set IP forwarding to On
.
Click Create to finish creating the NAT instance.
Repeat steps 2–6 to create two additional NAT instances with the names and zones specified in the table below. The rest of the configuration remains the same.
Instance 2 | Name | pcf-nat-gateway-sec |
---|---|---|
Zone | Select the second zone from your region. Example: For region us-west1 , select zone us-west1-b . |
|
Internal IP | Select Custom and enter an IP address in the Internal IP address field. Example: 192.168.101.3 . As described above, this address must in the CIDR range you set for the pcf-infrastructure-subnet-GCP-REGION subnet, must exist in a reserved IP range set later in BOSH Director, and cannot be the same as the Gateway IP address set later in Ops Manager. |
|
Instance 3 | Name | pcf-nat-gateway-ter |
Zone | Select the third zone from your region. Example: For region us-west1 , select zone us-west1-c . |
|
Internal IP | Select Custom and enter an IP address in the Internal IP address field. Example: 192.168.101.4 . As described above, this address must in the CIDR range you set for the pcf-infrastructure-subnet-GCP-REGION subnet, must exist in a reserved IP range set later in BOSH Director, and cannot be the same as the Gateway IP address set later in Ops Manager. |
Navigate to VPC Networks > Routes.
Click CREATE ROUTE.
Complete the form as follows:
pcf-nat-pri
pcf-virt-net
0.0.0.0/0
800
pcf
Specify an instance
pcf-nat-gateway-pri
Click Create to finish creating the route.
Repeat steps 2–4 to create two additional routes with the names and next hop instances specified in the table below. The rest of the configuration remains the same.
Route 2 | Name: pcf-nat-sec Next hop instance: pcf-nat-gateway-sec |
---|---|
Route 3 | Name: pcf-nat-ter Next hop instance: pcf-nat-gateway-ter |
GCP lets you assign tags to VM instances and create firewall rules that apply to VMs based on their tags. For more information about tags, see Labeling Resources in the GCP documentation. This step assigns tags and firewall rules to Ops Manager components and VMs that handle incoming traffic.
With your single project or shared-VPC host project selected, navigate to the Networking > VPC network pane and select Firewall rules.
Apply the firewall rules in the following table:
Firewall Rules | |
---|---|
Rule 1 | This rule allows SSH from public networks. Name: pcf-allow-ssh Network: pcf-virt-net Allowed protocols and ports: tcp:22 Source filter: IP ranges Source IP ranges: 0.0.0.0/0 Target tags: allow-ssh |
Rule 2 | This rule allows HTTP from public networks. Name: pcf-allow-http Network: pcf-virt-net Allowed protocols and ports: tcp:80 Source filter: IP ranges Source IP ranges: 0.0.0.0/0 Target tags: allow-http , router |
Rule 3 | This rule allows HTTPS from public networks. Name: pcf-allow-https Network: pcf-virt-net Allowed protocols and ports: tcp:443 Source filter: IP ranges Source IP ranges: 0.0.0.0/0 Target tags: allow-https , router |
Rule 4 | This rule allows GoRouter health checks. Name: pcf-allow-http-8080 Network: pcf-virt-net Allowed protocols and ports: tcp:8080 Source filter: IP ranges Source IP Ranges: 0.0.0.0/0 Target tags: router |
Rule 5 | This rule allows communication between BOSH-deployed jobs. Name: pcf-allow-pas-all Network: pcf-virt-net Allowed protocols and ports: tcp;udp;icmp Source filter: Source tags Target tags: pcf , pcf-opsman , nat-traverse Source tags: pcf , pcf-opsman , nat-traverse |
Rule 6 (Optional) | This rule allows access to the TCP router. Name: pcf-allow-cf-tcp Network: pcf-virt-net Source filter: IP ranges Source IP ranges: 0.0.0.0/0 Allowed protocols and ports: tcp:1024-65535 Target tags: pcf-cf-tcp |
Rule 7 (Optional) | This rule allows access to the SSH proxy. Name: pcf-allow-ssh-proxy Network: pcf-virt-net Source filter: IP ranges Source IP ranges: 0.0.0.0/0 Allowed protocols and ports: tcp:2222 Target tags: pcf-ssh-proxy , diego-brain |
Note: If you want your firewall rules to only permit traffic within your private network, modify the Source IP Ranges from the table accordingly.
If you are only using your GCP project to deploy Ops Manager, then you can delete the following default firewall rules:
default-allow-http
default-allow-https
default-allow-icmp
default-allow-internal
default-allow-rdp
default-allow-ssh
If you are deploying TKGI only, continue to Next Steps.
If you are deploying TAS for VMs or other runtimes, proceed to the following step.
For a shared-VPC installation, select the service project in the GCP console. This step and the following steps allocate resources to the service project, not the host project.
From the GCP console, select SQL and click CREATE INSTANCE.
Ensure MySQL is selected and click Next.
Under MySQL, select instance type Second Generation.
Click Configure MySQL under your choice for instance type: Development, Staging, or Production.
Configure the instance as follows:
pcf-pas-sql
Authorize Networks: Click Add network and create a network named all
that allows traffic from 0.0.0.0/0
.
Note: If you assigned static IP addresses to your NAT instances, you can instead limit access to the database instances by specifying the NAT IP addresses.
Click Create.
Navigate to the Instances page and select the database instance you just created.
Select the Databases tab.
Click Create database to create the following databases:
account
app_usage_service
autoscale
ccdb
console
diego
locket
networkpolicyserver
nfsvolume
notifications
routing
silk
uaa
credhub
Select the USERS tab.
Click Create user account to create a unique username and password for each database you created above. For Host name, select Allow any host. You must create a total of fourteen user accounts.
Note: Ensure that the networkpolicyserver database user has the ALL PRIVILEGES
permission.
With your single project or shared-VPC service project selected in the GCP console, select Storage > Browser.
Using CREATE BUCKET, create buckets with the following names. For Default storage class, select Multi-Regional:
PREFIX-pcf-buildpacks
PREFIX-pcf-droplets
PREFIX-pcf-packages
PREFIX-pcf-resources
PREFIX-pcf-backup
Where PREFIX
is a prefix of your choice, required to make the bucket name unique.
For load balancing, you can use a global HTTP load balancer or an internal, regional load balancer with a private IP address.
Single-project, standalone installations typically use a global HTTP load balancer. For more information, see Create HTTP Load Balancer below for how to set this up.
Shared-VPC installation typically use an internal TCP/UDP load balancer to minimize public IP addresses. For more information, see Create Internal Load Balancer below for how to set this up.
To create an internal load balancer for Ops Manager on GCP, do the following.
Create an internal-facing TCP/UDP load balancer for each region of your Ops Manager deployment.
Note: GCP Internal Load Balancer (iLB) is a regional product. Within the same VPC/network, client VMs in a different region from the iLB cannot access the iLB. For more information, see the GCP documentation.
Assign private IP addresses to the load balancers.
After you have deployed Ops Manager, follow instructions in Create or Update a VM Extension to add a custom VM extension that applies internal load balancing to all VMs deployed by BOSH.
For example, the following manifest code adds a VM extension backend-pool
to Ops Manager VMs:
vm_extensions:
- name: backend-pool
cloud_properties:
ephemeral_external_ip: true
backend_service:
name: name-of-backend-service
scheme: INTERNAL
To create a global HTTP load balancer for Ops Manager on GCP:
Navigate to Compute Engine > Instance groups.
Click CREATE INSTANCE GROUP.
Complete the form as follows:
pcf-http-lb
us-west1
, select zone us-west1-a
.pcf-virt-net
.pcf-pas-subnet-my-gcp-region
subnet that you created previously.Create a second instance group with the following details:
pcf-http-lb
us-west1
, select zone us-west1-b
.pcf-virt-net
.pcf-pas-subnet-my-gcp-region
subnet that you created previously.Create a third instance group with the following details:
pcf-http-lb
us-west1
, select zone us-west1-c
.pcf-virt-net
.pcf-pas-subnet-my-gcp-region
subnet that you created previously.Navigate to Compute Engine > Health checks.
Click CREATE HEALTH CHECK.
Complete the form as follows:
pcf-cf-public
8080
/health
30
5
10
2
Click Create.
Navigate to Network services > Load balancing.
Click CREATE LOAD BALANCER.
Under HTTP(S) Load Balancing, click Start configuration.
For the Name, enter pcf-global-pcf
.
Select Backend configuration
Complete the form as follows:
pcf-http-lb-backend
.HTTP
.http
.10 seconds
.pcf-http-lb (us-west1-a)
. Click Done.pcf-http-lb (us-west1-b)
. Click Done.pcf-http-lb (us-west1-c)
. Click Done.pcf-cf-public
health check that you created.Click Create.
Click Host and path rules to populate the default fields and a green check mark.
Select Frontend configuration, and add the following:
pcf-cf-lb-http
HTTP
pcf-global-pcf
.80
Click Add Frontend IP and port and add the following:
Note: Skip this step if you do not have either a self-signed or trusted SSL certificate. When you configure the tile for your chosen runtime, you are given the opportunity to create a new self-signed certificate. Upon creating a certificate, you can complete the Add Frontend IP and port section.
pcf-cf-lb-https
HTTPS
pcf-global-pcf
address you create for the previous Frontend IP and Port.443
In the Name field, enter a name for the certificate.
In the Public key certificate field, copy in the contents of your public certificate, or upload your certificate as a .pem file. If the certificate is runtime-generated, copy and paste the generated contents from the runtime’s Certificate field into the BOSH Director Public key certificate field.
Review the completed frontend configuration.
Click Review and finalize to verify your configuration.
Click Create.
The load balancer for tailing logs with WebSockets for Ops Manager on GCP operates on TCP port 443
.
From the GCP console, select Network services > Load balancing > Create load balancer.
Under TCP Load Balancing, click Start configuration.
In the Create a load balancer configuration screen, make the following selections:
Under Connection termination, select No (TCP).
Click Continue.
In the New TCP load balancer window, enter pcf-wss-logs
in the Name field.
Click Backend configuration to configure the Backend service:
pcf-gorouter
8080
/health
30
5
10
2
The Backend configuration section shows a green check mark.Click Frontend configuration to open its configuration window and complete the fields:
TCP
pcf-gorouter-wss
.443
Click Review and finalize to verify your configuration.
Click Create.
From the GCP console, select Network services > Load balancing > Create load balancer.
Under TCP Load Balancing, click Start configuration.
Under Internet facing or internal only, select From Internet to my VMs.
Under Connection termination, select No (TCP).
Click Continue.
In the New TCP load balancer window, enter pcf-ssh-proxy
in the Name field.
Select Backend configuration, and enter the following field values:
None
10%
No health check
Select Frontend configuration, and add the following:
TCP
pcf-ssh-proxy
.2222
(Optional) Review and finalize your load balancer.
Click Create.
Note: This step is optional and only required if you enable TCP routing in your deployment.
To create a load balancer for TCP routing in GCP:
From the GCP console, select Network services > Load balancing > Create load balancer.
Under TCP Load Balancing, click Start configuration.
Under Connection termination, select No (TCP). Click Continue.
On the New TCP load balancer screen, enter a unique name for the load balancer in the Name field. For example, pcf-cf-tcp-lb
.
Select Backend configuration, and enter the following field values:
From the Health check dropdown, create a health check with the following details:
pcf-tcp-lb
80
/health
30
5
10
2
Click Save and continue.
Select Frontend configuration, and add the front end IP and port entry as follows:
TCP
pcf-cf-tcp-lb
.Port: 1024-65535
Click Review and finalize to verify your configuration.
Click Create.
In this step, you redirect queries for your domain to the IP addresses of your load balancers.
Locate the static IP addresses of the load balancers you created in Preparing to Deploy Ops Manager on GCP:
pcf-global-pcf
pcf-wss-logs
pcf-ssh-proxy
pcf-cf-tcp-lb
Note: You can locate the static IP address of each load balancer by clicking its name under Network services > Load balancing in the GCP console.
Log in to the DNS registrar that hosts your domain. Examples of DNS registrars include Network Solutions, GoDaddy, and Register.com.
Create A records with your DNS registrar that map domain names to the public static IP addresses of the load balancers located above:
Create and map this record... | To the IP of this load balancer | Required |
---|---|---|
\*.sys.MY-DOMAIN Example: \*.sys.example.com |
pcf-global-pcf |
Yes |
\*.apps.MY-DOMAIN Example: \*.apps.example.com |
pcf-global-pcf |
Yes |
doppler.sys.MY-DOMAIN Example: doppler.sys.example.com |
pcf-wss-logs |
Yes |
loggregator.sys.MY-DOMAIN Example: loggregator.sys.example.com |
pcf-wss-logs |
Yes |
ssh.sys.MY-DOMAIN Example: ssh.sys.example.com |
pcf-ssh-proxy |
Yes, to allow SSH access to apps |
tcp.MY-DOMAIN Example: tcp.example.com |
pcf-cf-tcp-lb |
No, only set up if you have enabled the TCP routing feature |
Save changes within the web interface of your DNS registrar.
In a terminal window, run the following dig
command to confirm that you created your A record successfully:
dig SUBDOMAIN.EXAMPLE-URL.com
Where SUBDOMAIN.EXAMPLE-URL
is the subdomain for your load balancer.
You should see the A record that you just created:
;; ANSWER SECTION: xyz.EXAMPLE.COM. 1767 IN A 203.0.113.1
(Optional) To prepare for deploying either a TAS for VMs or TKGI tile on GCP, you can download the required runtime tile in advance:
After initiating the tile download, proceed to the next step, Deploying Ops Manager on GCP.