This section explains the troubleshooting techniques for GCP cloud.

Cloud Configuration Issues

The following are the troubleshooting techniques for cloud configuration issues:

Check the cloud status in NSX Advanced Load Balancer UI. The status should have the reason for the cloud not being up.

  • Invalid Configuration- Ensure that the values in the GCP cloud configuration are correct. NSX Advanced Load Balancer GCP cloud will be in failed state if it fails to validate cloud configuration.

  • Permission Issues - Ensure that the service account configured either in the Controller virtual machine or as part of the NSX Advanced Load Balancer GCP cloud configuration has the required permissions in all the required projects. Refer to GCP Full Access Roles and Permissions for more details.

  • Service Engine Image - After a GCP cloud is successfully created, a Service Engine image should be present in Service Engine GCP project. The prefix of the image name is avi-se and image will have a label named cloudid: <avi-cloud-uuid>.

Service Engine Issues

The following are the troubleshooting techniques for Service Engine issues:

Bootup Issues

  1. If the Service Engine VM is failing to boot up, then check the Machine Type set in the Service Engine group. If the disk allocated is too low, then VM will fail to boot up.

  2. Check the GCP console logs to narrow down the issue.

Service Engine failing to connect to NSX Advanced Load Balancer Controller

  1. Ensure that Service Engine VMs have connectivity to the Controller. Check the VPC settings in the network configuration of the NSX Advanced Load Balancer GCP cloud.

  2. Check the firewall rules on the Service Engine VM and the Controller. The firewall rules should allow NSX Advanced Load Balancer Controller to SE Communication and vice versa. For more information, see Configuring Firewall Rules in GCP.

Virtual Service Issues

The following are the troubleshooting techniques for virtual service issues:

Reachability Issues

  1. The firewall rules for Service Engine VMs should allow traffic from the clients.

  2. The back end servers firewall rules must allow traffic from the Service Engines. For more information, see Configuring Firewall Rules in GCP.

  3. If static routes are configured for the backend server’s reachability from Service Engines, then ensure that the next hop is configured as the gateway of the Service Engine subnet.

VIP as Routes - applicable if VIP allocation strategy mode is Routes

  1. Ensure that the VIP subnet (NSX Advanced Load Balancer Internal Network) does not overlap with any subnet in GCP data VPCs.

  2. VIP subnets (NSX Advanced Load Balancer internal network) must be unique across NSX Advanced Load Balancer clusters.

  3. Verify that there are no out-of-band or stale overlapping routes for the VIP in GCP data VPCs. This can prevent the traffic from reaching the Service Engines for the VIP. NSX Advanced Load Balancer Controller does not manage routes that are not created by it.

VIP as ILB - applicable if VIP allocation strategy mode is ILB

  • In a VPC, ILB VIP subnet cannot be the same as the Service Engine subnet. Ensure that the VIP subnet differs from the Service Engine subnet for that VPC.

  • For ILB healthchecks, firewall rules for Service Engine VMs must allow traffic from the following source IP ranges:

    • 35.191.0.0/16

    • 130.211.0.0/22

  • Ensure that the cloud routers (if configured) are not shared across NSX Advanced Load Balancer clusters.

Logging

Check the following logs for errors at the following locations in NSX Advanced Load Balancer Controller:

  1. /opt/avi/log/cc_agent_.log</code> — These logs are created for each NSX Advanced Load Balancer Cloud. All the operations being performed for the specific cloud are logged here.

  2. /opt/avi/log/glog/cloudconnectorgo — All the cloudconnectorgo service specific logs are logged in these log files. Errors relating to RPC handling and other common infrastructure components are logged here.

  3. /var/log/upstart/cloudconnectorgo.log — All the crashes/ panics are logged in this log file.

Common Errors

The following are the common errors:

projects/project-id/zones/us-central1-b/machineTypes/n2-standard-4 was not found — Ensure that the instance_flavor added in the Service Engine group is available in all the zones configured in the NSX Advanced Load Balancer GCP cloud.

See Available regions and zones to check flavor (machine-type) availability in GCP regions and zones.