The NSX Advanced Load Balancer supports VIP as GCP Internal Load Balancer (ILB). The VIP reachability is through ILB, where VIP is allocated from a GCP subnet and the VIP is the front end IP of the ILB. The ILB back ends have all the Service Engines on which the virtual service is placed. GCP Internal Load Balancer for the virtual service is created in the Service Engine project configured in the cloud. You can configure ILB as VIP for the cloud through the NSX Advanced Load Balancer CLI or REST APIs. This topic explains the creation of GCP cloud with inband network configuration.
For other network configurations, see GCP Cloud Network Configuration.
Configuring VIP as Internal Load Balancer
You need to set vip_allocation_strategy field in GCP cloud configuration to ILB. All the VIPs created in this cloud are allocated from GCP subnet and virtual services are configured as ILB with front end IP as the VIP.
Creating GCP Credentials
The following are the steps to create GCP credentials:
Navigate to and click the Create button.
Specify the user name and the GCP service account key file data. See GCP Full Access Deployment for more details on authentication.
Starting with NSX Advanced Load Balancer 22.1.3, the following UI is available:
Navigate to . Click .
Navigate to the Credentials section under the GCP tab.
Click SET CREDENTIALS.
Click the vertical ellipsis in the Credentials field and then click CREATE.
In the NEW USER CREDENTIALS screen, select GCP from the Credentials Type drop-down menu.
Paste the Service Account Key in JSON format.
Click SAVE to create the credentials.
Creating GCP Cloud
The following are the steps to create GCP cloud:
Login to the NSX Advanced Load Balancer Shell. The following CLI is used:
[admin:10-138-10-49]: > configure cloud gcp-cloud [admin:10-138-10-49]: cloud> vtype cloud_gcp [admin:10-138-10-49]: cloud> gcp_configuration [admin:10-138-10-49]: cloud:gcp_configuration> cloud_credentials_ref gcp-service-account [admin:10-138-10-49]: cloud:gcp_configuration> region_name us-central1 [admin:10-138-10-49]: cloud:gcp_configuration> zones us-central1-a [admin:10-138-10-49]: cloud:gcp_configuration> zones us-central1-b [admin:10-138-10-49]: cloud:gcp_configuration> se_project_id service-engine-project [admin:10-138-10-49]: cloud:gcp_configuration> network_config config inband_management [admin:10-138-10-49]: cloud:gcp_configuration:network_config> inband [admin:10-138-10-49]: cloud:gcp_configuration:network_config:inband> vpc_subnet_name subnet-1 [admin:10-138-10-49]: cloud:gcp_configuration:network_config:inband> vpc_project_id network-project [admin:10-138-10-49]: cloud:gcp_configuration:network_config:inband> vpc_network_name network-1 [admin:10-138-10-49]: cloud:gcp_configuration:network_config:inband> save [admin:10-138-10-49]: cloud:gcp_configuration:network_config> save [admin:10-138-10-49]: cloud:gcp_configuration> vip_allocation_strategy mode ilb [admin:10-138-10-49]: cloud:gcp_configuration:vip_allocation_strategy> ilb [admin:10-138-10-49]: cloud:gcp_configuration:vip_allocation_strategy:ilb> save [admin:10-138-10-49]: cloud:gcp_configuration:vip_allocation_strategy> save [admin:10-138-10-49]: cloud:gcp_configuration> save [admin:10-138-10-49]: cloud> save
+-----------------------------+--------------------------------------------+ | Field | Value | +------------------------------+--------------------------------------------+ | uuid | cloud-90c34ff6-f3fd-4bbd-a6b6-32e725c521ea | | name | gcp-cloud | | vtype | CLOUD_GCP | | apic_mode | False | | gcp_configuration | | | cloud_credentials_ref | gcp-service-account | | region_name | us-central1 | | zones[1] | us-central1-a | | zones[2] | us-central1-b | | se_project_id | service-engine-project | | network_config | | | config | INBAND_MANAGEMENT | | inband | | | vpc_subnet_name | subnet-1 | | vpc_project_id | network-project | | vpc_network_name | network-1 | | vip_allocation_strategy | | | mode | ILB | | dhcp_enabled | True | | mtu | 1500 bytes | | prefer_static_routes | False | | enable_vip_static_routes | False | | license_type | LIC_CORES | | state_based_dns_registration | True | | ip6_autocfg_enabled | False | | dns_resolution_on_se | False | | enable_vip_on_all_interfaces | False | | tenant_ref | admin | | license_tier | ENTERPRISE_18 | | autoscale_polling_interval | 60 seconds | +------------------------------+--------------------------------------------+ [admin:10-138-10-49]: >
Creating Virtual Service
The following are the CLI details to create virtual service:
configure vsvip <name> vip subnet_uuid <subnet-name> auto_allocate_ip save save
[admin:controller]: > configure virtualservice ilb-vs-1 [admin:controller]: virtualservice> cloud_ref gcp-cloud [admin:controller]: virtualservice> pool_ref pool-1 [admin:controller]: virtualservice> services New object being created [admin:controller]: virtualservice:services> port 80 [admin:controller]: virtualservice:services> save [admin:controller]: vsvip_ref <name>
+------------------------------------+-----------------------------------------------------+ | Field | Value | +------------------------------------+-----------------------------------------------------+ | uuid | virtualservice-63190c5f-3e58-4d84-9931-b860f17c20e6 | | name | ilb-vs-1 | | enabled | True | | services[1] | | | port | 80 | | enable_ssl | False | | port_range_end | 80 | | application_profile_ref | System-HTTP | | network_profile_ref | System-TCP-Proxy | | pool_ref | pool-1 | | se_group_ref | Default-Group | | vrf_context_ref | global | | enable_autogw | True | | analytics_profile_ref | System-Analytics-Profile | | weight | 1 | | delay_fairness | False | | max_cps_per_client | 0 | | limit_doser | False | | type | VS_TYPE_NORMAL | | cloud_type | CLOUD_NONE | | use_bridge_ip_as_vip | False | | flow_dist | LOAD_AWARE | | ign_pool_net_reach | False | | ssl_sess_cache_avg_size | 1024 | | remove_listening_port_on_vs_down | False | | close_client_conn_on_config_update | False | | bulk_sync_kvcache | False | | tenant_ref | admin | | cloud_ref | gcp-cloud | | east_west_placement | False | | scaleout_ecmp | True | | active_standby_se_tag | ACTIVE_STANDBY_SE_1 | | flow_label_type | NO_LABEL | | vip[1] | | | vip_id | 0 | | ip_address | 10.138.0.134 | | enabled | True | | network_ref | subnet-2 | | port_uuid | avi-ilbip-32e725c521ea-fa4d0dada487 | | subnet_uuid | subnet-2 | | subnet | 10.138.0.128/25 | | auto_allocate_ip | True | | auto_allocate_floating_ip | False | | avi_allocated_vip | True | | avi_allocated_fip | False | | ipam_network_subnet | | | network_ref | subnet-2 | | subnet | 10.138.0.128/25 | | subnet_uuid | subnet-2 | | auto_allocate_ip_type | V4_ONLY | | vsvip_ref | vsvip-P4fAJB | | use_vip_as_snat | False | | traffic_enabled | True | | allow_invalid_client_cert | False | +------------------------------------+-----------------------------------------------------+
You can create the VIP 10.138.0.134 as Internal Load Balancer in GCP with Service Engines as back end.
Health check probes are in the address range of 130.211.0.0/22 and 35.191.0.0/16. You need to add firewall rules to allow these addresses.
If all instances are unhealthy in the back end service, the ILB will load balance the traffic among all instances.
Limitations
The following are the limitations of ILB:
Though ILB supports UDP and TCP, in the case of UDP, the health check type is not UDP (due to Google limitation) and so the fail-over time is higher.
The virtual IP cannot be shared by other virtual services as the forwarding ports cannot be updated. A new forwarding rule with the same IP but different port is not allowed in GCP.
A virtual service can only have up to 5 ports as the forwarding rule can have only 5 ports.
The health check is done on the same port as the VIP and not on the instance IP.
If the Service Engine is in N number of back end services, it receives N health check probes per health check interval.
Ensure that the VIP is not configured in the same subnet as that of the SE.
Bring Your Own IP (BYOIP)
With BYOIP you can use public IPs in GCP and use that to load-balance traffic across backed servers.
Configuring BYOIP
The following are the steps to configure BYOIP:
Configure floating IP for the virtual service whose VIP is an Internal Load Balancer in GCP. This adds a static route in GCP with destination range as the public IP, next hop as the internal Load Balancer and with front end IP as the VIP.
Advertise the public IP used in GCP though GCP cloud routers.
Configuring Cloud Routers
The following are the steps to configure cloud routers:
Create the cloud router in the network project and in the region where the Service Engine virtual machines are created. For more information, see Creating Cloud Routers.
You can add multiple cloud routers for an NSX Advanced Load Balancer cloud. It will be updated with the FIP.
Add the cloud router names in the internal load balancer configuration of vip_allocation_strategy field of the cloud configuration.
The following is the CLI to configure cloud routers:
[admin:10-138-10-49]: > configure cloud gcp-cloud Updating an existing object. Currently, the object is: [admin:10-138-10-49]: cloud> gcp_configuration [admin:10-138-10-49]: cloud:gcp_configuration> vip_allocation_strategy [admin:10-138-10-49]: cloud:gcp_configuration:vip_allocation_strategy> ilb [admin:10-138-10-49]: cloud:gcp_configuration:vip_allocation_strategy:ilb> [admin:10-138-10-49]: cloud:gcp_configuration:vip_allocation_strategy:ilb> cloud_router_names cloud-router-1 [admin:10-138-10-49]: cloud:gcp_configuration:vip_allocation_strategy:ilb> save [admin:10-138-10-49]: cloud:gcp_configuration:vip_allocation_strategy> save [admin:10-138-10-49]: cloud:gcp_configuration> save [admin:10-138-10-49]: cloud> save
+------------------------------+--------------------------------------------+ | Field | Value | +------------------------------+--------------------------------------------+ | uuid | cloud-90c34ff6-f3fd-4bbd-a6b6-32e725c521ea | | name | gcp-cloud | | vtype | CLOUD_GCP | | apic_mode | False | | gcp_configuration | | | cloud_credentials_ref | gcp-service-account | | region_name | us-central1 | | zones[1] | us-central1-a | | zones[2] | us-central1-b | | se_project_id | development-237409 | | network_config | | | config | INBAND_MANAGEMENT | | inband | | | vpc_subnet_name | subnet-11 | | vpc_project_id | development-237409 | | vpc_network_name | dev-vnet-1 | | vip_allocation_strategy | | | mode | ILB | | ilb | | | cloud_router_names[1] | cloud-router-1 | | dhcp_enabled | True | | mtu | 1500 bytes | | prefer_static_routes | False | | enable_vip_static_routes | False | | license_type | LIC_CORES | | state_based_dns_registration | True | | ip6_autocfg_enabled | False | | dns_resolution_on_se | False | | enable_vip_on_all_interfaces | False | | tenant_ref | admin | | license_tier | ENTERPRISE_18 | | autoscale_polling_interval | 60 seconds | +------------------------------+--------------------------------------------+
Adding Floating IP to the Virtual Service
The following is the CLI to add floating IP to the virtual service. This creates a static route in GCP and updates the cloud routers with the public IP.
[admin:10-138-10-49]: > configure virtualservice ilb-vs-1 Updating an existing object. Currently, the object is: [admin:10-138-10-49]: virtualservice> vip index 1 [admin:10-138-10-49]: virtualservice:vip> floating_ip 35.78.2.109 [admin:10-138-10-49]: virtualservice:vip> save [admin:10-138-10-49]: virtualservice> save
+------------------------------------+-----------------------------------------------------+ | Field | Value | +------------------------------------+-----------------------------------------------------+ | uuid | virtualservice-63190c5f-3e58-4d84-9931-b860f17c20e6 | | name | ilb-vs-1 | | enabled | True | | services[1] | | | port | 80 | | enable_ssl | False | | port_range_end | 80 | | application_profile_ref | System-HTTP | | network_profile_ref | System-TCP-Proxy | | pool_ref | pool-1 | | se_group_ref | Default-Group | | vrf_context_ref | global | | enable_autogw | True | | analytics_profile_ref | System-Analytics-Profile | | weight | 1 | | delay_fairness | False | | max_cps_per_client | 0 | | limit_doser | False | | type | VS_TYPE_NORMAL | | cloud_type | CLOUD_NONE | | use_bridge_ip_as_vip | False | | flow_dist | LOAD_AWARE | | ign_pool_net_reach | False | | ssl_sess_cache_avg_size | 1024 | | remove_listening_port_on_vs_down | False | | close_client_conn_on_config_update | False | | bulk_sync_kvcache | False | | tenant_ref | admin | | cloud_ref | gcp-cloud | | east_west_placement | False | | scaleout_ecmp | True | | active_standby_se_tag | ACTIVE_STANDBY_SE_1 | | flow_label_type | NO_LABEL | | vip[1] | | | vip_id | 0 | | ip_address | 10.138.0.134 | | enabled | True | | network_ref | subnet-2 | | port_uuid | avi-ilbip-32e725c521ea-fa4d0dada487 | | subnet_uuid | subnet-2 | | subnet | 10.138.0.128/25 | | auto_allocate_ip | True | | floating_ip | 35.78.2.109 | | auto_allocate_floating_ip | False | | avi_allocated_vip | True | | avi_allocated_fip | False | | ipam_network_subnet | | | network_ref | subnet-2 | | subnet | 10.138.0.128/25 | | subnet_uuid | subnet-2 | | auto_allocate_ip_type | V4_ONLY | | vsvip_ref | vsvip-P4fAJB | | use_vip_as_snat | False | | traffic_enabled | True | | allow_invalid_client_cert | False | +------------------------------------+-----------------------------------------------------+
Limitations
Multiple clusters cannot share the same cloud router as the update cannot be coordinated between them.
Multiple NSX Advanced Load Balancer clouds cannot share the same cloud router.
If a cloud router is removed from the cloud configuration, you need to manually delete the NSX Advanced Load Balancer Floating IPs (FIP) (on the cloud router) from the cloud router in GCP, as the controller would no longer manage that router.
Troubleshooting Cloud Router
For each cloud router update, a corresponding event is generated as follows:
GCP_CLOUD_ROUTER_UPDATE_SUCCESS- An internal event that is generated each time the cloud router is successfully updated.GCP_CLOUD_ROUTER_UPDATE_FAIL- An external event that is generated when an update for the cloud router fails.
The cloud router can also have other IPs which were not created by NSX Advanced Load Balancer. These IPs remain as is and all the IPs that are configured by NSX ALB in the cloud router will have a description to indicate that they are added by NSX Advanced Load Balancer.
If a FIP is not getting advertised, you can view the custom IP ranges in cloud router details as follows:
Navigate to .
Click the cloud router required.
Click Custom IP Ranges tab to view the IP range and description as shown below:
If the FIP is not listed under custom IP ranges, you can view the Events page for the Controller in NSX Advanced Load Balancer UI as follows:
Navigate to .
Click the
GCP_CLOUD_ROUTER_UPDATE_FAILevent to view the error message as shown below:
Updating SE Project ID
SE Project ID field in a cloud specifies the project in which SEs are created. Once you set this field, you can amend this field only if the following conditions are met:
VIPAllocationStrategy - ILB
There are no virtual services or SEs present in the cloud.
No pool is configured with auto-scaling groups.
VIPAllocationStrategy - Routes
All virtual services are created with VIPAllocationMode set to
Routes.None of the virtual services has public IP allocated.
The following is the list of areas impacted on changing the SE Project ID:
New SE creation will always happen in the new SE project.
The networks specified in the network_config option in the cloud must be shared with the new SE project as well.
The existing SEs in the older project will be managed by the product.
The NSX Advanced Load Balancer cleans up images in the old SE project.
The NSX Advanced Load Balancer cleans up the topic and subscription in the old SE project.
