GCP instance groups can be configured in the NSX Advanced Load Balancer pools for a virtual service. The Controller periodically polls GCP instance groups configured in the NSX Advanced Load Balancer pool and update the pool servers with the instances in the GCP instance groups.
In addition to polling, GCP pub-sub notification service is also used to get the notifications of the instances created and deleted in a GCP instance group. If a new instance is added to a GCP instance group which is configured in an NSX Advanced Load Balancer pool, the NSX Advanced Load Balancer updates the pool membership to include the newly provisioned instance. Conversely, upon deletion of the instance from GCP instance group, the NSX Advanced Load Balancer will delete this server from its pool membership. This enables seamless, elastic and automated management of backend server resources without any operator intervention or configuration updates.
The NSX Advanced Load Balancer supports both GCP managed and un-managed instance groups. Instance groups can be in any GCP project but must have required permissions as stated below. For details on the permissions required in GCP, see Roles and Permissions (GCP Full Access). You can configure the service account to have the permissions for GCP Instance Group Auto Scaling feature in Service Engine project and for instance group (server) project.
Configuring Pool through CLI
The following are the steps to configure GCP instance groups through the NSX Advanced Load Balancer CLI:
Set the external_autoscale_groups field in the NSX Advanced Load Balancer pool with the list of GCP instance group in InstanceGroupName@InstanceGroupProjectID format.
The instance groups can be shared across multiple pools.
Example
You can add two GCP instance groups to a pool where both of them belong to different GCP projects. The following are the CLI details:
[admin:controller]: > configure pool pool-1 [admin:controller]: pool> cloud_ref gcp-cloud [admin:controller]: pool> external_autoscale_groups instance-group-name-1@instance-group-project-1 [admin:controller]: pool> external_autoscale_groups instance-group-name-2@instance-group-project-2 [admin:controller]: pool> save +---------------------------------------+-----------------------------------------------------+ | Field | Value | +---------------------------------------+-----------------------------------------------------+ | uuid | pool-ea2ee84d-a51e-451f-b59e-4906a4a0a4e2 | | name | pool-1 | | default_server_port | 80 | | graceful_disable_timeout | 1 min | | connection_ramp_duration | 10 min | | max_concurrent_connections_per_server | 0 | | lb_algorithm | LB_ALGORITHM_LEAST_CONNECTIONS | | lb_algorithm_hash | LB_ALGORITHM_CONSISTENT_HASH_SOURCE_IP_ADDRESS | | inline_health_monitor | True | | use_service_port | False | | capacity_estimation | False | | capacity_estimation_ttfb_thresh | 0 milliseconds | | vrf_ref | global | | fewest_tasks_feedback_delay | 10 sec | | enabled | True | | request_queue_enabled | False | | request_queue_depth | 128 | | host_check_enabled | False | | sni_enabled | True | | rewrite_host_header_to_sni | False | | rewrite_host_header_to_server_name | False | | external_autoscale_groups[1] | instance-group-name-1@instance-group-project-1 | | external_autoscale_groups[2] | instance-group-name-2@instance-group-project-2 | | lb_algorithm_core_nonaffinity | 2 | | lookup_server_by_name | False | | analytics_profile_ref | System-Analytics-Profile | | tenant_ref | admin | | cloud_ref | gcp-cloud | | server_timeout | 0 milliseconds | | delete_server_on_dns_refresh | True | | enable_http2 | False | | ignore_server_port | False | | routing_pool | False | +---------------------------------------+-----------------------------------------------------+
For roles and permissions of auto-scaling Service Engine project, see Roles and Permissions (GCP Full Access).
Configuring Cloud
The following are the steps to configure cloud.
You can configure the polling interval by using autoscale_polling_interval field in the cloud configuration.
It is recommended to increase the polling interval to 5 minutes if GCP pub-sub is configured for the instance group notification. This is required for the reconciliation of the NSX Advanced Load Balancer pool configuration after every periodic interval.
Configuring Cloud through CLI
The following are the CLI details to configure cloud through the NSX Advanced Load Balancer CLI.
[admin:controller]: > configure cloud gcp-cloud Updating an existing object. [admin:controller]: cloud> autoscale_polling_interval 300 Overwriting the previously entered value for autoscale_polling_interval [admin:controller]: cloud> save +------------------------------+--------------------------------------------+ | Field | Value | +------------------------------+--------------------------------------------+ | uuid | cloud-32cd1a1e-bfc0-40f9-940b-1b37408ffa67 | | name | gcp-cloud | | vtype | CLOUD_GCP | | apic_mode | False | | gcp_configuration | | | cloud_credentials_ref | gcp-service-account | | region_name | us-central1 | | zones[1] | us-central1-a | | zones[2] | us-central1-b | | se_project_id | se-project-id | | network_config | | | config | INBAND_MANAGEMENT | | inband | | | vpc_subnet_name | subnet-1 | | vpc_project_id | network-project-id | | vpc_network_name | dev-net-1 | | vip_allocation_strategy | | | mode | ROUTES | | dhcp_enabled | True | | mtu | 1500 bytes | | prefer_static_routes | False | | enable_vip_static_routes | False | | license_type | LIC_CORES | | state_based_dns_registration | True | | ip6_autocfg_enabled | False | | dns_resolution_on_se | False | | enable_vip_on_all_interfaces | False | | tenant_ref | admin | | license_tier | ENTERPRISE | | autoscale_polling_interval | 300 seconds | +------------------------------+--------------------------------------------+
Tracking of Instances in GCP Instance Group
For tracking instance in GCP instance group, the polling of instance groups and notifications from GCP StackDriver logging are used.
Server Updates using GCP Stackdriver Logging and GCP Pub/Sub
The following are the server updates using GCP StackDriver logging and GCP pub/sub:
The NSX Advanced Load Balancer Controller creates one GCP pub-sub topic and one GCP pub-sub subscription in the Service Engine project for each cloud.
The Controller creates a Stackdriver log sink for each instance group in the server project.
Whenever an instance is added or removed from a GCP instance group, a log entry is created in GCP StackDriver logging.
The log entry gets matched with the query of the configured instance groups sink and if it matches, it will be exported to the pub-sub topic in the Service Engine project.
The Controller gets the notification from GCP pub-sub whenever instance is added or removed from the instance group and it updates all the NSX Advanced Load Balancer pools with the configured instance group.
Server Updates through Polling
Periodic polling of the GCP instance groups is done to sync the topic, subscriptions, sinks and servers, if some updates were missed.
The NSX Advanced Load Balancer pool is updated with the servers once the GCP pub-sub notifications are processed by the Controller.
[admin:10-138-10-50]: > show pool pool1 +---------------------------------------+----------------------------------------------------------------------------------+ | Field | Value | +---------------------------------------+----------------------------------------------------------------------------------+ | uuid | pool-ea2ee84d-a51e-451f-b59e-4906a4a0a4e2 | | name | pool1 | | default_server_port | 80 | | graceful_disable_timeout | 1 min | | connection_ramp_duration | 10 min | | max_concurrent_connections_per_server | 0 | | servers[1] | | | ip | 10.20.0.8 | | hostname | instance-group-1-bf52 | | enabled | True | | ratio | 1 | | external_uuid | https://www.googleapis.com/compute/v1/projects/instance-group-project-1/zones/us | | | -central1-c/instances/instance-group-1-bf52 | | verify_network | False | | resolve_server_by_dns | False | | static | False | | rewrite_host_header | False | | autoscaling_group_name | instance-group-1@instance-group-project-1 | | servers[2] | | | ip | 10.20.0.9 | | hostname | instance-group-1-9phd | | enabled | True | | ratio | 1 | | external_uuid | https://www.googleapis.com/compute/v1/projects/instance-group-project-1/zones/us | | | -central1-b/instances/instance-group-1-9phd | | verify_network | False | | resolve_server_by_dns | False | | static | False | | rewrite_host_header | False | | autoscaling_group_name | instance-group-1@instance-group-project-1 | | servers[3] | | | ip | 10.20.0.7 | | hostname | instance-group-1-s078 | | enabled | True | | ratio | 1 | | external_uuid | https://www.googleapis.com/compute/v1/projects/instance-group-project-1/zones/us | | | -central1-f/instances/instance-group-1-s078 | | verify_network | False | | resolve_server_by_dns | False | | static | False | | rewrite_host_header | False | | autoscaling_group_name | instance-group-1@instance-group-project-1 | | lb_algorithm | LB_ALGORITHM_LEAST_CONNECTIONS | | lb_algorithm_hash | LB_ALGORITHM_CONSISTENT_HASH_SOURCE_IP_ADDRESS | | inline_health_monitor | True | | use_service_port | False | | capacity_estimation | False | | capacity_estimation_ttfb_thresh | 0 milliseconds | | vrf_ref | global | | fewest_tasks_feedback_delay | 10 sec | | enabled | True | | request_queue_enabled | False | | request_queue_depth | 128 | | host_check_enabled | False | | sni_enabled | True | | rewrite_host_header_to_sni | False | | rewrite_host_header_to_server_name | False | | external_autoscale_groups[1] | instance-group-1@instance-group-project-1 | | lb_algorithm_core_nonaffinity | 2 | | lookup_server_by_name | False | | analytics_profile_ref | System-Analytics-Profile | | tenant_ref | admin | | cloud_ref | gcp-cloud | | server_timeout | 0 milliseconds | | delete_server_on_dns_refresh | True | | enable_http2 | False | | ignore_server_port | False | | routing_pool | False | +---------------------------------------+----------------------------------------------------------------------------------+
