The auto-rebalance feature helps in automatically migrating or scaling virtual services when the load on the Service Engines goes beyond or falls below the configured threshold values.
The following are few of the trigger types aggregated at an SE level:
Packets per second (PPS)
Throughput in Mbps
Open connections
CPU
The minimum and the maximum threshold is configured along with one of these options for the trigger type. By default, auto-rebalance is based on CPU trigger type.
In auto-rebalancing without DNS-based autoscaling, additional SEs can be added under the same virtual service IP address based on the load.
For certain deployments, every virtual service IP address has primary SEs and secondary SEs associated with it. When the capacity of primary SEs reaches its threshold, traffic does not get re-directed to the secondary SE as the primary SEs are stills serving incoming requests.
Limitations
Scaling is limited, especially in tunnel mode.
In AWS deployments, primary SE limitations (PPS) bottleneck the performance, independent of the number of secondary SEs.
The above limitation can be avoided by creating multiple virtual services or multi-VIP virtual services. So, the third construct of DNS-based SE auto-rebalancing is useful.
Auto-rebalancing
With SE auto-rebalancing feature, the virtual service resolves to multiple IP addresses.
The auto-rebalance feature without DNS-based autoscaling, performs load balance only for the given IP addess for the virtual service. With SE auto-rebalance, DNS takes care of the load balancing decision. Based on the DNS resolution for the virtual service, incoming traffic is redirected to multiple IPs for the same virtual service. Once the primary VIP reaches the threshold value for different triggers, the secondary VIP will be spun up. Once the secondary VIPs get saturated, the third VIP will be spun up.
For this feature to work, a VIP autoscale group is created with a Service Engine group.
Autoscaling in AWS
In case a particular zone goes down, VIP autoscaling automatically tries to spin VIP in other zones to maintain the scale. Once the other zone comes up, VIP autoscaling moves the VIPs back to that zone so that VIPs are equally distributed across the zones.
DNS Integration
As the VIPs get added and deleted, the DNS entries are automatically updated correspondingly. When a VIP is automatically added, the corresponding DNS entry is added only after that VIP is in OPER_UP state. Before a VIP is deleted, DNS entry is withdrawn prior to VIP deletion.
Autoscaling Decision
The autoscaling decision is based on traffic and it is controlled by the NSX Advanced Load Balancer Controller (assuming it is in limits of VIP autoscale policy).
The following are the considerations while adding a new VIP.
VIPs are equally distributed across zones (for HA purpose). The NSX Advanced Load Balancer Controller checks for zones that have SEs least loaded. This helps in cutting down new instance cost.
If DNS load balancing is performed in a round-robin fashion, it is possible that over time, VIP load might be skewed with one VIP occupying a full SE and another sharing with another virtual service. In such scenarios, VIP which is sharing the load might need to be moved to another SE. Another solution to address this problem is to use a weighted DNS load balancing algorithm and adjust the weight so that VIPs are equally loaded in terms of SE size.
The location of back-end servers (in case of static pool list and not server autoscaling). This is needed to maximize the traffic within a zone and reduce cross-zone traffic.
High Availability
Based on HA, the NSX Advanced Load Balancer Controller selects the availability zone and the subnet in which the new VIP will be added or deleted. VIPs are distributed equally across the availability zones. During a scale-out request, the list of existing VIPs and the VIP autoscale configuration are checked to determine the new VIP auto-scale zone to create the new VIP.
Configuring SE Auto-rebalancing
Prerequisites
At least one subnet is required to launch the VIP (preferably more for HA).
IPAM capability (in the case of AWS, it is provided by AWS).
Create an SE Group and virtual service and associate these to a VIP autoscale group.
Create the virtual service with at least one virtual service IP address. Currently, creating a virtual service without an IP address is not allowed.
Using CLI
Login to Avi CLI and go to the configure serviceenginegroup mode and use the vip_asg command to configure asg (autoscaling group) zones and asg policy as shown below.
[admin:controller]: > configure serviceenginegroup Default-Group Updating an existing object. Currently, the object is: +---------------------------------------+---------------------------------------------------------+ | Field | Value | +---------------------------------------+---------------------------------------------------------+ | uuid | serviceenginegroup-1a591c76-89d1-40fa-b1ae-39eb23731b47 | | name | Default-Group | [admin:controller]: serviceenginegroup> [admin:controller]: serviceenginegroup> vip_asg [admin:controller]: serviceenginegroup:vip_asg> configuration [admin:controller]: serviceenginegroup:vip_asg:configuration> zones subnet_uuid subnet-62f1b707 New object being created [admin:controller]: serviceenginegroup:vip_asg:configuration:zones> save [admin:controller]: serviceenginegroup:vip_asg:configuration> zones subnet_uuid subnet-782f480f New object being created [admin:controller]: serviceenginegroup:vip_asg:configuration:zones> save [admin:controller]: serviceenginegroup:vip_asg:configuration> where Tenant: admin Cloud: aws-cloud +---------------+-----------------+ | Field | Value | +---------------+-----------------+ | zones[1] | | | subnet_uuid | subnet-62f1b707 | | zones[2] | | | subnet_uuid | subnet-782f480f | +---------------+-----------------+ [admin:controller]: serviceenginegroup:vip_asg:configuration> sav [admin:controller]: serviceenginegroup:vip_asg> sav [admin:controller]: serviceenginegroup> sav +---------------------------------------+---------------------------------------------------------+ | Field | Value | +---------------------------------------+---------------------------------------------------------+ | uuid | serviceenginegroup-1a591c76-89d1-40fa-b1ae-39eb23731b47 | | name | Default-Group | [admin:controller]: > show serviceenginegroup Default-Group | vip_asg | | | configuration | | | zones[1] | | | availability_zone | us-west-2a | | subnet_uuid | subnet-62f1b707 | | fip_capable | False | | zones[2] | | | availability_zone | us-west-2b | | subnet_uuid | subnet-782f480f | | fip_capable | False |
Set the minimum and maximum size for SE for auto scaling policy to get trigger and set the value for DNS cooldown period.
[admin:controller]: serviceenginegroup> [admin:controller]: serviceenginegroup> vip_asg [admin:controller]: serviceenginegroup:vip_asg> policy [admin:controller]: serviceenginegroup:vip_asg:policy> min_size 2 [admin:controller]: serviceenginegroup:vip_asg:policy> max_size 10 [admin:controller]: serviceenginegroup:vip_asg:policy> dns_cooldown 120 [admin:controller]: serviceenginegroup:vip_asg:policy> save [admin:controller]: serviceenginegroup:vip_asg> save [admin:controller]: serviceenginegroup> save +---------------------------------------+---------------------------------------------------------+ | Field | Value | +---------------------------------------+---------------------------------------------------------+ | uuid | serviceenginegroup-1a591c76-89d1-40fa-b1ae-39eb23731b47 | | name | Default-Group | [admin:controller]: > show serviceenginegroup Default-Group | vip_asg | | | policy | | | min_size | 2 | | max_size | 10 | | dns_cooldown | 120 seconds | | suspend | False | | configuration | | | zones[1] | | | availability_zone | us-west-2a | | subnet_uuid | subnet-62f1b707 | | fip_capable | False | | zones[2] | | | availability_zone | us-west-2b | | subnet_uuid | subnet-782f480f | | fip_capable | False | [admin:controller]: > configure serviceenginegroup Default-Group Updating an existing object. Currently, the object is: [admin:controller]: serviceenginegroup> auto_rebalance No change in field value [admin:controller]: serviceenginegroup> save
Configuring VIP Autoscale Subnet using CLI
Autoscale group for an SE group can also be configured using the NSX Advanced Load Balancer UI.Navigate to and select the Advanced option. Select the option from the drop-down menu available for VIP Autoscale Subnet.
