Avi Load Balancer is an elastic fabric architecture. Various resources, such as the SEs and application servers, can be scaled up and down on-demand, based on load and capacity requirements.

For the public cloud ecosystems which can provide elastic autoscaling capabilities for workloads, Avi Load Balancer uses these capabilities and even manages their behavior based on the metrics collected by Avi Load Balancer.

Avi Load Balancer provides the following scaling functionality:

  • Scaling the virtual service to more (or fewer) SEs, so that traffic can be serviced by more (or fewer) load-balancing instances as the Avi Load Balancer SEs reach (underutilize) capacity.

  • Scaling the application server pool to more (or fewer) application instances, so that traffic can be serviced by a right-sized back end pool.

Both types of scaling can be performed automatically through pre-set Avi Load Balancer policies, based on load and capacity measurements done using Avi Load Balancer.

Ecosystem Integration

Avi Load Balancer supports the above-mentioned autoscaling features in all ecosystems. This section discusses integration considerations related to the below public clouds:

  • Amazon Web Services

  • Microsoft Azure

Virtual Service Scaling

Each SE has a maximum capacity for processing traffic, typically measured in terms of traffic throughput or SSL transactions per second. The SE capacity is a function of various parameters, such as SE VM size (number of vCPUs, or memory), type of traffic, and the ecosystem in which the SE is functioning.

In the default configuration, a virtual service is placed on a single SE. However, if the SE is not sufficient to handle traffic for the virtual service, the virtual service can be scaled out to added SEs. Here, more than one SE handles traffic for the virtual service.

Scaling out or scaling in virtual services can be performed manually or automatically.

In the case of automated scaling of virtual service placements, one of the following SE parameters can be used to configure thresholds beyond which a virtual service must be scaled out to a new SE, or scaled back into fewer SEs:

  • CPU utilization of the SE

  • Bandwidth, in Mbps, being served by the SE

  • Connections per second (CPS) being served by the SE

  • Packets per second (PPS)

For more information on virtual service scaling, see Virtual Service Scaling.

Application Server Scaling

Along with the virtual service load balancing, it is important to ensure enough capacity is available at the application instance tier to handle traffic loads.

As public cloud infrastructure is charged based on usage or uptime, it is important to have enough capacity based on usage, along with the ability to scale resources on-demand.

Public clouds provide autoscaling features. The templates for autoscaling servers can be used to spawn virtual machines and configure them. The scale-out or scale-in can either be done manually or based on certain load conditions.

The relevant features are:

Avi Load Balancer Integration with Public Cloud

The following are the two variations of Avi Load Balancer support for autoscaling groups:

  • Autoscale decision managed by the public cloud.

  • Autoscale decision managed by Avi Load Balancer.

Autoscale Decision Managed by Public Cloud

In this method of autoscaling, the appropriate autoscaling group is added to the server pool on an Avi Load Balancer Controller. The Controller tracks the autoscaling group. As virtual machine instances are added or removed from the group, Avi Load Balancer adds or removes the virtual machine from its pool member list.

In this manner, Avi Load Balancer distributes traffic requests to the requisite virtual machine instances.

The scaling in or scaling out of the pool is controlled based on policies associated with the autoscale group, and the Controller does not influence this operation.

Autoscale Decision Managed by Avi Load Balancer

In this method of autoscaling, Avi Load Balancer takes over the decision to scale the virtual machine instances. Also, the public cloud autoscale group is added to the Avi Load Balancer server pool.

An autoscale policy is created on the Controller and is associated with the pool. This autoscale policy contains parameters and thresholds for triggering the scale-out and scale-in event, based on a wide range of metrics and alerts that Avi Load Balancer supports.

When the threshold is crossed, the Controller communicates with the public cloud to initiate a scale-out or a scale-in operation and also manages the pool membership.

A key advantage of this method is the ability to use a much richer set of metrics for performing scaling decisions, as compared to the metrics available with the public cloud.

Server Garbage Collection Parameter

Whenever Avi Load Balancer decides on scale-in, any server which is already down will be selected for scale-in.

Also, the down servers can be garbage collected by Avi Load Balancer, after a configured delay. To configure the delay parameter, use the delay_for_server_garbage_collection under the autoscale_policy options.

AZ Aware Autoscaling

While scaling in, Avi Load Balancer autoscale will ensure the balance of servers across different AWS availability zones. For instance, if there are four servers in a pool (two servers each in AZ1 and AZ2), and scale-in happens for two servers, you will be left with two servers, one in each AZ.

Note:

This feature is available only for AWS.

Configuring Autoscale Integration with Avi Load Balancer

For more information on configuring autoscale groups with public clouds, see the following sections:

  • Amazon Web Services (autoscaling managed by public cloud): Avi Load Balancer Integration with AWS Auto Scaling Groups

  • Amazon Web Services (autoscale managed by Avi Load Balancer): See the topic Configuration and Metrics Collection on NSX Advanced Load Balancer for AWS Server Autoscaling in VMware NSX Advanced Load Balancer Installation Guide.

  • Microsoft Azure: See Avi Load BalancerIntegration with Azure VM Scale Setsin Avi Load BalancerInstallation Guide.