This section explains the design decisions of Advanced Load Balancing for VMware Cloud Foundation.
Deployment Model for the Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-001 |
Initial setup should be done only on one NSX Advanced Load Balancer Controller VM out of the three deployed to create an NSX Advanced Load Balancer Controller cluster. |
NSX Advanced Load Balancer Controller cluster is created from an initialized NSX Advanced Load Balancer Controller which becomes the cluster leader. Follower NSX Advanced Load Balancer Controller nodes need to be uninitialized to join the cluster. |
NSX Advanced Load Balancer Controller cluster creation will fail if more than one NSX Advanced Load Balancer Controller is initialized. |
AVI-CTLR-002 |
Apply vSphere DRS anti-affinity rules for the NSX Advanced Load Balancer Controller cluster nodes.
Note:
For a default management vSphere cluster that consists of four ESXi hosts, you can put in maintenance mode only a single ESXi host at a time. |
Ensure that NSX Advanced Load Balancer Controller VMs are distributed across ESXi hosts . |
You must perform additional configuration to set up an anti-affinity rule. |
AVI-CTLR-003 |
Protect NSX Advanced Load Balancer Controller cluster nodes using vSphere High Availability. |
Supports the availability objectives for the NSX Advanced Load Balancer Controller cluster without requiring manual intervention during an ESXi host failure event. |
None |
Decision ID |
Decision Design |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-004 |
Create an NSX-T Cloud Connector on NSX Advanced Load Balancer Controller for each NSX transport zone requiring load balancing. |
A NSX-T Cloud Connector configured on the NSX Advanced Load Balancer Controller will provide load balancing for workloads belonging to a Transport Zone on NSX-T.
Note:
1. A NSX Transport Zone can be unique to a vCenter cluster, a VI Workload Domain or can be shared across VI workload domains. 2. Multiple NSX-T Cloud connectors can be configured on the NSX Advanced Load Balancer Controller if load balancing is required across multiple Transport Zones configured on NSX-T. |
None |
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-005 |
Choose to deploy the Service Engines on a dedicated edge VI workload domain. |
Allows for centralized placement of the Service Engines. |
Capacity growth might be a challenge. Might not work in all cases due to scale restrictions of the edge VI workload domain. |
AVI-CTLR-006 |
Create separate Service Engine Groups to host Virtual Services from different VI workload domains. |
Allows for application isolation. |
Might require additional Service Engine resources. |
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-VI-VC-001 |
When using two availability zones, add the NSX Advanced Load Balancer Controller cluster nodes to the first availability zone VM group. |
Ensures that, by default, the NSX Advanced Load Balancer Controller cluster nodes are powered on in the primary availability zone hosts group. |
After the implementation of the second availability zone for the management domain, you must update the VM group for the primary availability zone virtual machines to include the NSX Advanced Load Balancer Controller cluster nodes. |
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-VI-VC-002 |
Create a VM groups for the NSX Advanced Load Balancer SE VMs. |
Ensures that the NSX Advanced Load Balancer SE VMs can be managed as a group and added to VM/ Host rules. |
User must add each NSX Advanced Load Balancer SE VM to the primary availability zone. |
AVI-VI-VC-003 |
Create a should-run VM-Host affinity rule to run all NSX Advanced Load Balancer SEs on the group of hosts in the first availability zone. |
Ensures that all NSX Advanced Load Balancer SE VMs are in the first availability zone. |
During normal operation, there would not be any NSX Advanced Load Balancer SEs running in the second availability zone. Therefore all apps would be active in the first availability zone. |
Integration of the Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-007 |
Create one NSX-T Cloud connector object on the Controller per transport zone configured on the NSX manager cluster that requires Load Balancing services.
Note:
Transport zone could be dedicated to a VI workload domain or shared across VI workload domains. |
Provides automated deployment of load-balanced applications through NSX-T Cloud integration. Allows for maximum flexibility, control, and isolation in terms of application deployment. |
None |
AVI-CTLR-008 |
Provide either a overlay-backed NSX segment connected to a Tier-1 logical router or a VLAN-backed NSX segment for the Service Engine management for the NSX-T Cloud of overlay type. |
This network is used for the Controller to the Service Engine connectivity. |
None |
AVI-CTLR-009 |
Provide one or more NSX managed VLAN segments as data networks for the NSX-T Cloud connector of VLAN type.
Note:
A single NSX-T Cloud connector of VLAN type can contain multiple data networks. Each data network should belong to a unique NSX managed VLAN segment. |
The Service Engines are placed on NSX managed VLAN segments. |
None |
AVI-CTLR-010 |
Provide a Tier-1 router and a connected overlay-backed NSX segment as data network for the NSX-T Cloud of overlay type.
Note:
A single NSX-T Cloud connector of overlay type can contain multiple data networks. Each data network must belong to a unique Tier-1 router. |
The Service Engines are placed on Overlay Segments created on these Tier-1 logical router(s). |
None |
AVI-CTLR-011 |
Provide an object name prefix when creating the NSX-T Cloud Connector on the NSX Advanced Load Balancer Controller. |
Used for uniquely identifying NSX-T Cloud Connector created resources on NSX Manager cluster and vCenter Server. |
None |
Isolation Model for Load-Balanced Applications in Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-012 |
Create tenants to provide desired level of isolation for the VMware Cloud Foundation.
Note:
NSX Advanced Load Balancer - Basic Edition does not provide tenant isolation. |
Provides required level of configuration and data plane isolation for workloads. |
Additional Service Engine resources might be required. |
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-013 |
Create multiple Service Engine Groups as desired to isolate applications.
Note:
Some of the criteria for grouping applications in different Service Engine Group(s) could be based on:
|
Allows efficient isolation of applications and allows for better capacity planning. Allows flexibility of life-cycle-management. |
None |
AVI-CTLR-014 |
Create separate set of Service Engine Groups for each VI workload domain and scope the Service Engine Group to the VI workload domain vCenter server.
Note:
|
Allows isolation of the Service Engines across VI workload domains. Enables per VI workload domain life-cycle-management. |
None |
AVI-CTLR-015 |
Configure Service Engine Group for Active/ Active HA mode.
Note:
Legacy Active/ Standby HA mode might be required for certain applications. |
Provides optimum resiliency, performance, and utilization. |
Certain applications might not work in Active/ Active mode. For instance, applications that require preserving client IP. In such cases, use the Legacy Active/ Standby HA mode. |
AVI-CTLR-016 |
Enable 'Dedicated dispatcher CPU' on Service Engine Groups that contain the Service Engine VMs of 4 or more vCPUs.
Note:
This setting should be enabled on SE Groups that are servicing applications that have high network requirement. |
This will enable a dedicated core for packet processing enabling high packet pipeline on the Service Engine VMs.
Note:
By default, the packet processing core also processes load balancing flows. |
None . |
AVI-CTLR-017 |
Set 'Placement across the Service Engines' setting to 'distributed'. |
This allows for maximum fault tolerance and even utilization of capacity. |
Might require more Service Engine VMs as compared to 'compact' placement mode. |
AVI-CTLR-018 |
Enable CPU and Memory reservation on the Service Engine Group. . |
The Service Engines are a critical infrastructure component providing load-balancing services to mission critical applications. |
None |
AVI-CTLR-019 |
Configure a consistent Service Engine Name Prefix that indicates the Service Engine VM for instance, 'avise-xxxx'.
Note:
Where ‘xxxx’ could be used as an arbitrary identifier. |
This allows efficient grouping and filtering. |
None |
AVI-CTLR-020 |
Choose the Service Engine Group mode as Legacy HA Active/ Standby if the Controller is set to use basic edition. . |
NSX Advanced Load Balancer Controller in Basic Edition only supports Legacy HA Active/ Standby mode. |
Applications will not be deployed in an Active/ Active fashion, thereby losing out on elastic capacity management. NSX Advanced Load Balancer Enterprise Edition will allow Active/ Active as well as Legacy Active/ Standby deployments. |
Physical Design of the Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Description |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-PHY-001 |
Provide high performance disks (SSD/ Flash) to hosts that run the Controller VMs. |
The Controllers need high performance disks to process the analytics pipeline. |
None |
AVI-PHY-002 |
Enable AES-NI instructions setting in the BIOS for ESXi hosts. |
AES-NI instruction set provides efficiency in SSL performance. |
Most modern machines have AES-NI enabled by default, if not enabled by default, you need to reboot ESXi hosts to enable this setting. |
AVI-PHY-003 |
Disable C-State and P-State settings in BiOS on the ESXi hosts.
Note:
This is an optional design decision. |
Provides maximum performance. |
This might require a reboot and reconfigure of the BIOS causing an outage for each ESXi host. |
vCenter Design of the Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Description |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-VI-VC-004 |
Create anti-affinity 'VM/ Host' rule that prevents collocation of the Controller VMs. |
vSphere will take care of placing the Controller VMs in a way that always ensures maximum HA for the Controller cluster. |
None |
AVI-VI-VC-005 |
Create a virtual machine group for the Controller VMs. |
Ensures that the Controller VMs can be managed as a group. |
You must add virtual machines to the allocated groups manually. |
AVI-VI-VC-006 |
In vSphere HA, for each Controller and Service Engine VMs, set the restart priority policy to high and host isolation response to disabled. |
This ensures fast recovery for the NSX Advanced Load Balancer. |
None |
AVI-VI-VC-007 |
Create one Content Library on the management domain to store Controller OVA. |
Deploying OVA from the Content Library will be operationally easy to do. |
Might not be necessary if deploying Controller VMs using automation tools such as vRO, Ansible, etc. |
AVI-VI-VC-008 |
Create one Content Library on each of the VI workload domain to store Service Engine OVA. |
The Controller's NSX-T Cloud Connector requires a Content Library configured to create the Service Engines. |
None |
VCenter Server Design of the Advanced Load Balancing for VMware Cloud Foundation
NSX-T Data Center Design of the Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-NSX-001 |
Create or use an NSX-T Manager cluster User/ Role with password with the described privileges.
Note:
It is recommended not to use the local ‘admin’ user of NSX-T Data Center. |
Required for the Controller to perform lifecycle management of the Service Engines.
Note:
Update the NSX-T User Credential on the Controller when password for this user account is rotated. |
None |
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-NSX-002 |
Create necessary NSX DFW and/ or Gateway Firewall rules for the NSX Advanced Load Balancer control plane as described to ensure connectivity from:
|
These firewall rules are needed to allow required communication for the NSX Advanced Load Balancer control plane.
Note:
If DFW is enabled and these rules are not configured, this might result in NSX Advanced Load Balancer control plane not functioning as expected. |
None |
AVI-NSX-003 |
Create necessary NSX DFW and/ or Gateway Firewall rules for the configured load-balanced applications as described to ensure connectivity from:
|
These firewall rules are needed to allow required communication for the configured load-balanced applications.
Note:
If DFW is enabled and these rules are not configured, this might result in the configured load-balanced applications not functioning as expected. |
None |
Licensing VMware Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-021 |
Choose the VMware NSX Advanced Load Balancer Enterprise with Cloud Services licensing tier.
Note:
1. New VMware NSX Advanced Load Balancer deployments running v21.1.3 or later will be setup by default in Enterprise with Cloud Services licensing tier. 2. If running v21.1.2 or earlier, choose the VMware NSX Advanced Load Balancer Enterprise licensing tier. |
Provides full-featured access to the NSX Advanced Load Balancer platform.
Note:
If running v21.1.3 or later, alternative is to either use: i) Enterprise edition licensing tier. This provides a full-featured enterprise feature set but does not give access to Cloud Services and advanced App Security features. ii) Basic edition licensing tier. This provides equivalent functionality of NSX-T Data Center native Load Balancer. If running v21.1.2 or earlier, alternative is to use the Basic edition licensing tier. This provides equivalent functionality of NSX-T Data Center native Load Balancer. |
None |
How to size Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-022 |
Deploy one Controller cluster for each NSX Manager cluster for configuring and managing load balancing services. |
Required to form a highly available Controller cluster. |
None |
AVI-CTLR-023 |
Deploy each node in the Controller cluster with a minimum of 8 vCPUs, 32 GB memory and 216 GB of disk space. |
Support up to 200 virtual services. Support up to 100 NSX Advanced Load Balancer Service Engines. Can scale-up with expansion of the Controller sizes anytime.
Note:
Under sizing, the Controllers can lead to unstable control plane functionality. |
None |
Network Design for Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-VI-VC-010 |
Deploy the Controller cluster nodes on the VMware Cloud Foundation management network. |
Allows for ease of management for the Controllers. Allows for configuring a floating cluster VIP; a single IP address that will be assigned to the cluster leader. Administrative tasks, connectivity to the Service Engines and connectivity to network services will all use this network. |
None |
AVI-NSX-004 |
Configure a management network to deploy the Service Engines. Management network needs to be NSX segment and could be either of:
Note:
This network should have connectivity to the IP addresses of each of the Controllers. |
This is required to configure the Controller NSX-T Cloud Connector. |
None |
AVI-NSX-005 |
Configure one or more data network(s) for the Service Engines to service load-balanced applications. Data networks need to be NSX-T managed and could be either of:
Note:
For overlay-backed NSX segments, one logical segment is required per Tier-1 router. |
The Service Engines require data networks to provide access for load-balanced applications. |
None |
AVI-CTLR-024 |
Latency between the Controllers must be <10ms. |
The Controller quorum is latency sensitive.
Note:
The Control plane might go down if latency is high. |
None |
AVI-CTLR-025 |
Latency between the Controllers and the Service Engines should be <75ms. |
Required for correct operation of the Service Engines.
Note:
May lead to issues with heartbeats and data synchronization between the Controller and the Service Engines. |
None |
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-026 |
Use static IPs or DHCP with reservation ensuring a permananet lease for the Controllers. |
The Controller cluster uses management IPs to form and maintain quorum for the control plane.
Note:
The Controller control plane might go down if the management IPs of the Controller change. |
None |
AVI-VI-001 |
Reserve an IP in the management subnet to be used as the cluster IP for the Controller cluster. |
A floating IP that will always be accessible regardless of a specific individual Avi cluster node. |
None |
AVI-NSX-006 |
Configure DHCP on the networks/ logical segments used for data traffic. |
Having DHCP enabled for data networks makes the Service Engine configuration simple.
Note:
Alternatively, operators could use static IPs, but can have to program IP pools for the data networks to be used by the Service Engines and also add a static route for the data network's gateway on the Controller . |
None |
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-026 |
Use static IPs or DHCP with reservation ensuring a permananet lease for the Controllers. |
The Controller cluster uses management IPs to form and maintain quorum for the control plane.
Note:
The Controller control plane might go down if the management IPs of the Controller change. |
None |
AVI-VI-001 |
Reserve an IP in the management subnet to be used as the cluster IP for the Controller cluster. |
A floating IP that will always be accessible regardless of a specific individual Avi cluster node. |
None |
AVI-NSX-006 |
Configure DHCP on the networks/ logical segments used for data traffic. |
Having DHCP enabled for data networks makes the Service Engine configuration simple.
Note:
Alternatively, operators could use static IPs, but can have to program IP pools for the data networks to be used by the Service Engines and also add a static route for the data network's gateway on the Controller . |
None |
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-VI-003 |
Configure time synchronization by using an NTP time for the Controller.
Note:
Recommendation is to use the same source as SDDC Manager, vCenter Server and NSX Manager cluster. |
Prevents from time synchronization issues. Not required to provide connectivity to an external NTP server. |
An operational NTP service must be available in the environment. Ensure that NTP traffic between the Controllers, the Service Engines and the NTP servers is allowed on the required network ports and not firewalled. |
Lifecycle Management for Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-027 |
Use the Controller to performance lifecycle management of the NSX Advanced Load Balancer. |
|
Deployment, patching, updates, and upgrades of NSX Advanced Load Balancer are performed without native SDDC automation. |
AVI-CTLR-028 |
When a VI workload domain is upgraded, upgrade NSX Advanced Load Balancer before upgrading NSX-T Data Center based on the compatibility matrix with vCenter Server and NSX-T Data Center.
Note:
Check the version compatibility matrix in the Advanced Load Balancing for VMware Cloud Foundation validated solution document before upgrading. |
Ensures NSX Advanced Load Balancer cloud integration with NSX-T Data Center and vCenter Server continues to function as expected.
Note:
Upgrading vCenter Server and/ or NSX-T Data Center before NSX Advanced Load Balancer might lead to issues with the NSX-T Cloud Connector integration on the Controller due to version incompatibility. |
None |
AVI-CTLR-029 |
If the Controller is providing services to multiple VI workload domains, choose to upgrade the Controller and only the Service Engine Groups that are associated with the VI workload domain that is being upgraded.
Note:
This is optional. Alternatively, choose to upgrade the entire Controller cluster, which will upgrade the Controllers and all the Service Engines. |
|
None |
Information Security and Access of Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-030 |
Create a strong password for the local admin account on NSX Advanced Load Balancer:
|
This reduces the risk of the account being compromised. This is a requirement to setup user accounts, including the admin account. |
None. |
AVI-CTLR-031 |
Rotate passwords at least every 3 months. |
Ensures security of the user accounts. |
None |
AVI-CTLR-032 |
Limit the use of the local accounts for both interactive or API access and solution integration. |
Local accounts are not specific to user identity and do not offer complete auditing from an endpoint back to the user identity. |
You must define and manage service accounts, security groups, group membership, and security controls in Active Directory. |
AVI-CTLR-033 |
Create user accounts with desired Roles on the Controller to limit the scope and privileges for accounts used for both interactive or API access and solution integrations.
Note:
A custom ‘Role’ might be created if a user account needs to have specific permissions that are not available out of the box on the Controllers. |
The principle of least privilege is a critical aspect of access management and should be part of a comprehensive defense-in-depth security strategy. |
You may need to define and manage custom roles and security controls to limit the scope and privileges used for interactive access or solution integration. |
Monitoring and Alerting of Advanced Load Balancing for VMware Cloud Foundation
Decision ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-035 |
Choose one or more types of notification of choice for monitoring/ alerting. It is recommended to enable alerts on the following pre-defined `System` alerts:
|
Ensure good health through proactive alerting of the NSX Advanced Load Balancer cluster. |
None |
Data Protection of Advanced Load Balancing for VMware Cloud Foundation
Design ID |
Design Decision |
Design Justification |
Design Implication |
|---|---|---|---|
AVI-CTLR-036 |
Create a backup schedule to take periodic backups at least every 24 hours. |
Backed up configuration will aid in rebuilding and recovering the NSX Advanced Load Balancerconfiguration from catastrophic failires. |
Backup server should support SCP as the transport protocol. |
