Design Decisions of Advanced Load Balancing for VMware Cloud Foundation

This section explains the design decisions of Advanced Load Balancing for VMware Cloud Foundation.

Deployment Model for the Advanced Load Balancing for VMware Cloud Foundation

Table 1. Design Decisions for Deploying the Controller for the VMware NSX Advanced Load Balancer

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-001	Initial setup should be done only on one NSX Advanced Load Balancer Controller VM out of the three deployed to create an NSX Advanced Load Balancer Controller cluster.	NSX Advanced Load Balancer Controller cluster is created from an initialized NSX Advanced Load Balancer Controller which becomes the cluster leader. Follower NSX Advanced Load Balancer Controller nodes need to be uninitialized to join the cluster.	NSX Advanced Load Balancer Controller cluster creation will fail if more than one NSX Advanced Load Balancer Controller is initialized.
AVI-CTLR-002	Apply vSphere DRS anti-affinity rules for the NSX Advanced Load Balancer Controller cluster nodes. Note: For a default management vSphere cluster that consists of four ESXi hosts, you can put in maintenance mode only a single ESXi host at a time.	Ensure that NSX Advanced Load Balancer Controller VMs are distributed across ESXi hosts .	You must perform additional configuration to set up an anti-affinity rule.
AVI-CTLR-003	Protect NSX Advanced Load Balancer Controller cluster nodes using vSphere High Availability.	Supports the availability objectives for the NSX Advanced Load Balancer Controller cluster without requiring manual intervention during an ESXi host failure event.	None

Table 2. Design Decisions for deploying Service Engines for the VMware NSX Advanced Load Balancer

Decision ID	Decision Design	Design Justification	Design Implication
AVI-CTLR-004	Create an NSX-T Cloud Connector on NSX Advanced Load Balancer Controller for each NSX transport zone requiring load balancing.	A NSX-T Cloud Connector configured on the NSX Advanced Load Balancer Controller will provide load balancing for workloads belonging to a Transport Zone on NSX-T. Note: 1. A NSX Transport Zone can be unique to a vCenter cluster, a VI Workload Domain or can be shared across VI workload domains. 2. Multiple NSX-T Cloud connectors can be configured on the NSX Advanced Load Balancer Controller if load balancing is required across multiple Transport Zones configured on NSX-T.	None

Table 3. Design Decisions for deploying Service Engines on a Dedicated Edge VI Workload Domain for NSX Advanced Load Balancer Platform

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-005	Choose to deploy the Service Engines on a dedicated edge VI workload domain.	Allows for centralized placement of the Service Engines.	Capacity growth might be a challenge. Might not work in all cases due to scale restrictions of the edge VI workload domain.
AVI-CTLR-006	Create separate Service Engine Groups to host Virtual Services from different VI workload domains.	Allows for application isolation.	Might require additional Service Engine resources.

Table 4. Design Decisions on Deployment of NSX Advanced Load Balancer Controllers in Multiple Availability Zones

Decision ID	Design Decision	Design Justification	Design Implication
AVI-VI-VC-001	When using two availability zones, add the NSX Advanced Load Balancer Controller cluster nodes to the first availability zone VM group.	Ensures that, by default, the NSX Advanced Load Balancer Controller cluster nodes are powered on in the primary availability zone hosts group.	After the implementation of the second availability zone for the management domain, you must update the VM group for the primary availability zone virtual machines to include the NSX Advanced Load Balancer Controller cluster nodes.

Table 5. Design Decisions for placing applications on NSX Advanced Load Balancer Service Engines in a Multi Availability Zone environment

Decision ID	Design Decision	Design Justification	Design Implication
AVI-VI-VC-002	Create a VM groups for the NSX Advanced Load Balancer SE VMs.	Ensures that the NSX Advanced Load Balancer SE VMs can be managed as a group and added to VM/ Host rules.	User must add each NSX Advanced Load Balancer SE VM to the primary availability zone.
AVI-VI-VC-003	Create a should-run VM-Host affinity rule to run all NSX Advanced Load Balancer SEs on the group of hosts in the first availability zone.	Ensures that all NSX Advanced Load Balancer SE VMs are in the first availability zone.	During normal operation, there would not be any NSX Advanced Load Balancer SEs running in the second availability zone. Therefore all apps would be active in the first availability zone.

Integration of the Advanced Load Balancing for VMware Cloud Foundation

Table 6. Design Decisions for creating an NSX-T Cloud on the Controller for the VMware Cloud Foundation

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-007	Create one NSX-T Cloud connector object on the Controller per transport zone configured on the NSX manager cluster that requires Load Balancing services. Note: Transport zone could be dedicated to a VI workload domain or shared across VI workload domains.	Provides automated deployment of load-balanced applications through NSX-T Cloud integration. Allows for maximum flexibility, control, and isolation in terms of application deployment.	None
AVI-CTLR-008	Provide either a overlay-backed NSX segment connected to a Tier-1 logical router or a VLAN-backed NSX segment for the Service Engine management for the NSX-T Cloud of overlay type.	This network is used for the Controller to the Service Engine connectivity.	None
AVI-CTLR-009	Provide one or more NSX managed VLAN segments as data networks for the NSX-T Cloud connector of VLAN type. Note: A single NSX-T Cloud connector of VLAN type can contain multiple data networks. Each data network should belong to a unique NSX managed VLAN segment.	The Service Engines are placed on NSX managed VLAN segments.	None
AVI-CTLR-010	Provide a Tier-1 router and a connected overlay-backed NSX segment as data network for the NSX-T Cloud of overlay type. Note: A single NSX-T Cloud connector of overlay type can contain multiple data networks. Each data network must belong to a unique Tier-1 router.	The Service Engines are placed on Overlay Segments created on these Tier-1 logical router(s).	None
AVI-CTLR-011	Provide an object name prefix when creating the NSX-T Cloud Connector on the NSX Advanced Load Balancer Controller.	Used for uniquely identifying NSX-T Cloud Connector created resources on NSX Manager cluster and vCenter Server.	None

Isolation Model for Load-Balanced Applications in Advanced Load Balancing for VMware Cloud Foundation

Table 7. Design Decisions for creating a Tenants for isolation on the VMware NSX Advanced Load Balancer for the VMware Cloud Foundation

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-012	Create tenants to provide desired level of isolation for the VMware Cloud Foundation. Note: NSX Advanced Load Balancer - Basic Edition does not provide tenant isolation.	Provides required level of configuration and data plane isolation for workloads.	Additional Service Engine resources might be required.

Table 8. Design Decisions for Service Engine Group Design for VMware NSX Advanced Load Balancer for the VMware Cloud Foundation

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-013	Create multiple Service Engine Groups as desired to isolate applications. Note: Some of the criteria for grouping applications in different Service Engine Group(s) could be based on: Multiple line of business Prod v/s non-Prod Different scale and performance requirements	Allows efficient isolation of applications and allows for better capacity planning. Allows flexibility of life-cycle-management.	None
AVI-CTLR-014	Create separate set of Service Engine Groups for each VI workload domain and scope the Service Engine Group to the VI workload domain vCenter server. Note: Applicable where a single Controller cluster serving multiple VI workload domains. If applications need to be shared across VI workload domains, then the Service Engine Group could be scoped to multiple vCenter Servers.	Allows isolation of the Service Engines across VI workload domains. Enables per VI workload domain life-cycle-management.	None
AVI-CTLR-015	Configure Service Engine Group for Active/ Active HA mode. Note: Legacy Active/ Standby HA mode might be required for certain applications.	Provides optimum resiliency, performance, and utilization.	Certain applications might not work in Active/ Active mode. For instance, applications that require preserving client IP. In such cases, use the Legacy Active/ Standby HA mode.
AVI-CTLR-016	Enable 'Dedicated dispatcher CPU' on Service Engine Groups that contain the Service Engine VMs of 4 or more vCPUs. Note: This setting should be enabled on SE Groups that are servicing applications that have high network requirement.	This will enable a dedicated core for packet processing enabling high packet pipeline on the Service Engine VMs. Note: By default, the packet processing core also processes load balancing flows.	None .
AVI-CTLR-017	Set 'Placement across the Service Engines' setting to 'distributed'.	This allows for maximum fault tolerance and even utilization of capacity.	Might require more Service Engine VMs as compared to 'compact' placement mode.
AVI-CTLR-018	Enable CPU and Memory reservation on the Service Engine Group. .	The Service Engines are a critical infrastructure component providing load-balancing services to mission critical applications.	None
AVI-CTLR-019	Configure a consistent Service Engine Name Prefix that indicates the Service Engine VM for instance, 'avise-xxxx'. Note: Where ‘xxxx’ could be used as an arbitrary identifier.	This allows efficient grouping and filtering.	None
AVI-CTLR-020	Choose the Service Engine Group mode as Legacy HA Active/ Standby if the Controller is set to use basic edition. .	NSX Advanced Load Balancer Controller in Basic Edition only supports Legacy HA Active/ Standby mode.	Applications will not be deployed in an Active/ Active fashion, thereby losing out on elastic capacity management. NSX Advanced Load Balancer Enterprise Edition will allow Active/ Active as well as Legacy Active/ Standby deployments.

Physical Design of the Advanced Load Balancing for VMware Cloud Foundation

Table 9. Design Decisions for Physical Design of ESXi Hosts to support the VMware NSX Advanced Load Balancer

Decision ID	Design Description	Design Justification	Design Implication
AVI-PHY-001	Provide high performance disks (SSD/ Flash) to hosts that run the Controller VMs.	The Controllers need high performance disks to process the analytics pipeline.	None
AVI-PHY-002	Enable AES-NI instructions setting in the BIOS for ESXi hosts.	AES-NI instruction set provides efficiency in SSL performance.	Most modern machines have AES-NI enabled by default, if not enabled by default, you need to reboot ESXi hosts to enable this setting.
AVI-PHY-003	Disable C-State and P-State settings in BiOS on the ESXi hosts. Note: This is an optional design decision.	Provides maximum performance.	This might require a reboot and reconfigure of the BIOS causing an outage for each ESXi host.

vCenter Design of the Advanced Load Balancing for VMware Cloud Foundation

Table 10. Design Decisions for the Virtual Infrastructure to support the VMware NSX Advanced Load Balancer

Decision ID	Design Description	Design Justification	Design Implication
AVI-VI-VC-004	Create anti-affinity 'VM/ Host' rule that prevents collocation of the Controller VMs.	vSphere will take care of placing the Controller VMs in a way that always ensures maximum HA for the Controller cluster.	None
AVI-VI-VC-005	Create a virtual machine group for the Controller VMs.	Ensures that the Controller VMs can be managed as a group.	You must add virtual machines to the allocated groups manually.
AVI-VI-VC-006	In vSphere HA, for each Controller and Service Engine VMs, set the restart priority policy to high and host isolation response to disabled.	This ensures fast recovery for the NSX Advanced Load Balancer.	None
AVI-VI-VC-007	Create one Content Library on the management domain to store Controller OVA.	Deploying OVA from the Content Library will be operationally easy to do.	Might not be necessary if deploying Controller VMs using automation tools such as vRO, Ansible, etc.
AVI-VI-VC-008	Create one Content Library on each of the VI workload domain to store Service Engine OVA.	The Controller's NSX-T Cloud Connector requires a Content Library configured to create the Service Engines.	None

VCenter Server Design of the Advanced Load Balancing for VMware Cloud Foundation

NSX-T Data Center Design of the Advanced Load Balancing for VMware Cloud Foundation

Table 11. Design Decisions for NSX-T Data Center Access Control for NSX Advanced Load Balancer Controller

Decision ID	Design Decision	Design Justification	Design Implication
AVI-NSX-001	Create or use an NSX-T Manager cluster User/ Role with password with the described privileges. Note: It is recommended not to use the local ‘admin’ user of NSX-T Data Center.	Required for the Controller to perform lifecycle management of the Service Engines. Note: Update the NSX-T User Credential on the Controller when password for this user account is rotated.	None

Table 12. Design Decisions for the NSX-T Data Center Distributed Firewall Rules

Decision ID	Design Decision	Design Justification	Design Implication
AVI-NSX-002	Create necessary NSX DFW and/ or Gateway Firewall rules for the NSX Advanced Load Balancer control plane as described to ensure connectivity from: Admin to the Controllers The Controllers to the Controllers The Controllers to Service Engines	These firewall rules are needed to allow required communication for the NSX Advanced Load Balancer control plane. Note: If DFW is enabled and these rules are not configured, this might result in NSX Advanced Load Balancer control plane not functioning as expected.	None
AVI-NSX-003	Create necessary NSX DFW and/ or Gateway Firewall rules for the configured load-balanced applications as described to ensure connectivity from: Client to VIPs Service Enginess to Backend Pool Servers	These firewall rules are needed to allow required communication for the configured load-balanced applications. Note: If DFW is enabled and these rules are not configured, this might result in the configured load-balanced applications not functioning as expected.	None

Licensing VMware Advanced Load Balancing for VMware Cloud Foundation

Table 13. Design Decisions for Licensing VMware NSX Advanced Load Balancer

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-021	Choose the VMware NSX Advanced Load Balancer Enterprise with Cloud Services licensing tier. Note: 1. New VMware NSX Advanced Load Balancer deployments running v21.1.3 or later will be setup by default in Enterprise with Cloud Services licensing tier. 2. If running v21.1.2 or earlier, choose the VMware NSX Advanced Load Balancer Enterprise licensing tier.	Provides full-featured access to the NSX Advanced Load Balancer platform. Note: If running v21.1.3 or later, alternative is to either use: i) Enterprise edition licensing tier. This provides a full-featured enterprise feature set but does not give access to Cloud Services and advanced App Security features. ii) Basic edition licensing tier. This provides equivalent functionality of NSX-T Data Center native Load Balancer. If running v21.1.2 or earlier, alternative is to use the Basic edition licensing tier. This provides equivalent functionality of NSX-T Data Center native Load Balancer.	None

How to size Advanced Load Balancing for VMware Cloud Foundation

Table 14. Design Decisions for sizing the Controllers for the NSX Advanced Load Balancer

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-022	Deploy one Controller cluster for each NSX Manager cluster for configuring and managing load balancing services.	Required to form a highly available Controller cluster.	None
AVI-CTLR-023	Deploy each node in the Controller cluster with a minimum of 8 vCPUs, 32 GB memory and 216 GB of disk space.	Support up to 200 virtual services. Support up to 100 NSX Advanced Load Balancer Service Engines. Can scale-up with expansion of the Controller sizes anytime. Note: Under sizing, the Controllers can lead to unstable control plane functionality.	None

Network Design for Advanced Load Balancing for VMware Cloud Foundation

Table 15. Design Decisions for the Networking Design for VMware NSX Advanced Load Balancer

Decision ID	Design Decision	Design Justification	Design Implication
AVI-VI-VC-010	Deploy the Controller cluster nodes on the VMware Cloud Foundation management network.	Allows for ease of management for the Controllers. Allows for configuring a floating cluster VIP; a single IP address that will be assigned to the cluster leader. Administrative tasks, connectivity to the Service Engines and connectivity to network services will all use this network.	None
AVI-NSX-004	Configure a management network to deploy the Service Engines. Management network needs to be NSX segment and could be either of: VLAN-backed NSX segment Overlay-backed NSX segment connected to a Tier-1 router Note: This network should have connectivity to the IP addresses of each of the Controllers.	This is required to configure the Controller NSX-T Cloud Connector.	None
AVI-NSX-005	Configure one or more data network(s) for the Service Engines to service load-balanced applications. Data networks need to be NSX-T managed and could be either of: VLAN-backed NSX segment, or, Overlay-backed NSX segment connected to a Tier-1 router Note: For overlay-backed NSX segments, one logical segment is required per Tier-1 router.	The Service Engines require data networks to provide access for load-balanced applications.	None
AVI-CTLR-024	Latency between the Controllers must be <10ms.	The Controller quorum is latency sensitive. Note: The Control plane might go down if latency is high.	None
AVI-CTLR-025	Latency between the Controllers and the Service Engines should be <75ms.	Required for correct operation of the Service Engines. Note: May lead to issues with heartbeats and data synchronization between the Controller and the Service Engines.	None

Table 16. Design Decisions for the IP Addressing Scheme for VMware NSX Advanced Load Balancer

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-026	Use static IPs or DHCP with reservation ensuring a permananet lease for the Controllers.	The Controller cluster uses management IPs to form and maintain quorum for the control plane. Note: The Controller control plane might go down if the management IPs of the Controller change.	None
AVI-VI-001	Reserve an IP in the management subnet to be used as the cluster IP for the Controller cluster.	A floating IP that will always be accessible regardless of a specific individual Avi cluster node.	None
AVI-NSX-006	Configure DHCP on the networks/ logical segments used for data traffic.	Having DHCP enabled for data networks makes the Service Engine configuration simple. Note: Alternatively, operators could use static IPs, but can have to program IP pools for the data networks to be used by the Service Engines and also add a static route for the data network's gateway on the Controller .	None

Table 17. Design Decisions for the IP Addressing Scheme for VMware NSX Advanced Load Balancer

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-026	Use static IPs or DHCP with reservation ensuring a permananet lease for the Controllers.	The Controller cluster uses management IPs to form and maintain quorum for the control plane. Note: The Controller control plane might go down if the management IPs of the Controller change.	None
AVI-VI-001	Reserve an IP in the management subnet to be used as the cluster IP for the Controller cluster.	A floating IP that will always be accessible regardless of a specific individual Avi cluster node.	None
AVI-NSX-006	Configure DHCP on the networks/ logical segments used for data traffic.	Having DHCP enabled for data networks makes the Service Engine configuration simple. Note: Alternatively, operators could use static IPs, but can have to program IP pools for the data networks to be used by the Service Engines and also add a static route for the data network's gateway on the Controller .	None

Table 18. Design Decisions for the Time Synchronization for VMware NSX Advanced Load Balancer

Decision ID	Design Decision	Design Justification	Design Implication
AVI-VI-003	Configure time synchronization by using an NTP time for the Controller. Note: Recommendation is to use the same source as SDDC Manager, vCenter Server and NSX Manager cluster.	Prevents from time synchronization issues. Not required to provide connectivity to an external NTP server.	An operational NTP service must be available in the environment. Ensure that NTP traffic between the Controllers, the Service Engines and the NTP servers is allowed on the required network ports and not firewalled.

Lifecycle Management for Advanced Load Balancing for VMware Cloud Foundation

Table 19. Design Decisions for Lifecycle Management of the VMware NSX Advanced Load Balancer

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-027	Use the Controller to performance lifecycle management of the NSX Advanced Load Balancer.	Lifecycle of NSX Advanced Load Balancer is not managed by SDDC Manager. The Controller manages lifecycle for all NSX Advanced Load Balancer components including the Controllers and all the associated Service Engines.	Deployment, patching, updates, and upgrades of NSX Advanced Load Balancer are performed without native SDDC automation.
AVI-CTLR-028	When a VI workload domain is upgraded, upgrade NSX Advanced Load Balancer before upgrading NSX-T Data Center based on the compatibility matrix with vCenter Server and NSX-T Data Center. Note: Check the version compatibility matrix in the Advanced Load Balancing for VMware Cloud Foundation validated solution document before upgrading.	Ensures NSX Advanced Load Balancer cloud integration with NSX-T Data Center and vCenter Server continues to function as expected. Note: Upgrading vCenter Server and/ or NSX-T Data Center before NSX Advanced Load Balancer might lead to issues with the NSX-T Cloud Connector integration on the Controller due to version incompatibility.	None
AVI-CTLR-029	If the Controller is providing services to multiple VI workload domains, choose to upgrade the Controller and only the Service Engine Groups that are associated with the VI workload domain that is being upgraded. Note: This is optional. Alternatively, choose to upgrade the entire Controller cluster, which will upgrade the Controllers and all the Service Engines.	Allows isolated upgrade for the VI workload domain Upgrade only the Service Engines that reside on the VI workload domain that is being upgraded VI workload domain that is currently not being upgraded, but shares the same Controller is left untouched.	None

Information Security and Access of Advanced Load Balancing for VMware Cloud Foundation

Table 20. Design Decisions for Information Security and Access of the VMware NSX Advanced Load Balancer

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-030	Create a strong password for the local admin account on NSX Advanced Load Balancer: Minimum 8 char long Contains at least one char in each of 3/4 of the following categories: Uppercase letters Lowercase letters Digits Special characters	This reduces the risk of the account being compromised. This is a requirement to setup user accounts, including the admin account.	None.
AVI-CTLR-031	Rotate passwords at least every 3 months.	Ensures security of the user accounts.	None
AVI-CTLR-032	Limit the use of the local accounts for both interactive or API access and solution integration.	Local accounts are not specific to user identity and do not offer complete auditing from an endpoint back to the user identity.	You must define and manage service accounts, security groups, group membership, and security controls in Active Directory.
AVI-CTLR-033	Create user accounts with desired Roles on the Controller to limit the scope and privileges for accounts used for both interactive or API access and solution integrations. Note: A custom ‘Role’ might be created if a user account needs to have specific permissions that are not available out of the box on the Controllers.	The principle of least privilege is a critical aspect of access management and should be part of a comprehensive defense-in-depth security strategy.	You may need to define and manage custom roles and security controls to limit the scope and privileges used for interactive access or solution integration.

Monitoring and Alerting of Advanced Load Balancing for VMware Cloud Foundation

Table 21. Design Decisions for Monitoring and Alerting for Advanced Load Balancing for VMware Cloud Foundation

Decision ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-035	Choose one or more types of notification of choice for monitoring/ alerting. It is recommended to enable alerts on the following pre-defined `System` alerts: System-VS-Alert System-SSL-Alert System-SE-Alert System-Controller-Alert System-CC-Alert	Ensure good health through proactive alerting of the NSX Advanced Load Balancer cluster.	None

Data Protection of Advanced Load Balancing for VMware Cloud Foundation

Table 22. Design Decisions for Data Protection of Advanced Load Balancing for VMware Cloud Foundation

Design ID	Design Decision	Design Justification	Design Implication
AVI-CTLR-036	Create a backup schedule to take periodic backups at least every 24 hours.	Backed up configuration will aid in rebuilding and recovering the NSX Advanced Load Balancerconfiguration from catastrophic failires.	Backup server should support SCP as the transport protocol.