This section contains the list of new features, issues resolved, key changes, and known issues for 22.1.6 release.

Patch Release Notes for 22.1.6

22.1.6-2P5
Release Date: 03 October 2024

  • AV-212570: Service Engine may fail when async_ssl is enabled.

  • AV-213608: An API call to a virtualservice or httppolicyset that has a GeoDB match rule configured can cause Service Engine failure.

  • AV-214070: Enable/disable operation for multiple virtual services, in some cases, may time out during bulk trigger execution.

  • AV-217781: OEL8.10 (4.18.0-553.8.1.el8_10.x86_64 and 4.18.0-553.16.1.el8_10.x86_64) support for Linux Server Cloud.

22.1.6-2P4
Release Date: 08 August 2024

  • AV-168720: Service Engine may fail when a virtual service is configured with a StringGroup without any string entries, is placed on the SE.

  • AV-190793: In some GSLB site persistence configurations, if a cookie generated by one virtual server is transmitted to another virtual server within the same site or different site, requests may loop back and forth between the sites and then result in a 503 error.

  • AV-208104: In the case of HTTP/2 header processing error, the virtual service reports a protocol error for the subsequent request.

  • AV-209123: Service timeout error on the NSX Advanced Load Balancer UI when creating a Service Engine group.

  • AV-212227: In VMware deployments, traffic disruption may occur if the interface route is down due to a race condition while bringing the SE up.

  • AV-212397: SE memory leak during client certificate authentication, if the certificate subject does not include a common name.

22.1.6-2P3
Release Date: 12 June 2024

  • AV-207823: When preserve client IP is enabled on a UDP-virtual service, if the response from a backend server to the service engine is fragmented, the UDP flow may break.

  • AV-205033: The End-to-End timing from the per-request display in Virtual Service Log UI is missing.

  • AV-204909: Client traffic may experience latency or timeouts for a virtual service when using HSM and HTTP/2 or WAF are configured.

  • AV-198989: When both WAF and Thales HSM are enabled, se_dp processes can fail.

22.1.6-2P2
Release Date: 03 June 2024

  • AV-195157: DNS resolution is affected due to incorrect GSLB status being synced across all the sites.

  • AV-197591: High CPU utilization when Least Load Algorithm is configured in the pool and connection multiplexing is disabled in the Application Profile of the virtual service.

  • AV-204295: Shared memory allocation failures for debug, trace, or event rings in LSC based deployments can cause Service Engine failure.

22.1.6-2P1
Release Date: 20 March 2024

  • AV-199964: Connection set up takes longer (more than 10 milliseconds), on some Service Engines prone to excessive logging.

What's New in 22.1.6

Release Date: 28 February 2024

This is a maintenance release and provides fixes for high-priority issues.

See Checklist for Upgrade to NSX Advanced Load Balancer Version 22.1.6.

Issues Resolved in 22.1.6

  • AV-172878: After updating the NSX Advanced Load Balancer Controller, when the Service Engines are pending update, the Service Engine group's Status is displayed as Unknown in the SEG Update screen.

  • AV-175344: Log Manager's task queue stalls causing unbounded growth on the Controller.

  • AV-175551: Controller service system-portal event file uses older PID format, resulting in multiple events files leading to logging and rsync performance degradation.

  • AV-179858: Unable to modify or save an existing DNS application profile due to a validation error in the Admin Email, entered in the Domain Names/Subdomains screen.

  • AV-181982: Configuration disappeared on GSLB site if glb_local_worker service is not running on the leader.

  • AV-184622: The VirtualService inventory API endpoint excludes blank configuration fields from the response data, instead of including them with an empty string value.

  • AV-185059: CSR certificates managed through the certificate management profile get stuck in a renewal loop, leading to repeated renewal attempts every few seconds and generating corresponding temporary files.

  • AV-185882: Unable to update the secure channel root certificate when the cloud is not set to No Orchestrator or if there are SEs running in the system.

  • AV-186738: Configuring a virtual service with App Cookie Persistence, Detect NTLM App enabled, and Connection Multiplexing disabled, leads to Service Engine failure after an HTTP request has been sent and the connection times out due to the Keep-Alive timer.

  • AV-188816: Over time, certain GSLB processes experience memory consumption issues, leading to excessive memory usage across nodes.

  • AV-188904: A Trailing RST on a closed L7 SSL virtual service connection may result in SE failure.

  • AV-189818: Unable to edit or update the checkpoint object after setting a checkpoint as active in adaptive replication mode. The replication stalls with the following error, Sync Stalled, reason: replicating federationcheckpoint:<checkpoint_name>

  • AV-190126: Using Broadcom NIC as management with Mellanox NIC for datapath causes issues in bringing up the NIC.

  • AV-190461: Frequent updates to StringGroups attached to a DataScript, that also makes repeated calls to avi.stringgroup functions may result in failures in string group lookups.

  • AV-190485: vCenter cloud keeps generating DELETE_SE_FAIL, CC_SE_DELETION_FAILURE events every 5 minutes for some SEs.

  • AV-190475: se_dp failure occurs due to memory corruption in rare cases within the GRO layer.

  • AV-190615: Deploying a Controller node with ovf property for IPv6 address, avi.mgmt-ip-v6.CONTROLLER set as null instead of leaving it as blank, leads to erroneous IP configuration.

  • AV-190853: Performance issues when handling large requests in WAF with a large Positive Security Model.

  • AV-191920: Upon upgrading the Controller to version 22.1.4 or higher, enabling both RHI and Scale-out ECMP triggers an error state for the virtual service, displaying the message: We have encountered a problem during your request: Scale out ECMP and RHI cannot be enabled at the same time. To fix the misconfiguration, see BGP Support for Scaling Virtual Services.

  • AV-191360: After successfully adding a WAF exception through VS logs, the subsequent exception additions fail.

  • AV-191387: When an incoming request contains an Avi-generated cookie for HTTP Cookie Persistence, that cookie is forwarded to the backend server with the request. However, in some cases, servers may expire the Avi-specific cookie during certain transactions. As a result, clients fail to present the cookie in subsequent transactions, leading to persistence failure on NSX Advanced Load Balancer. After consuming the Avi-generated HTTP cookie and persisting to the selected server, Avi will remove that cookie from the request sent to the server.

  • AV-191149: Objsync may cause memory build-up and might lead to OOM eventually on the Service Engine due to objsync peer connection failures due to port either 9001 or 4001 not being open in DFW in NSX or no management plane connectivity between SEs in various enviornments.

  • AV-191509: A large number of event files generated on Controller clusters resulting in high CPU utilization on the Controller.

  • AV-191545: The source port range of the BFD control packets does not adhere to the RFC5881.

  • AV-191551: SE fails to connect to the Controller when deployed using a VM template created by the user rather than deploying using the SE OVA.

  • AV-191615: When a WebSocket is utilized with front-end using HTTP/2 and backend using HTTP/1, then NSX Advanced Load Balancer does not terminate the v1 WebSocket on the backend if the "Upgrade" header sent by the server is not "websocket" (all in lowercase), the upgrade header's value being case-sensitive.

  • AV-191642: A PKI profile with a large CRL (greater than 4 MB) fails in replication across federation because of gRPC message size limitation.

  • AV-191670: In VMware NSX environments, in some scenarios when VIPs are created and added, NSX Advanced Load Balancer retains stale routes causing VIPs to go down.

  • AV-191808: The write access restriction to the Controller file system from the ControlScript is compromised as mounting a device (dev) path was allowed.

  • AV-191913: Using a GeoDB object configured with the option Is Federated through the UI causes NSX Advanced Load Balancer to fail. This option has been deactivated now.

  • AV-192083: Failure in Objsync connection over management interfaces between SEs might lead to memory exhaustion.

  • AV-192417: If the GSLB leader changes due to network partition and the old leader disables the new leader, then both the leaders wipe out each other's configurations.

  • AV-192508: Changing a specific pool server from an initially configured IPv6 address to an IPv4 address, is unrestricted, creating a mismatch where the server is designated as IPv6 but configured with an IPv4 address, ultimately leading to a Service Engine failure.

  • AV-192901: Updating passwords in vCenter can transition the Avi vCenter cloud to failed state.

  • AV-192951: Unable to use Infoblox DNS and Infoblox IPAM profiles when they are handled by different Infoblox instances.

  • AV-192601: SE failure can occur if the memory allocation fails when True Client IP is used.

  • AV-193075: Requests with X-Accel-Redirect on the response may fail.

  • AV-193221: Outbound NAT does not preserve UDP flows in Active/Standby HA mode.

  • AV-193663: Metrics Manager's database connections with Postgres are unclosed, causing a connection leak.

  • AV-193665: When configuring an Analytics Profile through the CLI with a format_config object and subsequently accessing the Analytics Profile page in the UI, an exception occurs because the format_config field is not supported though the UI.

  • AV-194178: The Horizon UAG system's default DataScript is decoding the CRLF in the client request making it vulnerable to injection attack.

  • AV-194313: On the NSX-T cloud there may be a spam of CC_IP_ATTACH events if the NSX Manager has more than 1000 routes on any T1.

  • AV-195217: In LSC hosts, when configuring Mellanox devices in combination with Broadcom components, the ring size computation logic can cause initialization errors and stall the SE during connection to the Controller.

  • AV-195418: ControlScript execution fails due to incorrect IP address value in the DOCKER_GATEWAY environment variable, when the Controller IP is in the range 172.16.0.0/16 to 172.31.0.0/16.

  • AV-195595: External log streaming to servers or load balancers which erroneously respond to simplex log stream causing Service Engine memory growth, eventually leading to SE failure.

  • AV-195716: Although licenses are available in Cloud Services, changing the Bandwidth Type of SE Group in the Cloud Services tier failed.

  • AV-196642: The Service Engine may fail when the virtual service is updated during a TLS handshake.

  • AV-196619:AttributeError: 'Response' object has no attribute 'uuid when attempting POST/PUT operations on the gslbservice object through the macro API.

  • AV-196914: VsVIP objects having the same IP address may cause SE failure.AV-197046: IPAM allocation for A records with multiple subnets will fail when the first subnet is exhausted.

  • AV-197319: If WAF learning is enabled and in addition for the same WAF policy a second WAF Positive Security Model group is created which is matching on the PATH_INFO variable. This can cause SE failure.AV-197350: Log streaming fails owing to the streaming endpoint restarts or receipt of any unexpected responses.

  • AV-197737: Go SDK does not allow values within the range of uint32, causing it to fail while unmarshalling JSON data containing uint32 values in the request/response.

  • AV-198105:Failed renewal for expired certificates result in the accumulation of numerous temporary files in the /tmp directory over time, leading to a significant increase in inode usage and storage consumption in the Controller.

  • AV-198269: Positive Security Model (PSM) programming failure in WAF applications with learning enabled, when unique parameters exceed the configured maximum values.

  • AV-199434: If the Controller is connected to the proxy server and the proxy server goes down during an active connection to Cloud Services, the Controller displays the error GET <https://10.49.50.118/api/albservices/status> 500 (Internal Server Error).

  • AV-198574: NSX Cloud: VS fails to preserve an IPv6 client IP.

Security Advisories

  • This release resolves CVE-2024-22264. For more information on this vulnerability including impacted product suites and release lines, see VMSA-2024-0009. This release train is not affected by CVE-2024-22266.

  • NSX Advanced Load Balancer is not affected by CVE-2021-44832.

Key Changes in 22.1.6

  • NSX Advanced Load Balancer Basic Edition: End of Availability and End of General Support:

    • VMware announced the End of Availability of NSX Advanced Load Balancer Basic Edition for new deployments and End of General Support (EoGS) for existing deployments.

    • For new deployments, the End of Availability for NSX Advanced Load Balancer Basic Edition was 30th January 2024.

    • For the existing NSX ALB Basic edition deployments, VMware will provide ongoing support for active NSX Advanced Load Balancer releases up to version 30.1.x. The End of General Support (EoGS) for NSX Advanced Load Balancer Basic Edition is scheduled for 30th June 2025.

    • To seamlessly continue enjoying advanced functionalities and comprehensive support, it is recommended transitioning from NSX Advanced Load Balancer Basic Edition to VMware Avi Load Balancer Enterprise Edition, or VMware Avi Load Balancer Enterprise with Cloud Services Edition.

    • For any additional questions or clarification, reach out to your dedicated sales team. See the KB article for more information.

  • Starting with NSX Advanced Load Balancer version 22.1.6, cookies generated by NSX Advanced Load Balancer for HTTP Cookie Persistence are no longer sent in the server-side requests. These cookies are maintained by the load balancer and are only meant to be used on the client-side.

  • WAF: Automatic application rules updates will be discontinued in July 2024. Further communication and guidance will be provided in the upcoming releases.

Checklist for Upgrade to NSX Advanced Load Balancer Version 22.1.6

Refer to this section before initiating upgrade.

  • Upgrade to NSX Advanced Load Balancer 22.1.6 is only supported from the following versions:

    • Version 20.1.1 through 20.1.9

    • Version 21.1.1 through 21.1.6

    • Version 22.1.1 through 22.1.5

    • Starting with NSX Advanced Load Balancer version 22.1.3, the minimum memory requirement for Service Engines is increased to 2GB. Before upgrading to any version in the 22.1.x release, ensure the Service Engines are configured to a capacity greater than 2 GB. The current considerations for memory sizing as listed under Sizing Service Engines in the VMware NSX Advanced Load BalancerConfiguration Guide continue to apply.

      For more information on flexible upgrades, see Upgrade Overview in the VMware NSX Advanced Load BalancerAdministration Guide.

  • Ensure the options Scale out ECMP and RHI are not enabled together for any virtual service. On upgrading the Controller to version 22.1.4 or higher, this triggers an error state for the virtual service, displaying the message: We have encountered a problem during your request: Scale out ECMP and RHI cannot be enabled at the same time. o fix the misconfiguration, see BGP Support for Scaling Virtual Services

  • Before upgrading to version 22.1.2 and higher, export the Avi metrics database. In case of rolling back from NSX Advanced Load Balancer 22.1.2 to an earlier version, import the metrics database to prevent loss of metrics data.

    For more information, see FAQs on Controller Cluster in the VMware NSX Advanced Load BalancerAdministration Guide.

  • Starting with NSX Advanced Load Balancer version 22.1.3, the minimum memory recommended for an Essentials Controller is 24G. Ensure that the memory of an Essentials Controller is at least 24G before upgrade.

  • The ControlScripts framework has been updated. This requires the ControlScripts to be modified prior to upgrade or on upgrade.

    For more information, see the Scripts topic in the VMware NSX Advanced Load BalancerConfiguration Guide.

  • As mentioned in the Key Changes, starting with NSX Advanced Load Balancer version 22.1.1, there is an enforcement on the string length in the name field for all objects. Use the script available here to identify all the objects that exceed the name length threshold. Ensure that the object names are modified before upgrading.

  • Disable Large Receive Offload (LRO) before upgrading to NSX Advanced Load Balancer version 22.1.3 or later to prevent packet loss in Preserve-Client IP environments.