This section contains the list of new features, issues resolved, key changes, and known issues for 22.1.1 release.

Issues Resolved in 22.1.1 Patch Releases

Issues Resolved in 22.1.1-2p6

Release Date: 03 April 2023

  • AV-171698: In some cases, WAF requests can become slow if client_request_max_body_size in the WAF Profile is set to high values.

  • AV-171581: Upgrade to 20.04 Ubuntu Controllers failed if FQDNs were used instead of Controller IPs due to issues with updating DNS resolvers after upgrade.

Issues Resolved in 22.1.1-2p5

Release Date: 21 March 2023

  • AV-168413: During continuous config operations, the agent memory usage might increase over a period of time.

  • AV-166279: Service Engine failure seen with NTLM requests with unicode characters in the username.

  • AV-165161: Service Engine may fail while processing consecutive RST_STREAM frames from a HTTP/2 server that belong to the same stream.

  • AV-164049: vCenter cloud creation fails to discover vCenter objects, if there are any distributed virtual port group with traffic filtering and marking feature enabled.

  • AV-163620: Memory leak when flag collect_client_fingerprints is enabled in the application profile.

  • AV-161259: SE failure when updating HTTP Policy sets to stop using IP Reputation database and when the SE handles HTTP persistent connections during the update.

  • AV-160898: Under some conditions, in virtual services referring to a WAF policy in which WAF CRS is selected at the SE boot up, some rules in the CRS section of WAF do not run all transformations before evaluating a request causing false negatives in rules 941160, 941170, 941210, 941220, 941310, 941350 and 942190.

  • AV-160593: When Client Insights for a virtual service is set to Active, the virtual service is vulnerable to an HTTP desync attack on the /__avirum__ endpoint.

  • AV-160229: In NSX Advanced Load Balancer version 22.1.1, SE creation might fail in the NSX-T cloud setup if the management network is of a type opaque network.

  • AV-158267: Service Engine failure seen with NTLM requests with unicode characters in the username.

  • AV-157546: Connections may be dropped at the SE when the TCP timestamp option is not present in TCP data.

  • AV-156765: Once Cloud Services get disconnected, it does not get connected without manual intervention.

Issue Resolved in 22.1.1-2p4

  • AV-158634: On upgrading to 22.1.1-2p3, the NSX Advanced Load Balancer UI fails to load.

Issues Resolved in 22.1.1-2p3

Release Date: 13 October 2022

  • AV-155512: Spaces in VIP Address Allocation Network between characters displays the error Request field uuid contains bad character.

  • AV-153369: Create and edit APIs are slow.

Issues Resolved in 22.1.1-2p2

Release Date: 16 September 2022

  • AV-154173: On enabling debugging for a virtual service, and disabling it, there are still debug logs written by the SE.

  • AV-154157: When using exclusions on a WAF policy with case-insensitive, non-regex match on the path field, the performance of WAF goes down drastically, especially if the exclusions are on a group level.

  • AV-153348: In VMware Cloud, unable to uncheck the Use Content Library checkbox even if the content library has not been selected in edit mode.

Issues Resolved in 22.1.1-2p1

Release Date: 01 September 2022

  • AV-152250: When using the Certificate Management Profile to auto-renew certificates, auto-renewal of certificates is triggered multiple times until the certificate is deleted from the Controller.

  • AV-151763: Service Engine failure when a HTTP/2 server sends an RST_STREAM after a HEADERS frame with END_STREAM flag set.

  • AV-151491: Virtual service creation fails when the shared option is selected for datastore scope in the SE group.

  • AV-150990: Unable to edit VRF in NSX-T Cloud in Overlay mode through the UI.

  • AV-150977: Unable to set auth mapping profile in the Basic edition.

  • AV-148246: Parallel execution of the SSL certificate renewal scripts may fail.

Issues Resolved in 22.1.1

  • * AV-127214: SE failure due to incompatibility in hardware versions for LSC deployments on VMware ESXi VMs.

  • AV-132402: Setting non-default argument separator in the WAF Profile takes no effect.

  • AV-137080: BFD echo mode does not work with NSX Advanced Load Balancer.

  • AV-139518: On converting a No Orchestrator cloud to NSX-T cloud, some fields specific to the NSX-T cloud are read only and can not be configured. The option to and from the NSX-T Cloud type is not supported and the option to do so has been removed from Convert Cloud Type.

  • AV-136469: When adding a GSLB pool member for a follower site through the NSX Advanced Load Balancer UI, clicking the Virtual Services drop down list displays an error VirtualService object not found!.

  • AV-140199: For the TLS client, handshake API does not work as expected when connection is terminated after log server restart.

  • AV-141435: Shell login hangs when the number of connections reaching WAIT_TIMEDOUT increase on the Shell server.

  • AV-141493: When the Controller of version 21.1.3 or higher is configured with Cloud Services, rolling back Service Engines to version earlier than 21.1.3 results in failure of the corresponding SEs.

  • AV-142030: Password reset link for admin account fails with the error message {error: “Invalid token”}.

  • AV-142116: When incoming fragmented IPv4 packets (carrying TCP payload) post-reassembly get redirected to SE Linux interface in DPDK mode of operation, exhibit issue with IP checksum.

  • AV-142174: Service Engine can fail if a virtual service is deleted while an ICAP request is being processed.

  • AV-142218: False positives in Bot Management as requests are classified as Bad Bot based on the fact that the source IP is from public cloud providers range.

  • AV-142620: Under VS VIP configuration, under Private IP, when the VIP Address Allocation Network is updated, the NSX Advanced Load Balancer UI was retaining the IP address associated with the network configured earlier.

  • AV-143099: SSL certificate generation using control scripts for flows trying to connect to external SSL certificate authority (for example, LetsEncrypt, Venafi, Sectigo) may fail.

  • AV-143121: With Infoblox IPAM, if an invalid domain is specified in the config, host record creation requests result in a timed-out error from Infoblox leading to the leader node UI and CLI becoming unresponsive.

  • AV-143198: Service Engine may fail if the L7 virtual service listening service is configured with L4 app profile using override_application_profile and is followed by the virtual service’s network profile update.

  • AV-146331: DNS section for virtual service VIPs were not loading for AWS and Azure Cloud types.

  • AV-143699: When using WAF and CRS rules, a CRS rule which is part of a default deactivated CRS group (for example, group CRS_950_Data_Leakages) is executed.

  • AV-143798:

    • Controller cluster goes down because the node appeared to run out of listening sockets

    • Intermittent 401 errors when trying to create or edit configuration via Terraform.

    • Internal goroutine API calls to the Controller display 401 errors

  • AV-143988: POST API call made to Macro API /api/macro containing GSLB objects fails with the error message ” error”: “_perf() got multiple values for keyword argument ‘defer_octavius_request’“.

  • AV-144016: SE might crash when updating a WAF policy that is referenced by a virtual service in fault state, with open connections.

  • AV-144226: In a combination of virtual services with different network profiles, when Ignore Time Wait is enabled in some network profiles and disabled in the others, Ignore Time Wait enabled in a TCP proxy profile is not honored.

  • AV-144235: Packet capture is not working on a virtual service when dedicated dispatcher is enabled on the SE.

  • AV-144262: Creating/ updating IP address groups fails with the error {“error”: “Check checks.IpAddrGroupCheck Panicked!”} when UUID is present in the system configuration (ApiAccess and SshAccess). Upgrade fails in the WaitUntilClusterReadyLocally task due to timeout on waiting for the image_manager queue.

  • AV-144544: When using write-access OpenStack cloud connector in large OpenStack environments, the NSX Advanced Load Balancer API can time out during bulk virtual service VIP operations.

  • AV-144971: Updating large IpAddrGroups can fail with a service timeout.

  • AV-145264: Creating a DNS-type Health monitor without any input in the dns_monitor field (keeping the dns_mmonitor field blank) results in a failure.

  • AV-145662: NSX-T cloud creation is failing if there is no input in the Object Name Prefix, although this field is not mandatory in the UI.

  • AV-145696: When the virtual service VIP is deleted from the Controller, the corresponding AWS Route 53 records are not removed.

  • AV-145754: HTTP requests received with both Content-Length and Transfer-Encoding:Chunked headers, will be generating a significant application log with the message Client sent a request with both chunked Transfer-Encoding and Content-Length header.

  • AV-146000: When sending RST packets, longstanding flows (for more than 30 sec) during upgrade leads to longer timeouts.

  • AV-146188: Deleting an FQDN from virtual service VIP deletes all the FQDNs of a VIP on AWS Route 53.

  • AV-146644: The error NUM_VIRTUALSERVICES: limit value 200, object count 200 is displayed when creating the 200th virtual service in UI of medium and large Controller sizes.

  • AV-146648: se_agent segmentation fault when Controller cluster size changes while a user-agent cache request ( required for bot management) is ongoing from SE to Controller.

  • AV-146774: When the albservicesconfig object is updated through the CLI or the API, there is a subsequent delay in syncing IP reputation and app signature, depending upon the configured time interval for service.

  • AV-147689: IP addresses allocated to a VS VIP are not released when the creation or update of VS VIP fails.

  • AV-148117: In case of an LSC cloud type with se_dp_isolation enabled, when the system is in stress, the show serviceengine cpu command might get stuck occasionally.

What’s New in 22.1.1

Release Date: 15 July 2022

Cloud Connector

  • AWS: UI support for schedule-based scale-out and scale-in of ASG servers. For more information, see Auto Scaling in the VMware NSX Advanced Load Balancer Configuration Guide.

  • NSX-T: Support to create tenant-scoped clouds of type NSX-T. For more information, see Tenant-scoped Clouds in the VMware NSX Advanced Load Balancer Administration Guide.

  • VMware: Enhanced vCenter cloud for better performance and support for Content Library.

    For more information, see Installing NSX Advanced Load Balancer in VMware vSphere Environments in the VMware NSX Advanced Load Balancer Installation Guide.

Core LB Features
GSLB
Networking
Monitoring and Observability

  • Support to exclude or include system events in All Events and Config Audit Trail pages.

  • Support for RTM in prometheus-metrics API calls.

SDK and Integrations

  • Multi-tenancy support for VMware ALB VRO Plugin.

  • Swagger support for Basic, Essentials, and Enterprise licensing tiers.

System
User Interface

  • UI enhancements across configuration objects.

  • The SE UUID column is introduced to the Service Engine page as an optional column.

  • Controller name and site name (if the site name is available), are displayed on the browser tab.

  • Support to search NSX Advanced Load Balancer objects using markers from the NSX Advanced Load Balancer UI.

Web Application Firewall (WAF) and API Security

Key Changes in 22.1.1

  • In version 22.1.1, the string length of the name field for all objects cannot exceed 256 characters. Creation and modification of an object fails if the name exceeds the maximum string length except for the following objects, for which the maximum string length is 280 characters:

    • DNSPolicy

    • HTTPPolicySet

    • NetworkSecurityPolicy

    • VsVip

    • Pool

    • PoolGroup

      If any name exceeds the maximum character threshold on upgrade, the upgrade will fail during the migration step and rollback. See Checklist for Upgrade for more information.

  • Only single X-forward-proto will be sent to the server. If the client request contains an X-forward-proto header, then NSX Advanced Load Balancer rewrites it.

  • Search of usable networks in IPAM is now insensitive to case. Jumbo frame support for NSX Advanced Load Balancer environments revised. If user-defined bot mapping is specified in the bot detection policy, no input is required in the system bot mapping reference. The name of individual BotMappingRule objects in a BotMapping object is mandatory. Hence, you will not be able to create any new objects without a name. ControlScripts that make API calls back to the Controller API using localhost must be updated to use the DOCKER_GATEWAY environment variable instead. It is recommended for a Service Engine to have at least 4 GB of memory when GeoDB is in use.

  • Prior to NSX Advanced Load Balancer version 22.1.1, it was only possible to control the update (PUT) action on any resource field. Starting with NSX Advanced Load Balancer version 22.1.1, if the access is disallowed for any field, creation of objects is not permitted as well.

  • Network objects in NSX Advanced Load Balancer now sync with the name of the associated port group in vCenter. Previously, changing name of the port group and name of the network in NSX Advanced Load Balancer was independent of each other.

  • In case of LSC deployments on VMware ESXi VMs, the hardware compatibility version is 11 or earlier.

Ecosystem Changes

  • vCenter Read Access was deprecated as announced in the 21.1.3 release notes. Ensure that any vCenter cloud in Read-Access mode is converted either to Write-Access, Full-Access or No-Access/No Orchestrator mode before upgrading to 22.1.1.

Known Issue in 22.1.1

  • AV-155317: DPDK driver crash with bond interfaces configured on LSC deployments containing a mix of Mellanox and other network interfaces.

  • AV-182114:

    Symptoms: When the SEs are created with insufficient licenses, the NSX Advanced Load Balancer UI shows that the SE is enabled, when the SE is in the disabled state. On clicking -DISABLE, the SE is stuck and displays the error message, Cannot change state since disable operation is in progress.

    Workaround: From the CLI, manually disable the SE which exhibits this behavior.

Checklist for Upgrade to NSX Advanced Load Balancer Version 22.1.1

Refer to this section before initiating upgrade.

  • Upgrade to NSX Advanced Load Balancer to 22.1.1 is only supported from the following versions:

    • Version 18.2.6 through 18.2.13

    • Version 20.1.1 through 20.1.9

    • Version 21.1.1 through 21.1.4

    • Version 21.1.5 through 22.1.1

      See Upgrade Overview in the VMware NSX Advanced Load BalancerAdministration Guide to know more about flexible upgrades.

  • Starting with NSX Advanced Load Balancer version 22.1.1, the minimum memory recommended for an Essentials Controller is 16 GB. Ensure that the memory of an Essentials Controller is at least 16 GB before upgrade. vCenter Read Access is no longer supported.

  • vCenter Read Access was deprecated as announced in the 21.1.3 release notes. Ensure that any vCenter cloud in Read-Access mode is converted either to Write-Access, Full-Access or No-Access/No Orchestrator mode before upgrading to 22.1.1.

  • The ControlScripts framework has been updated. This requires the ControlScripts to be modified prior to upgrade or on upgrade. For more information, see ControlScripts topic in the VMware NSX Advanced Load Balancer Configuration Guide.

  • As mentioned in the Key Changes, starting with NSX Advanced Load Balancer version 22.1.1, there is an enforcement on the string length in the name field for all objects. Use the script available here to identify all the objects that exceed the name length threshold. Ensure that the object names are modified before upgrading.

For updates on Cloud Services in version 21.1.1, see Cloud Services Release Notes.

Supported Platforms

For more information, see System Requirements: Ecosystem topic in the VMware NSX Advanced Load Balancer Installation Guide.

Product Documentation

For more information, please see the following documents, also available within this Knowledge Base.

Installation Guides

Copyrights and Open Source Package Information

For copyright information and packages used, refer to open_source_licenses.pdf.

Avi Networks software, Copyright © 2015-2022 by Avi Networks, Inc. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under license. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or the GNU Lesser General Public License (LGPL) Version 2.1. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://www.opensource.org/licenses/lgpl-2.1.php

Additional Reading

VMware Ports and Protocols