VMware SASE 5.2.2 | 20 June 2024

  • VMware SASE™ Orchestrator Version R5220-20231214-GA

  • VMware SD-WAN™ Gateway Version R5221-20240206-GA

  • VMware SD-WAN™ Edge Version R5223-20240410-GA

Check for additions and updates to these release notes.

What Is in The Release Notes

The release notes cover the following topics:

Note on 5.2.1 Version

Release 5.2.1 was created as the manufacturing image shipped with the SD-WAN Edge model 710-W, which becomes available for sale on February 1, 2024. A user would observe this Edge software version if they factory reset this model and in no other context.

As a result, SASE and SD-WAN Release 5.2.2 is the first Maintenance Release for 5.2.0.

This release is recommended for all customers who require the features and functionality first made available in Release 5.2.0, as well as those customers impacted by the issues listed below which have been resolved since Release 5.2.0.

Important:

Release 5.2.2 contains all Edge, Gateway, and Orchestrator fixes that are listed in the 5.2.0 Release Notes, including all UI Builds.

Compatibility

Release 5.2.2 Orchestrators, Gateways, and Hub Edges support all previous VMware SD-WAN Edge versions greater than or equal to Release 4.2.0.

The following SD-WAN interoperability combinations were explicitly tested:

Orchestrator

Gateway

Edge

Hub

Branch/Spoke

5.2.2

4.5.2

4.2.2

4.2.2

5.2.2

5.2.2

4.2.2

4.2.2

5.2.2

5.2.2

5.2.2

4.2.2

5.2.2

5.2.0

4.2.2

5.2.2

5.2.2

4.5.2

4.2.2

5.2.2

5.2.2

4.3.2

4.3.2

4.3.2

5.2.2

5.2.2

4.3.2

4.3.2

5.2.2

5.2.2

5.2.2

4.3.2

5.2.2

5.2.2

4.3.2

5.2.2

5.2.2

4.3.2

4.3.2

5.2.2

5.2.2

4.5.2

4.5.2

4.5.2

5.2.2

5.2.2

4.5.2

4.5.2

5.2.2

5.2.2

5.2.2

4.5.2

5.2.2

5.2.2

4.5.2

5.2.2

5.2.2

4.5.2

4.3.2

5.2.2

5.2.2

5.0.1

5.0.1

5.0.1

5.2.2

5.2.2

5.0.1

5.0.1

5.2.2

5.2.2

5.2.2

5.0.1

5.2.2

5.2.2

5.0.1

5.2.2

5.2.2

5.0.1

5.2.2

5.0.1

5.2.2

5.1.0

5.1.0

5.1.0

5.2.2

5.2.2

5.1.0

5.1.0

5.2.2

5.2.2

5.2.2

5.1.0

5.2.2

5.2.2

5.1.0

5.2.2

5.2.2

5.1.0

5.1.0

5.2.2

5.2.2

5.2.2

5.2.2

5.2.2

5.3.0

5.2.2

4.5.2

4.5.2

5.3.0

5.2.2

5.2.2

4.5.2

5.3.0

5.2.2

4.5.2

5.2.2

5.4.0

5.2.2

4.5.2

4.5.2

5.4.0

5.2.2

5.0.1

5.0.1

5.4.0

5.2.2

5.1.0

5.1.0

5.4.0

5.4.0

5.2.2

5.2.2

5.4.0

5.2.2

5.2.2

5.2.2
Note:

The above table is fully valid for customers using SD-WAN services only. Customers requiring access to VMware Cloud Web Security or VMware Secure Access need their Edges upgraded to Release 4.5.0 or later.

Important:

VMware SD-WAN Release 4.0.x has reached End of Support; Releases 4.2.x, 4.3.x, and 4.5.x have reached End of Support for Gateways and Orchestrators.

  • Release 4.0.x reached End of General Support (EOGS) on September 30, 2022, and End of Technical Guidance (EOTG) December 31, 2022. 

  • Release 4.2.x Orchestrators and Gateways reached End of General Support (EOGS) on December 30, 2022, and End of Technical Guidance on (EOTG) March 30, 2023.   

  • Release 4.2.x Edges reached End of General Support (EOGS) on June 30, 2023, and will reach End of Technical Guidance (EOTG) September 30, 2025.

  • Release 4.3.x Orchestrators and Gateways reached End of General Support (EOGS) on June 30, 2023, and End of Technical Guidance (EOTG) September 30, 2023.

  • Release 4.3.x Edges reached End of General Support (EOGS) on June 30, 2023, and will reach End of Technical Guidance (EOTG) September 30, 2025.

  • Release 4.5.x Orchestrators and Gateways reached End of General Support (EOGS) on September 30, 2023, and End of Technical Guidance on (EOTG) December 31, 2023.

  • For more information please consult the Knowledge Base article: Announcement: End of Support Life for VMware SD-WAN Release 4.x (88319).

Upgrade Paths for Orchestrator, Gateway, and Edge

The following lists the paths for customers wishing to upgrade their Orchestrator, Gateway, or Edge from an older release to Release 5.2.2.

Orchestrator

Orchestrators using Release 4.2.0 or later can be upgraded to Release 5.2.2. 

Gateway

Upgrading a Gateway using Release 4.2.0 or later to Release 5.2.2 is fully supported for all Gateway types.

Important:

When deploying a new Gateway using 5.2.2 the VMware ESXi instance must be at least version 6.7, Update 3 up to version 7.0. Using an earlier ESXi instance will result in the Gateway's Dataplane Service failing when trying to run Release 5.2.2 or later.

Important:

Prior to upgrading a Gateway to 5.2.2, the ESXi instance must be upgraded to at least version 6.7, Update 3 up to version 7.0. Using an earlier ESXi instance will result in the Gateway's Dataplane Service failing when trying to run Release 5.2.2 or later.

Edge

An Edge can be upgraded directly to Release 5.2.2 from any Release 4.x or later.

New Hardware Platform

VMware announces the release of the SD-WAN Edge 710-W, an advanced SD-WAN appliance. Available for order from February 1, 2024, it replaces the SD-WAN Edge 510 and 610 models (excluding LTE versions), which will reach end-of-sale on August 1, 2024.

Designed for entry-level use, the 710-W offers high-performance, secure, and scalable WAN connectivity for businesses. It integrates SD-WAN, routers, switches, firewalls, Wi-Fi, and more into a single unit. Please contact your sales team for further details about this new hardware model.

Note:

  • The Edge 710-W is supported when using Edge version 5.2.2.1 and upcoming version 5.2.3.0.

Important Notes

VMware Security Advisory 2024-0008

LAN-Side NAT Behavioral Change

When a LAN-side NAT is configured for many-to-one translations using Port Address Translation (PAT), traffic initiated from the opposite direction can allow unexpected access to fixed addresses based on the outside mask and original IP address. This new behavior applies to Destination NAT (DNAT), Source NAT (SNAT), and Source and Destination NAT (S+D NAT) rules.

For example, a SNAT rule with an inside network of 192.168.1.0/24 and an outside address of 10.1.1.100/32 permits outside-to-inside translation to 192.168.1.100.

To address this new behavior, SD-WAN now blocks traffic when a connection is initiated in the reverse PAT direction.

To restore the original behavior, a user needs to configure two rules of the same type as the original rule (SNAT, DNAT, S+D NAT) in a particular order. For example, using the earlier SNAT scenario a user needs to configure the following:

  1. SNAT rule with an inside network of 192.168.1.100/32 and an outside address of 10.1.1.100/32

  2. SNAT rule with an inside network of 192.168.1.0/24 and an outside address of 10.1.1.100/32

If the original rule is a DNAT or S+D NAT, then the user would need two DNAT or S+D NAT rules with the same structure and order.

In Release 4.5.0 and forward, a user can determine if flows are dropped for this type of traffic in the dispcnt logs of a diagnostic bundle by searching for the counter: lan_side_nat_reverse_pat_drop.

Hub or Cluster Interconnect Remains Early Access

Hub or Cluster Interconnect was introduced in Release 5.1.0 with the caveat:

"Enabling Hub or Cluster Interconnect introduces a fundamental change to the VMware SD-WAN routing protocol where it allows packets to traverse more than one hop in the network. While this change has been tested in representative topologies, it is not possible to test for all routing scenarios that may be encountered when making such a change of allowing distant routes to be distributed. As a result, VMware is releasing this feature as early access and will be closely monitoring deployments where it is enabled for unexpected routing behavior."

The caveat remains in effect for this feature for all Release 5.2.x versions and is only fully GA in Release 5.4.0.

Limitation with BGP over IPsec on Edge and Gateway, and Azure Virtual WAN Automation

The BGP over IPsec on Edge and Gateway feature is not compatible with Azure Virtual WAN Automation from Edge or Gateway. Only static routes are supported when automating connectivity from an Edge or Gateway to an Azure vWAN.

Limitation When Deactivating Autonegotiation on VMware SD-WAN Edge Models 520, 540, 620, 640, 680, 3400, 3800, and 3810

When a user deactivates autonegotiation to hardcode speed and duplex on ports GE1 - GE4 on a VMware SD-WAN Edge model 620, 640 or 680; on ports GE3 or GE4 on an Edge 3400, 3800, or 3810; or on an Edge 520/540 when an SFP with a copper interface is used on ports SFP1 or SFP2, the user may find that even after a reboot the link does not come up.

This is caused by each of the listed Edge models using the Intel Ethernet Controller i350, which has a limitation that when autonegotiation is not used on both sides of the link, it is not able to dynamically detect the appropriate wires to transmit and receive on (auto-MDIX). If both sides of the connection are transmitting and receiving on the same wires, the link will not be detected. If the peer side also does not support auto-MDIX without autonegotiation, and the link does not come up with a straight cable, then a crossover Ethernet cable will be needed to bring the link up.

For more information please see the KB article Limitation When Deactivating Autonegotiation on VMware SD-WAN Edge Models 520, 540, 620, 640, 680, 3400, 3800, and 3810 (87208).

Orchestrator API Changes

Orchestrator API Changes since 5.2.0

Changes to the VMware SASE Orchestrator Portal API ("API v1")

Issue# 125082

Symptom: An Edge overridden configuration for the VLAN DHCP and OSPF is lost and the profile configuration is applied instead. The user cannot optionally override the individual section, specifically DHCP and OSPF in the VLAN configuration.

Description: In the New Orchestrator UI, the design for the VLAN configuration allows only a general/parent override button and removes the capability to optionally override individual sections inside the VLAN, like DHCP and OSPF.

As a result, the API was made compatible with the new UI design and backend code also removed the capability to optionally override the individual sections like DHCP and OSPF in the VLAN configuration. Any user trying to configure individual override via API will not see the configuration getting overridden until the parent override (in other wordsd, the override flag under VLAN configuration) is first enabled.

Resolution:The issue is fixed in 5.2.2. To use the Edge overridden configuration of a VLAN DHCP and OSPF, the parent override button is enabled.

Changes to the VMware SASE Orchestrator API v2

Backward compatibility of Profile and Edge level deviceSettings in API v2 is broken due to Issue #125082, which is covered in detail above. The behavior change is as follows:

If a user makes API v2 calls for the following endpoints with override flags in OSPF and DHCP configurations for their VLANs, they will encounter a ValidationError which reads "override field is not allowed in the request".

  • /api/sdwan/v2/enterprises/{enterpriseLogicalId}/profiles/{profileLogicalId}/deviceSettings

  • /api/sdwan/v2/enterprises/{enterpriseLogicalId}/edges/{edgeLogicalId}/deviceSettings

Developer Documentation

All VMware SASE/SD-WAN API documentation resides on the Developer Documentation Portal at https://developer.vmware.com/apis.

Available Languages

The VMware SASE Orchestrator using version 5.2.2 is localized into the following languages: Czech, English, European Portuguese, French, German, Greek, Italian, Spanish, Japanese, Korean, Simplified Chinese, and Traditional Chinese.

Document Revision History

June 20th, 2024. Twelfth Edition.

May 13th, 2024. Eleventh Edition.

April 22nd, 2024. Tenth Edition.

  • Added a new Edge rollup build R5223-20240410-GA to the Edge/Gateway Resolved section. This is the second Edge rollup build and is the new default Edge GA build for Release 5.2.2.

  • Edge build R5223-20240410-GA includes the fixes for issues #125336 , #130495 , #130777, #138303, and #142529, each of which is documented in this section.

April 2nd, 2024. Ninth Edition.

  • Added an Important Note regarding CVE-2024-22247, which details a missing authentication and protection mechanism vulnerability that impacts an SD-WAN Edge. VMware's response to this vulnerability is documented in VMSA-2024-0008. More information on mitigating this vulnerability is found in the KB article: VMware Response to CVE-2024-22247 (VMSA-2024-0008) (97391).

March 08, 2024. Eighth Edition.

  • Added Open Issue #72762 to the Edge/Gateway Known Issues section.

  • Moved Fixed Issue #119811 from the Orchestrator Resolved Issues section to the Edge/Gateway Resolved Issues section for original GA build R5220-20231213-GA. This issue was misidentified as being caused by a defect in the Orchestrator when the defect and the fix are in the Edge software.

February 29, 2024. Seventh Edition.

  • Added a new Edge rollup build R5222-20240223-GA to the Edge/Gateway Resolved section. This is the second Edge rollup build and is the new default Edge GA build for Release 5.2.2.

  • Edge build R5222-20240223-GA includes the fixes for issues #138006, and #138691, each of which is documented in this section.

  • The following Important Note is added as part of this new Edge build:

    • All previous Release 5.2.2 Edge builds are deprecated in favor of R5222-20240223-GA.

    • Customers upgrading to 5.2.2 should only upgrade to Edge version R5222-20240223-GA.

Note:

This is an Edge-only release. The default Gateway Release 5.2.2 build remains R5221-20240206-GA.

  • Added Open Issue #139476 to the Edge/Gateway Known Issues section.

  • Added Fixed Issues #120620 and #137279 to the Edge/Gateway Resolved Issues section of the GA build R5220-20231213-GA.

February 14, 2024. Sixth Edition.

  • Added a new Edge/Gateway rollup build R5221-20240206-GA to the Edge/Gateway Resolved section. This is the first Edge/Gateway rollup build and is the new default Edge/Gateway GA build for Release 5.2.2.

  • Edge/Gateway build R5221-20240206-GA includes the fixes for issues #121606, #126571, #132877, and #134893, each of which is documented in this section.

  • This rollup also introduces support for the new Edge Hardware Platform: SD-WAN Edge 710-W.

  • Revised the wording for Fixed Issue #131891 to make it clear that an Edge deployed as a standalone can also experience this issue.

January 31, 2024. Fifth Edition.

  • Added #131029 to the Edge/Gateway Resolved Issues section GA Edge/Gateway GA build R5220-20231213-GA . This issue was omitted in error from the First Edition of the Release Notes.

January 26, 2024. Fourth Edition.

  • Added #104776 and #114854 to the Edge/Gateway Resolved Issues section GA Edge/Gateway GA build R5220-20231213-GA . These issues were omitted in error from the First Edition of the Release Notes.

  • Added Fixed Issue #115981 to the Orchestrator Resolved Issues section for the original GA Orchestrator build, R5220-20231214-GA. This issue was omitted in error from the First Edition of the Release Notes.

  • Revised the sentence in Recommended Use that reads:

    • "Release 5.2.2 contains all Edge, Gateway, and Orchestrator fixes that are listed in the 5.2.0 Release Notes."

    • To read:

    • "Release 5.2.2 contains all Edge, Gateway, and Orchestrator fixes that are listed in the 5.2.0 Release Notes, including all UI Builds."

    • This is to make it clear that all fixes found in the twelve UI builds documented in the 5.2.0 Release Notes are included in the 5.2.2 version.

  • Revised the sentence under Resolved in Orchestrator Version R5220-20231214-GA which read:

    • "Orchestrator version R5220-20231214-GA was released on 12-14-2023 and resolves the following issues since Orchestrator version R5204-20230831-GA and UI Build R5204-20231121-GA."

    • To read:

    • "Orchestrator version R5220-20231214-GA was released on 12-14-2023 and resolves the following issues since Orchestrator version R5204-20230831-GA and UI Build R5204-20231201-GA."

    • The key change being including the 12th and final UI Build for 5.2.0: R5204-20231201-GA, as all UI fixes for this build are also included in all builds of Orchestrator version 5.2.2.

January 10th, 2024. Third Edition.

  • Added a new Edge/Gateway Hotfix build R5220-20240104-GA-134893 to the Edge/Gateway Resolved section. This is the new Edge/Gateway GA build for Release 5.4.0.

  • Edge/Gateway Hotfix build R5220-20240104-GA-134893 includes the fix for issue #134893, which is documented in this section.

    Important:

    • All previous Release 5.2.0 and 5.2.2 Edge and Gateway builds are deprecated in favor of R5220-20240104-GA-134893.

    • Customers upgrading their Edges or Gateways to 5.2.2 should only upgrade to R5220-20240104-GA-134893.

  • Added Fixed Issue #131789 to the Orchestrator Resolved Issues section for the original GA Orchestrator build R5220-20231214-GA. This issue was omitted in error from the First Edition of the Release Notes.

December 22, 2023. Second Edtion.

  • Revised the Orchestrator API Changes section to add a backward compatibility issue for customers using API v2. This is the same issue impacting API v1 users with regards to DHCP and OSPF settings on a VLAN using an override flag.

  • Added Open Issue #131997 to the Orchestrator Known Issues section.

December 15, 2023. First Edition.

Edge and Gateway Resolved Issues

Resolved in Edge Version R5223-20240410-GA

Edge build R5223-20240410-GA was released on 04-22-2024 and is the third Edge rollup build for Release 5.2.2.

This Edge rollup build addresses the below critical issues since the second Edge rollup build, R5222-20240223-GA.

  • Fixed Issue 125336: A customer enterprise deployed using port forwarding rules may observe that traffic matching these rules is dropped by the SD-WAN Edge, and once the issue is encountered the behavior persists until a user toggles the Firewall status under Configure > Edge > Firewall.

    The issue is triggered by an Edge interface being down and then coming up later. The Edge marks this interface as pending, but when the interface comes up, the port designation for this interface is not changed for the inbound policy index and this results in a port mismatch which results in the Edge dropping the traffic matching this rule.

  • Fixed Issue 130495: For a customer enterprise using a Cloud Security Service (CSS) with GRE tunnels, if the customer activates a new Edge that is associated with a configuration profile shared by other Edges also using this CSS, the client users at those other locations may observe that traffic using the CSS drops.

    Upon receiving a control plane update for either CSS or Non SD-WAN Destination (NSD) via Edge, GRE tunnels may fluctuate. This is because the tunnel configuration is assumed to change, causing it to be torn down and recreated. The fix ensures that if no changes are detected, the GRE tunnel remains operational.

    On an Edge without a fix for this issue, the workaround is to activate the Edge to an isolated configuration profile and, once the Edge is up, only then transfer it to its proper profile.

  • Fixed Issue 130777: On a VMware SD-WAN Edge where a routed interface is configured to be a LAN interface with NAD Direct turned off, there is a discrepancy between using a Source/Destination Interface and an IP Address, and in some scenarios packets may be allowed when filtering using an IP Address and dropped when using the Interface and vice versa.

    In the reported instance for this issue, if a firewall rule is specified for a specific Edge interface, the Stateful Firewall drops the packets, but if the rule is changed to use an IP Address, the rule is not enforced and traffic is allowed to pass.

    On an Edge without a fix for this issue, avoid using IP addresses for firewall rules.

  • Fixed Issue 138303: An SD-WAN Edge may experience a Dataplane Service failure and restart to recover.

    The issue is the result of a race condition in the Edge’s DNS cache where an entry is being accessed and its reference is stored by a thread which, due to a context switch, another thread deletes. When the former thread is scheduled and tries to access the entry, it triggers a memory violation resulting in an Edge service failure because the reference to the DNS cache entry is no longer valid.

  • Fixed Issue 142529: When a customer switches a Cloud Security Service (CSS) GRE tunnel configuration from manual to automatic for an Edge, the Edge may experience three successive Dataplane Service failures and stop passing traffic entirely until restarted.

    The Edge will still be reachable on the management side through the Orchestrator UI and the service can be restarted through Remote Actions.

    When a CSS tunnel with GRE tunneling protocol is switched from manual to automatic, this causes a change in the tunneling protocol from GRE to IPsec. However the IPsec-related parameters are not configured, resulting in the IKE identification of NULL, and this triggers an exception in the Edge service and the resulting triple failure.

    On an Edge without a fix for this issue, if the customer wishes to switch from manual to automatic, the workaround is to delete the CSS configuration completely and create a new one in a maintenance window.

Resolved in Edge/Gateway Version R5222-20240223-GA

Edge build R5222-20240223-GA was released on 02-26-2024 and is the second Edge rollup build for Release 5.2.2.

This Edge rollup build addresses the below critical issues since the first Edge rollup build, R5221-20240206-GA.

Important:

  • All previous Release 5.2.2 Edge builds are deprecated in favor of R5222-20240223-GA.

  • Customers upgrading to 5.2.2 should only upgrade to Edge version R5222-20240223-GA.

Note:

This is an Edge-only release. The default Gateway Release 5.2.2 build remains R5221-20240206-GA.

  • Fixed Issue 138006: For a customer site deployed with a High Availability topology, the customer may observe the Standby Edge restart as a result of having high memory usage.

    If memory usage is sufficiently high, the Standby Edge defensively restarts its Dataplane Service to clear the issue. The restart can impact traffic on Enhanced HA topologies where the Standby is passing traffic. In a Standard HA topology, there is no functional impact.

    The high memory usage is the result of a memory leak which is caused by the Edge refreshing all IPsec tunnel certificates which triggers a tear down and rebuilding of all IPsec tunnels even if there is no change in the certificate and this high number of revoked certificates are not cleared from the Standby Edge's memory.

  • Fixed Issue 138691: A VMware SD-WAN Edge may experience a Dataplane Service failure, generate a core file and restart to recover. On a customer site deployed with a High Availability topology, both the Active and Standby Edge experience this issue and client users would experience traffic disruptions the same as in a standalone topology.

    As part of Fixed Issue #129923, Engineering added modifications to manage recursive route resolution. In this context, there is a potential for memory corruption of resolved connected route leading to the Edge service failure. This process introduces the potential for a use-after-free vulnerability in route objects, which can lead to the corruption of the route object's memory pool, and if this corrupted memory pool is accessed later, it can result in the Edge service failing.

Resolved in Edge/Gateway Version R5221-20240206-GA

Edge/Gateway build R5221-20240206-GA was released on 02-14-2024 and is the first rollup build for Release 5.2.2.

This Edge/Gateway rollup build addresses the below critical issues since the Hotfix build, R5220-20240104-GA-134893.

  • Fixed Issue 121606: Customer enterprises using Partner Gateways may observe that traffic drops for some traffic including Non SD-WAN Destinations using that Gateway.

    Release 5.1.0 and later Partner Gateways support a maximum of 64 IP addresses per IPsec interface. For a Partner Gateway, handoff IPs are getting added to this IPsec interface unconditionally. If the number of handoff IP addresses exceeds the 64 limit, older IP addresses are overwritten on the IPsec process and this results in the tunnels using those overwritten IP addresses to go down.

    For a Gateway using an earlier version, if all the Partner Gateway handoff IP addresses are configured as expected, there is no workaround beyond moving some of those addresses to a different PG (for example, moving an NSD via Gateway to a less utilized PG). However, if there are unnecessary PG handoff IP addresses, removing them may help if this action results in the reduction of the total handoff IP addresses to 63 or less. After a configuration change, the Gateway service needs to be restarted.

  • Fixed Issue 126571: A VMware SD-WAN Edge may experience multiple kernel panics and core dumps resulting in the Edge restarting repeatedly.

    These kernel panics are the result of an out of memory (OOM) condition. While writing a core to the Edge's persistent storage, page allocation for the file system and I/O writes exacerbate the memory consumption and eventually cause the OOM condition.

    Since core dumps are write-only from the kernel, it is unnecessary to keep anything in the Edge's memory cache. As a result, the fix for this issue involves bypassing the Edge's page-cache completely and writing out the core dump and then synchronizing to flush the file system and I/O cache.

  • Fixed Issue 132877: For a customer enterprise deployed with one or more Hub Clusters, customer traffic destined for another Spoke Edge may randomly drop if one or more Cluster members perform a certificate renewal.

    After a member of the Hub Cluster has their certificate renewed (which causes their tunnels to be torn down and rebuilt), the SD-WAN Gateway connected to the Cluster do not advertise some Spoke Edge routes, even though the tunnels are up. As a result the Gateway does not forward traffic for some Branch to Branch routes.

    On a customer enterprise where the connected Gateway does not have a fix for this issue, the only workaround is to restart the Spoke Edge(s) where customers are experiencing this issue as this restores the missing routes.

  • Fixed Issue 134893: Customers may observe that cloud/internet traffic which traverses a Gateway is failing when the connected Gateway uses version 5.2.2.

    In this situation, Edge-to-Edge traffic continues to work properly even while cloud/internet traffic fails when using the Gateway (in other words, multi-path traffic).

    SD-WAN software includes flow compatibility processing, which handles differences in ICMP processing between SD-WAN software versions prior to and after version 5.2.2. With this issue, when one or more Edges use version 5.2.1 or earlier, send ICMP traffic, and are connected to the Gateway using 5.2.2, the ICMP processing allows SD-WAN Gateway and Edge flows to be released without proper cleanup. This results in stale NAT entries on the Gateway that are never removed and causes NAT tables to reach capacity with no free entries available. Lacking free NAT entries, Edge cloud/internet traffic using an affected Gateway deployed as their primary would fail.

    As noted above this issue is only encountered if there are Edges using 5.2.1 or earlier which also send ICMP traffic while connected to a Gateway using 5.2.2. If all the Edges are using version 5.2.2 or later or do not send ICMP traffic, the issue is not encountered.

    Note:

    The fix is primarily for the SD-WAN Gateway as the field-found issues have been on this component. However, the Edge could potentially experience this issue and impact traffic for local users. As a result customers can also upgrade their Edges to the hotfix build to mitigate the risk of this issue for their sites.

  • Fixed Issue 136681: A VMware SD-WAN Edge may experience a Dataplane Service failure, generate a core file and restart to recover. On a customer site deployed with a High Availability topology, both the Active and Standby Edge experience this issue and client users would experience traffic disruptions the same as in a standalone topology.

    In an upgrade scenario, the Orchestrator configurations designed for the 5.2.0.2 version are transmitted to the Edge, where they are parsed and applied. As part of the configuration management process, the Edge parses Cloud Gateway configurations and modifies the default "v" routes (IPv4 and IPv6) by either adding or removing them. During this process, there is a potential for a use-after-free vulnerability in route objects, which can lead to the corruption of the route object's memory pool. If this corrupted memory pool is accessed later, it can result in the failure of the Edge service.

Resolved in Edge/Gateway Version R5220-20240104-GA-134893

Edge/Gateway build R5220-20240104-GA-134893 was released on 01-10-2024 and is a Hotfix build for Release 5.2.2.

This Edge/Gateway Hotfix build addresses the below critical issue since the original GA build, R5220-20231213-GA.

Important:

  • All previous Release 5.2.0 and 5.2.2 Edge and Gateway builds are deprecated in favor of R5220-20240104-GA-134893.

  • Customers upgrading their Edges or Gateways to 5.2.2 should only upgrade to R5220-20240104-GA-134893.

  • Fixed Issue 134893: Customers may observe that cloud/internet traffic which traverses a Gateway is failing when the connected Gateway uses version 5.4.0.

    In this situation, Edge-to-Edge traffic continues to work properly even while cloud/internet traffic fails when using the Gateway (in other words, multi-path traffic).

    SD-WAN software includes flow compatibility processing, which handles differences in ICMP processing between SD-WAN software versions prior to and after version 5.2.2. With this issue, when one or more Edges use version 5.2.1 or earlier, send ICMP traffic, and are connected to the Gateway using 5.2.2 or later, the ICMP processing allows SD-WAN Gateway and Edge flows to be released without proper cleanup. This results in stale NAT entries on the Gateway that are never removed and causes NAT tables to reach capacity with no free entries available. Lacking free NAT entries, Edge cloud/internet traffic using an affected Gateway deployed as their primary would fail.

    As noted above, this issue is only encountered if there are Edges using 5.2.1 or earlier which also send ICMP traffic while connected to a Gateway using 5.2.2 or later. If all the Edges are using version 5.2.2 or later, or do not send ICMP traffic, the issue is not encountered.

    Note:

    The fix is primarily for the SD-WAN Gateway as the field-found issues have been on this component. However, the Edge could potentially experience this issue and impact traffic for local users. As a result customers can also upgrade their Edges to the hotfix build to mitigate the risk of this issue for their sites.

Resolved in Edge/Gateway Version R5220-20231213-GA

Edge/Gateway version R5220-20231213-GA was released on 12-14-2023 and resolves the following issues since Edge version R5202-20231107-GA-125647 and Gateway version R5202-20230725-GA. This means that a fix for any Edge or Gateway issue listed in the 5.2.0 Release Notes is included in all Release 5.2.2 builds.

  • Fixed Issue 67001: On a customer enterprise where the Stateful Firewall is activated, the firewall blocks valid challenge ACK packets.

    The Stateful Firewall drops challenge ACK packets and does not log them even though the industry standard is to send a TCP RST for a challenge ACK message.

    This software update ensures the Stateful Firewall allows valid TCP RST packets and blocks invalid (for example, replayed or spoofed) TCP RST messages. 

  • Fixed Issue 69641: For a customer enterprise that uses one or more Business Policies which include rate limits, the customer may observe packet drops on all flows, even those unrelated to the rate limited Business Policy flows and on other segments and peers.

    Setting a rate limited Business Policy and sending higher demand traffic (more than the limit) with a high number of flows results in packets from other flows being dropped (even from other segments and peers) due to the net scheduler buffer limit being hit.

    On an Enterprise whose Edges do not have a fix for this issue, the workaround would be to remove the rate limit configuration and instead reclassify the traffic matching the rule with the lowest possible value (Low, Bulk).

  • Fixed Issue 74422: In cases of High Availability, the Edge may go offline if only the Standby Edge has a WAN link which is up and has a valid IP address.

    This issue occurs when a WAN link has DHCP enabled where only the Standby Edge has a WAN link available. When the Standby WAN link receives an IP address from the DHCP server, it sends the interface details to the Active Edge. The Active Edge makes a call to add the IP address as a route, however this function does not add the route to the Linux kernel. The Edge function only adds the route to the FIB (forwarding information base). As a result, the Edge's management process throws an error as there is no route present in Linux kernel route table for the packet to exit and the site is effectively offline.

  • Fixed Issue 87304: If a user deactivates a LAN interface on a VMware SD-WAN Edge using the VMware SASE Orchestrator UI, the interface will still be reported as 'UP' by SNMP.

    The key debug process for interfaces output does not include the physical port details for Edge LAN interfaces (for example, GE1 or GE2). As a result when SNMP polls those interfaces it always returns a result of UP regardless of how these interfaces are configured.

  • Fixed Issue 96334: On a VMware SASE Orchestrator where the IP address changes frequently, the VMware SD-WAN Edges may lose contact with the Orchestrator and be reported as offline.

    In this scenario where the Orchestrator IP address changes frequently, the Edges are responding to each Orchestrator IP address change by changing the management traffic source from a loopback interface to the GE1 port IP address, even when the Orchestrator is configured to exclusively use the loopback interface as the source. As result the Edges lose contact with the Orchestrator and while this does not impact customer traffic, this issue prevents configuration updates and monitoring from working as expected.

  • Fixed Issue 101935: A VMware SD-WAN Edge may be accessible via SSH during bootup even though the configuration denies access.

    When the Edge is rebooting, it can be accessed via SSH for a time window of 10-15 seconds. The SSH is blocked only after this time, as per the configuration.

  • Fixed Issue 103049: Polling a VMware SD-WAN Edge via SNMP may not work when SNMPv3 is configured.

     When a user turns on SNMP and sets up SNMPv3 user credentials via the Orchestrator prior to activating the Edge, if the user tries to poll the Edge via SNMP the Edge does not respond.

    On an Edge without a fix for this issue, the workaround is to change any SNMPv3 setting (like adding or updating a user) and then change it back to its original setting.

  • Fixed Issue 104776: For a customer configuring a VMware SD-WAN Edge interface for PPPoE, when the WAN Overlay settings for that interface include 802.1P Setting, the Edge does include the 802.1P PRI bits on outgoing traffic.

    The Edge does not include the configurable option for netifd to set 'EGRESS priority mappings' for PPPoE interfaces and the result is the Edge does not mark those packets with PRI.

  • Fixed Issue 105160: Upgrading the software of a VMware SD-WAN Edge may fail and the Edge does not retry the upgrade.

    When the issue is encountered, there is an exception in the Edge upgrade process which causes the Edge to update its configuration version for an Edge software upgrade without the Edge actually upgrading. As a result, the Edge will think it upgraded to the target version and make no attempt to retry the actually failed upgrade.

    The only way to correct an Edge in this state is to change the Edge version (downgrade or upgrade at the customer's discretion) and then after that Edge software update, retry the desired upgrade for the Edge.

  • Fixed Issue 106160: With an Edge configured DNS server and a next hop Gateway defined for an Edge interface for which clients query the DNS server, there is no response.

    The DNS request packet is received by the Edge DNS server as expected. However, the reply packet does a route table lookup based on an iptables connection tracking and finds the next hop Gateway IP address and resolves the MAC address. The result is the DNS reply packet will use the MAC address of the Gateway, not the sender.

  • Fixed Issue 107550: For customer enterprises where Non SD-WAN Destinations via Gateways are deployed, a user may observe some IPsec encrypted packets drop in the path.

    The current implementation uses an inner IP header Time-to-live (TTL) value and this does not match the RFC requirement. As a result, the TTL value must be constructed and if the packet originator uses a low TTL value then there is a chance that this packet will not reach its destination.

  • Fixed Issue 109906: A VMware SD-WAN Gateway may experience a Dataplane Service failure, generate a core, and restart to recover.

    This issue can be encountered when a corrupt out of band message is received which causes an array index overflow and triggers an exception and the failure of the Gateway's service.

  • Fixed Issue 111592: For a customer enterprise using a Hub/Spoke topology where Business Policies are configured to use internet backhaul, internet traffic using the backhaul rule may be either slow or not work at all.

    In some instances during the creation of the flow, the Business Policy matching is changed due to updated Deep Packet Inspection (DPI) information. This could lead to the loss of the logical ID of the Hub Edge or Non SD-WAN Destination, which is supposed to backhaul the packets.

  • Fixed Issue 112115: A VMware SD-WAN Edge under a high CPU load may experience a Dataplane Service failure and restart to recover.

    Under high CPU conditions, multiple service failures triggered by a mutex monitor can occour due to a lower priority thread acquiring the debug ring lock. The resolution to this issue is an enhancement to the Dataplane that makes that particular thread both lock-free and wait-free. 

  • Fixed Issue 112826: When a customer exports a list of Alerts to a CSV file through the Orchestrator UI, they only retrieve 512 items.

    Doing the same export using an API can deliver up to 2048 Alerts, which is the expected amount for the Orchestrator UI export as well. The 512 limitation impacts customers exporting Alerts where the number of Alerts exceeds that 512 limit for the specified time span.

  • Fixed Issue 114562: Rate limiting may not work for SSH flows.

    When a business policy is created to rate limit the transit SSH flows, the rate limit setting will not honored although the business policy is applied successfully. This is because these flows are considered control flows although the SSH is not for the Edge, but for some remote devices.

  • Fixed Issue 114854: For a user troubleshooting a VMware SD-WAN Edge model 610 with DPDK activated, running a packet capture from the Orchestrator, or using tcpdump.sh or vctcpdump shows that the VLAN tag is missing for return traffic.

    The lack of VLAN tags on return traffic impacts a user's ability to successfully troubleshoot a network issue with any version of an Edge 610.

  • Fixed Issue 114988: ICMPv6 message 'Packet Too Big' is not received either from or via a VMware SD-WAN Gateway.

    The Gateway data path consumes all ICMPv6 'Packet Too Big' messages locally. The fix ensures the Gateway sends the appropriate destination.

  • Fixed Issue 115262: BGP neighborship with a secondary IP address may not come up on a VMware SD-WAN Edge.

    If a user configures BGP neighbor first and then configures the corresponding secondary IP address on a VLAN interface, the BGP session may not come up. This is caused by the Edge not updating the BGP configured interface with the secondary IP address removal/addition on a VLAN interface.

  • Fixed Issue 115604: A VMware SD-WAN Edge or Gateway may experience a Dataplane Service failure and generate a core with an Assert in the logging.

    When an Edge or Gateway processes a corrupted packet, the software can hit an assert where actual user packet length is more than internal packet buffer. The Gateway is expected to drop this kind of packet and prevent it from being sent to the Edge, but instead processes it and this results in the service failure and restart.

  • Fixed Issue 115869: Tunnels re-establish to a VMware SD-WAN Gateway in the middle of its upgrade process.

    In a scaled environment where there are 1000s of tunnels and peers connecting to a Gateway, when a Gateway is upgraded, the traffic switches to the secondary Gateway by design to ensure a short downtime and resume traffic flow quickly. When the upgrade script is running on the Gateway being upgraded (from 4.5.1 to 5.2.0 or from 5.0.1 to 5.2.0 or 5.1.0 to 5.2.0), in the middle of the upgrade script execution, the tunnels start coming back up on the Gateway being upgraded and traffic again switches from the Secondary Gateway to the Gateway being upgraded. And then when the script ends, again the Gateway requires a reboot and once again the traffic is switched to a secondary Gateway. This can cause major customer traffic disruptions for Multipath type traffic using the Gateway.

    On a Gateway without a fix for this issue, the workaround is to move vc_proc_mon restart from the post-installation script to after the upgrade completes.

  • Fixed Issue 115904: When a user triggers a diagnostic bundle for a VMware SD-WAN Edge using the VMware SASE Orchestrator, the Edge may experience a Dataplane Service failure, generate a core, and restart to recover.

    A user can generate an Edge diagnostic bundle on the SD-WAN > Diagnostics > Diagnostic Bundle page. When this action is taken, a race condition between dns_name_cache (addition and/or delete) and the DNS name cache can occur which causes the Edge service to try and access an in use or deleted element, which triggers a service failure with a SIGSEGV or SIGBUS reason.

  • Fixed Issue 116049: The VPN State of a WAN link configured as a backup can go to a DEAD state instead of the expected STANDBY state.

    The impact to the customer is lessened because the Backup WAN link functionality (link becoming ACTIVE when other paths go down) remains unaffected. However the displayed UI status of the WAN link as DEAD can cause confusion for the customer, and if the backup WAN link is in fact down, the customer would not be able to tell through the Edge > Monitor page when this issue is experienced.

    This issue can occur where the enterprise is connected to a Partner Gateway and the BGP handoff IP address configured is not an IP address of any Edge interface in that segment. In that scenario, the Edge's backup link check messages can be dropped. As a result, the WAN links configured as backups for Edges can be marked as DEAD instead of STANDBY even though the link is up.

  • Fixed Issue 116257: For a VMware SD-WAN Edge connected through a Partner Gateway where a NAT handoff is configured for a remote server, return traffic to the Edge may drop from that server.

    If the traffic is initially not encrypted from the Edge to the remote server and then updated with an encrypted flag, once the route is updated, the reverse traffic is dropped on the Edge due to a route lookup failure.

    The issue can be temporarily resolved by flushing flows on the affected Edge.

  • Fixed Issue 116368: The routing logs on a VMware SD-WAN Gateway may reach capacity and not accumulate any additional entries.

    This issue is caused by the Gateway's routing software missing the log rotation configuration, whose purpose is to rotate the routing logs prior to reaching capacity so that new log entries can be added. Without this configuration, the routing logs do not rotate and result in Operators and Partners potentially missing critical log entries for a Gateway.

  • Fixed Issue 116428: On a customer deployment where multiple segments are configured and each non-Global segment has a custom name, when running Remote Diagnostics > Ping Test, the drop down menu to choose an interface does not show the custom name for each segment.

    Instead of the custom name for each non-Global segment, the user sees a generic name with a number: Segment 1, Segment 2, and so forth. The is the result of the Edge hard-coding the segment name for each non-Global segment.

  • Fixed Issue 116578: When running a FTPv6 server connected locally to a VMware SD-WAN Edge, the FTP data channel may not be be classified when looking at flow dumps.

    Flows may be misidentified by the Edge if the source and destination IPv6 addresses are highly similar.

  • Fixed Issue 116827: A VMware SD-WAN Edge may experience a Dataplane Service failure and restart to recover from the condition.

    The Edge can experience this issue because of a race condition during the Edge startup that causes the Edge service to fail due to uninitialized data.

  • Fixed Issue 117037: For a customer using a Hub/Spoke topology where multiple WAN links are used to send and receive traffic from the Spoke Edge to the Hub Edge, customers may observe lower than expected performance for traffic that is steered by Business Policies because the WAN links are not aggregating the WAN link's bandwidth.

    SD-WAN uses a counter for accounting the number of packets buffered in a resequencing queue. This counter is managed per peer and used to make sure only 4K packets are buffered per peer. Under some conditions, this counter can become negative. Prior to Release 4.2.x, when this counter became negative, the respective counter was immediately reset back to 0 after flushing the packets in the resequencing queue. However, starting in Release 4.3.x, this counter is updated automatically to ensure that the counter stays within expected bounds.

    The result of this change in behavior can cause cases where the counter accounting is incorrect and the resequencing queue can stay at a very high number to which SD-WAN reacts by flushing every single packet. This action not only prevents bandwidth aggregation but can reduce the effectiveness of flows that would otherwise be on a single link.

    On Edges without a fix for this issue, the workaround is to configure business policies that steer matching traffic to a single mandatory link.

  • Fixed Issue 117314: When an ICMP flow already exists between a Source and Destination IP address pair, a Firewall rule that uses an Object Group/Service Group (type and code) to filter ICMP packets may not work.

    As part of a revision to Firewall functionality, the changes introduced for caching ICMP type and code were reverted, and this impacted Firewall rules which used a Service Group with an ICMP type and code (for example, ICMP Redirect Type 5 and Code 0). If a flow is already present between a source and destination IP address, then ICMP traffic that should match the rules for this flow will not be honored, and only the first packets for a session will match the Firewall rules. The issue impacts either IPv4 or IPv6 ICMP flows.

    Flushing flows to create new ICMP flows will temporarily remediate the issue.

  • Fixed Issue 117320: For a customer where Stateful Firewall is activated and Syslog turned on, Syslog messages for traffic that matches a LAN side NAT rule does not include the source IP address.

    A fully complete Syslog message for any traffic is expected to include the source IP address, especially for traffic being NAT'd.

  • Fixed Issue 117565: Users on a customer enterprise configured with a Partner Gateway may observe that MultiPath traffic (traffic that traverses a VMware SD-WAN Gateway) drops.

    Traffic going direct to the internet/cloud or Hub-to-Spoke traffic is not affected as this traffic does not use a Gateway. The issue is triggered when the Partner Handoff is deactivated for the Gateway, which results in all Gateway IPsec (VCMP) tunnels going down for that Gateway to the customer enterprise. The issue is caused by the Gateway handoff IP address not being cleared after the handoff is deactivated and the Gateway continues to perform the same subnet check with this now invalid handoff IP address.

    Rebooting the Gateway will resolve a particular instance of the issue but does not prevent recurrences under the same conditions.

  • Fixed Issue 117638: When a user navigates to Monitor > Edge > Links and turns on Live Mode for a VMware SD-WAN Edge using Release 5.2.0, the SASE Orchestrator does not deliver any realtime statistics.

    In addition the user observes a 'Waiting for Edge ...' indication that eventually times out. The issue is caused on the Edge due to the way a 5.2.0 Edge build handles LTE/USB link statistics when uploading them to the Orchestrator.

  • Fixed Issue 117775: A Non SD-WAN Destination via Gateway (NSD) may intermittently get into a state where the IPsec tunnels flap (are torn down and built back up) constantly.

    The customer would observe that the tunnels are up for several seconds, and then down for several seconds, and then back up with this cycle repeating before stopping on its own. Because the issue is timing-based, the cycle could repeat over days and potentially infinitely. The issue occurs based on a race condition when an NSD tunnel with a large number of IKE Phase 2 connections is coming up, and there is traffic the Gateway is trying to forward on one of those IKE Phase 2 connections before it is fully up, causing the whole tunnel to be torn down and rebuilt, and the cycle repeats.

    On a site not using Gateways with a fix for this issue: because this is a timing-based issue workaround options are limited. One potential workaround is to configure the NSD tunnels in small batches so that they are negotiated faster and thus avoid the window in which traffic is arriving before the tunnel is ready.

  • Fixed Issue 117838: A VMware SD-WAN Edge may experience a Dataplane Service failure, generate a core, and restart to recover.

    The issue can be encountered when the Edge receives an ike packet with a mismatched version when compared to ike_ds. The is sue is caused when the Edge service is processing mismatched version packets and takes a mutex lock for the sake of modifying a cookie value in ike_ds, but if a version mismatch happens the Edge triggers a security association (SA) deletion which again tries to take mutex on the same ike_ds. The result is an Edge deadlock and service failure with resulting restart.

  • Fixed Issue 118097: An Operator or Partner user when debugging a VMware SD-WAN Gateway may find that the command debug.py --path command does not return a result.

    The issue is caused by an unhandled key when a transient tunnel is present, and this breaks the debug.py --path command while the Gateway is processing transient paths.

  • Fixed Issue 118333: For a customer site deployed with a High Availability topology where the HA Edge pair is either a model 520, 540, or 610, the customer may observe multiple HA failovers due to the site experiencing an active-active (split brain) condition.

    VMware SD-WAN Edge 520, 540, and 610's use a switch made by Marvel where if internet backhaul is configured can trigger a situation where the Standby Edge also becomes active while not demoting the Active Edge. Active-Active states are resolved by rebooting the Standby Edge and this will be recorded in the Edge Events.

  • Fixed Issue 118591: For a customer site deployed with an Enhanced High Availability topology, a VMware SD-WAN Edge in the Standby role may have a WAN interface flap frequently.

    In Enhanced HA when a high number of flows are sent or a high number of routes installed, The Standby Edge WAN interface state can be moved from UP to DOWN and back to UP.

  • Fixed Issue 118938: NTP traffic may be dropped by a VMware SD-WAN Edge.

    The issue is the result of the Edge performing a strict check on the destination IP address during the segnat table updates which results in NTP traffic being blocked. The fix for this issue adds an exception to NTP traffic during the segnat table update.

  • Fixed Issue 119491: For a VMware SD-WAN Edge where Edge Network Intelligence Analytics is activated, the customer may observe a gradual increase in Memory Usage on the Edge.

    The specific scenario is an Edge where Analytics is activated and is also receiving RADIUS traffic, in that instance an Edge memory leak can happen. If the memory leak continues for a sufficient period to bring the Edge's memory usage beyond the critical threshold of 70% of available RAM, the Edge will defensively restart its service to clear the leak which can result in customer traffic disruption for 10-15 seconds.

  • Fixed Issue 119544: When ICMP Echo Response is turned off on a VMware SD-WAN Edge's loopback interface, L7 health check will fail with the result that the Edge will trigger a teardown of a CSS tunnel where L7 health check is configured.

    In addition, when the management traffic goes direct, the Edge to Orchestrator communication is also lost.

    When the Edge tries to send a L7 health check request (HTTP SYN packet) it will reach the loopback interface and since ICMP Echo Response is turned off, this results in the dropping of HTTP packets. When L7 health check does not get an ACK for the SYN packet it has sent the L7 health check will fail, and it will lead to a teardown of CSS tunnel.

    Similarly, when the Edge tries to send HTTPS packets towards the Orchestrator, it will reach the loopback interface and since ICMP Echo Response is turned off, it will result in the dropping of HTTPS ACK packets.

  • Fixed Issue 119811: A customer may observe that in their Events list there are multiple MGD_WEBSOCKET_INIT and MGD_WEBSOCKET_CLOSE Edge Events per day.

    There can be multiple MGD_WEBSOCKET_INIT and MGD_WEBSOCKET_CLOSE events shown in the Orchestrator's Event list without any customer actions.

    These event messages are spurious and can be safely ignored as they carry an "Info" level of severity.

  • Fixed Issue 119853: For a customer site deployed with a High Availability topology, when there is a TCP flap between the Active and Standby Edges, the client users for that HA Edge would observe traffic loss.

    On a TCP flap triggered between the Active and Standby Edge, the HA Edges reset their link states, which causes paths using that link also being torn down and rebuilt and this results in traffic loss for user traffic using that HA Edge interface.

  • Fixed Issue 120173: For a customer enterprise site configured with a High Availability topology, the user may observe that the Standby Edge experience a Dataplane Service failure and restarts.

    There are instances where the flow synchronization packet has been corrupted when it is received by the Standby Edge. When the Edge service reads fields from this packet, there is an exception triggered when invalid data is present which results in the restart.

  • Fixed Issue 120308: A VMware SD-WAN Edge using a 5.2.x or earlier software release which is deployed on a VMware SASE Orchestrator using a 5.4.x software release can be configured to use the Prefix Delegation feature.

    The Orchestrator should only allow the Prefix Delegation feature to be configured for an Edge using a 5.4.x software version as this feature is only supported in 5.4.0 and later.

    There is an additional impact in this issue where if the Edge is configured for Prefix Delegation, that 5.2.x or earlier Edge cannot be upgraded to a 5.4.x software version.

  • Fixed Issue 120620: For a customer enterprise site where the SD-WAN Edge is configured as a DHCPv6 relay or server, client users may experience latency for packets that use this process.

    There can be a delay in the SD-WAN Edge sending DHCPv6 Router Advertisement (RA) related information. The Edge does not respond to Router Solicitations (RS) but only sends out periodic RAs and this causes the delay in sending an RA when the Edge is configured as a DHCPv6 relay or DHCPv6 server.

    For a site experiencing this issue without a fix, as a temporary measure, a low advertisement interval can be manually configured on the Edge in /var/etc/radvd.conf. For each interface section, MaxRtrAdvInterval and MinRtrAdvInterval should be set to very small values in seconds. This will ensure that the Edge sends RAs more frequently to compensate for dropped RS messages. However, manual modification of configuration files is not recommended, as any configuration change would be overwritten later and the preferred solution is to upgrade the Edge to a build with the fix included.

  • Fixed Issue 121024: Remote Access Service (RAS) traffic fails when a Business Policy matching internet traffic is configured on a VMware SD-WAN Edge.

    When a Business Policy matching internet traffic is configured on an Edge and the action is to streer this traffic Direct, then for any Remote Access Service traffic that reaches this Edge, the return traffic matches this Business Policy and it is dropped on the Edge.

    The only workaround is to configure a more specific Business Policy matching RAS subnets and set the action to send it Multipath (via the Gateway).

  • Fixed Issue 121031: On a customer enterprise site deployed with a High Availability topology, if a user changes an HA Edge device setting and saves changes, traffic to a newly configured segment that is steered by a business policy rule may fail.

    On an HA Edge, when a new segment is created and attached to an Edge interface, traffic bound for this new segment is impacted due to an issue in the business policy rule.

    On an HA Edge without a fix for this issue, the user can add a dummy rule or change the existing business policy rule for the newly created segment.

  • Fixed Issue 121281: For a customer enterprise site deployed with a High Availability topology, in rare instances the customer may observe that the Standby Edge has experienced a Dataplane Service failure, generated a core, and restarted to recover.

    There would be an Event showing the Standby Edge restarting and if the user configures an HA Failed alert, they would be notified of this event. The issue is experienced in rare scenarios when a route is synchronized between the Active and Standby and the Standby Edge service fails due to memory corruption while processing the route synchronization message.

  • Fixed Issue 121338: For a site where a WAN link is configured to be a Hot Standby, the VMware SD-WAN Edge includes that link as part of the available bandwidth.

    A Hot Standby WAN Link is by design idle and should not be included in an Edge's available bandwidth. Because the Hot Standby is included, the Edge makes inaccurate calculations for total available bandwidth and can result in packet loss.

  • Fixed Issue 121368: A VMware SD-WAN Gateway may experience a Dataplane Service failure, generate a core, and restart as a result.

    The issue is triggered by remote access users accessing the Internet/Cloud through the Gateway. If the Internet/Cloud endpoint responds with a large packet that requires fragmentation, the Gateway service fails while attempting to fragment the packet.

  • Fixed Issue 121513: For a customer enterprise using a Partner Gateway where that Gateway has the "Secure BGP Routes" option activated, client users may observe traffic quality issues.

    Traffic may be dropped at the Edges when traffic is initiated with a BGP peer IP address as Source behind a Partner Gateway when Secure BGP Routes are configured. This is because when traffic is initiated from a BGP endpoint as Source, it creates an unsecured flow in the Gateway, because the source route will be of type BGP-Peer, for which secure setting handling is not done. However, if the source route lookup at the Edges returns a secure route, there will be a mismatch in the secure setting of the incoming traffic and route lookup. This will result in source route lookup failure at the Edges.

  • Fixed Issue 121998: For a customer using the Stateful Firewall in a Hub/Spoke topology, traffic that matches a firewall rule configured for Spoke-to-Hub traffic where the rule includes a source VLAN may be dropped.

    When there is an application classification, business policy table, or firewall policy table version change, SD-WAN performs a firewall lookup for flows on its next packet. Due to a timing issue, that packet could be one from the management traffic (VCMP) side. As a result, during a firewall policy lookup key creation, SD-WAN swaps the Spoke Edge VLAN with the Hub Edge VLAN, and this leads to not matching the rule and dropping that traffic.

    For an Edge without a fix for this issue, a customer can change the Source from an Edge VLAN to 'Any'.

  • Fixed Issue 122029: A Cloud Security Service (commonly, Zscaler) deployed with GRE Automation where the VMware SD-WAN Edge uses a 5.2.x release does not work.

    This issue is caused by both the local IP address and public IP addresses not being sent and this information is required for the Orchestrator to send a configuration state of the tunnel down to the Edge. The tunnel needs to be in a pending state for the Orchestrator to send the automated GRE configuration.

    This issue is limited to 5.2.x Edges and a user could work around this only by downgrading the Edge to a 5.1.x or earlier release and bring the tunnel up and then upgrade to 5.2+ release.

  • Fixed Issue 122167: For a VMware SD-WAN Edge where a WAN link is configured as Backup, if the backup link goes down there is no 'Link dead' event generated by the Edge and sent to the Orchestrator.

    The issue is caused by the Edge not handling link state transitions properly when an interface is down.

  • Fixed Issue 122416: For a large scale customer enterprise site deployed with a High Availability topology where the site has a high number of Branch to Branch routes, when a failover is triggered client users may observe traffic loss since missing routes need to be relearned once tunnels are established.

    A large scale enterprise is understood as one with ~4K Branch Edges and a high number of routes is understood as at least ~8K.

    The issue is the result of an unexpected delay in receiving the Hub Edge configuration.

  • Fixed Issue 122426: If a customer performs an SNMP query for a VMware SD-WAN Edge interface configured to use DPDK, a customer may experience a longer than expected delay in getting results.

    The delay is caused by a back-end script for collecting the interface data that is not properly optimized.

  • Fixed Issue 122528: For a customer enterprise which uses WAN static routes with ICMP probes configured, the ICMP probes may stop functioning on multiple VMware SD-WAN Edges at once with all traffic using those routes dropping.

    Each Edge has an ICMP probe sequence counter with a maximum number of 65535 iterations. When this counter rolls over after 65535 iterations, the probes fail.

    On an Edge without a fix for this issue, the workaround is to remove the ICMP probe, restart the Edge service, and then restore the probe.

  • Fixed Issue 123128: For a customer site configured with a High Availability topology where the HA Edges are Edge models 520, 540, 610, or 620, a customer may observe that the Standby Edge restarts multiple times.

    This issue can be observed in Events. In addition, if an Enhanced HA topology is used where the Standby Edge also passes traffic, client user traffic using the WAN link(s) on the Standby Edge would also be impacted.

    This issue can only occur on the listed Edge models, each of which uses its kernel service to forward traffic and the kernel threads run at lower priority and packets can get queued in the kernel threads for more than 700ms. When the packets are queued for more than 700ms the Standby Edge misses the HA heartbeat, causing the Standby Edge to become active and that triggers an active/active state where the Standby Edge is restarted to recover the state.

  • Fixed Issue 123144: When on the Orchestrator UI page Monitor > Edge > Destinations, the page shows invalid Destination domain names.

    The Edge sends invalid Destination names like IP:port that are shown on the Orchestrator's UI due to a validation miss in the DNS hostnames.

  • Fixed Issue 123237: For a VMware SD-WAN Edge running Release 5.2.x and where the interfaces are configured with IPv6 only, the Diagnostics > Remote Diagnostics page does not load.

    When IPv6-only Edge interfaces are configured with IPv4 settings deactivated, an exception is thrown in a key script which causes the Diagnostics > Remote Diagnostics page to not load.

    The workaround is to activate IPv4 settings with a dummy configuration.

  • Fixed Issue 123267: When an VMware SD-WAN Edge 6x0 model (620, 640, or 680) is upgraded to a 5.x build, the Edge may go offline and need an additional reboot to recover.

    When this issue is experienced, Edge 6x0 interfaces GE1 - GE4 go offline after the upgrade and do not recover as expected. If the customer is only using GE1 - GE4 for LAN and WAN connections the Edge would be completely offline and the reboot would have to be done locally as a power cycle.

    The workaround on an Edge without a fix for this issue is to locate LAN, or at least one WAN connection to GE5, GE6, SFP1, or SFP2 as these interfaces remain up and the Edge would remain reachable for a reboot performed remotely through the Orchestrator.

  • Fixed Issue 123475: Connected Static Route (CSR) type flows that are matched to a Source + Destination LAN Side NAT rule may drop.

    Source + Destination LAN Side NAT rules may inappropriately apply only a destination NAT to the first packet for a flow, and observe a NAT collision on the return packet, for CSR → CSR flows.

    Note:

    Source NAT and Source + Destination NAT rules are not supported for CSR → CSR traffic.

  • Fixed Issue 123593: For a customer site using a High Availability topology where the customer is also using Edge Network Intelligence with Analytics turned on, in rare conditions the VMware SD-WAN HA Edge may not retrieve the Analytics configurations from the Edge Network Intelligence back-end.

    It is possible for both the Active and Standby Edges to acquire the token from the Edge Network Intelligence back-end. If the Standby Edge obtains the token after the Active Edge, the Active Edge's token will be stale, resulting in this scenario.

  • Fixed Issue 123594: For a customer enterprise site configured with a High Availability topology, when the VMware SD-WAN HA Edges are downgraded to a lower build, it may take up to an hour before the Standby Edge synchronizes with the Active Edge.

    The issue stems from the Edge's PIC watchdog timer not being rearmed during the Edge reboot process and having too short of a timeout which prevents it from triggering an additional Edge restart if the interface is stalled.

  • Fixed Issue 123956: A user cannot access the Local UI on a VMware SD-WAN Edge using a 5.2.x Release.

    The user's browser page does not load with a 404 Error. The issue is the result of exceptions thrown in both Remote Diagnostics and Local UI scripts.

  • Fixed Issue 124106: When LAN side NAT is configured for Many:1 translations where Port Address Translations (PAT) is used, traffic initiated from the opposite direction allows unexpected access to fixed addresses based on the outside mask and original IP address.

    LAN Side NAT rules allow connections to be initiated in both directions for Many:1 rules without a clear way to prevent traffic in the 1:many direction. 1:Many translations will now be blocked and users must create explicit 1:1 NAT rules to enable the traffic.

    This issue is fully covered in Important Notes: LAN-Side NAT Behavioral Change.

  • Fixed Issue 124162: When a user takes a packet capture on a VMware SD-WAN Edge interface, they may see a packet that appears to be corrupted.

    There is no actual packet corruption, the packet only appears corrupted in the PCAP file. This issue is due to a defect in the way the Edge writes packets to the packet capture interface, VLAN-tagged packets may be written incorrectly and will show up as a corrupted packet (invalid ether-type) in the PCAP file.

  • Fixed Issue 124180: A VMware SD-WAN Gateway may experience a Dataplane Service failure and restart to recover.

    The issue can be experienced during an improper deployment scenario involving the eth0/eth1 interfaces not being configured with an assigned IP address.

  • Fixed Issue 124263: A customer may observe high CPU utilization on a VMware SD-WAN Edge while handling IPv6 neighbor discovery (ND) and L2 encapsulation packets.

    Troubleshooting the Edge points to the api - vc_ip6_host_addr_to_network_addr process as consuming high amounts of CPU.

  • Fixed Issue 124784: A VMware SD-WAN Gateway deployed with a 4 core VM may experience a Dataplane Service failure, generate a core, and restart to recover when multiple customer enterprises using Edge clustering are connected to it.

    The Gateway can experience this issue if multiple clusters also configured with self healing routing are enabled. In this sceneario, for every set of statistics received from the Spoke Edges and Hub Cluster, the Gateway tries to iterate the DCE information which results in the CPU holding for more than 90 seconds, and this triggers the Gateway service failure.

  • Fixed Issue 125035: When a customer deploys a Fortinet 6.4.x VNF, the Edge's Management IP address is not accessible.

    Fortinet made changes in 6.4.x which results in FortiLink and transparent mode not working together. SD-WAN does not use FortiLink and as a result the resolution to this issue is to disable FortiLink before setting the transparent mode while deploying the VNF.

  • Fixed Issue 125377: When a user enables High Availability on for a site, the VMware SD-WAN HA Edge may experience a Dataplane Service failure and restart to recover.

    The issue is caused by a race condition on the HA state machine leads to the Edge service failure and possibly results in the HA Edge not coming up.

  • Fixed Issue 125421: A customer may observe that the WAN links on a VMware SD-WAN Edge are intermittently showing as down and then up on the Monitoring and Events page of the VMware SASE Orchestrator UI, with the potential the Edge may become unresponsive and fail to pass traffic until it is manually rebooted, or the Edge can experience a Dataplane Service failure and restart.

    This is an Edge memory leak issue that is encountered when the Edge Dataplane Service cannot open shared memory, causing stale PIs. This in turn causes open file descriptor exhaustion which will initially impact WAN links. However, if this issue is sufficiently advanced and results in Edge memory exhaustion the Edge can:

    1. Become unresponsive and unreachable through the Orchestrator, which requires an on-site reboot/power cycle.

    2. Can trigger an Edge service failure with a core file generated, with the Edge restarting to recover.

  • Fixed Issue 125467: For a VMware SD-WAN Gateway running a Release 5.1.0 or above version, if a Non SD-WAN Destination (NSD) tunnel is configured and if the NSD tunnel fails to form, this may trigger a Gateway Dataplane Service failure and cause the Gateway the restart.

    When a Gateway is trying to form an IPsec tunnel with an NSD, there is a potential timing scenario where the Gateway process can do a double deletion of the tunnel, and it is this that triggers an exception in the Gateway service and a resulting failure.

  • Fixed Issue 125487: Branch-to-Branch traffic flow may be disrupted by an ARP resolution issue.

    When encountering this issue, the Edge is forwarding the ARP request to the next hop IP address using the primary interface IP address instead of the subinterface IP address. The issue is triggered during flow creation when a non-connected route is used to reach the destination, and if the Edge's subinterface is used for that connectivity, the Edge does not properly fill the source IP address for the subinterface case.

  • Fixed Issue 125514: On a site deployed with a Standard High Availability topology, the Orchestrator UI may report that the Standby Edge state is "Unknown" if the HA interface is down.

    The WAN HA heartbeat is not accounted for with regards to peer reachability and as a result the Standby Edge state is moved to an unknown state even though the Standby Edge is reachable.

  • Fixed Issue 125647: For a site deployed with an Edge model 520 or 540, when an Edge is upgraded to a 5.2.0 version, client users connected to the Edge via a LAN port may experience a total loss of connectivity.

    Rebooting the Edge 520/540 does not recover the issue nor does downgrading the Edge to an older software version once it has been upgraded to a 5.2.0 version. When the Edge's console is deactivated in the Edge Security > Console Access settings on the Firewall configuration page of the Orchestrator (which is the default configuration for any Edge), the driver that manages the LAN1 through LAN8 ports of the Edge 520 or 540 does not configure itself properly, leading to those ports not being created at all.

    On an Edge without a fix for this issue, a customer can prevent this issue from occurring and/or restore connectivity on LAN ports on an affected Edge, by doing the following: navigate to Configure > Edge/Profile > Firewall > Edge Security, and under Console Access click on Enable and Save Changes.

  • Fixed Issue 126349: A customer may observe a VMware SD-WAN Edge with a CPU utilization rate of 90% or greater for a sustained period of time.

    The CPU utilization stays abnormally high above 90% where there are large number of flows with jitter that require the edge to correct. For Realtime flows, jitter correction is enabled on lossy links. When there are large number of flows with jitter correction enabled, the internal timers which are used for jitter buffering take significant amounts of CPU cycles. This could potentially cause performance degradation issues.

  • Fixed Issue 126458: For a customer site deployed with a High Availability topology where the HA Edges are Edge models 520/540, the customer may observe multiple HA failovers that are the result of an Active/Active state.

    The condition is triggered on HA configured 520/540 Edges, when the number of concurrent flows exceeds 300K.

    On Edge 520/540 HA Edges without a fix for this issue, the workaround is to increase the HA failover time from 700ms to 7000ms on the Configure > Edge > Device page as this will reduce the change of an Active/Active state.

  • Fixed Issue 126520: Users in an enterprise with a large number of active applications may observe that traffic matching Business Policy rules is not always steered properly.

    In environments with many active applications, the Edge DNS cache can become full, which would trigger an alert every 10 minutes that DNS entries have been missed. In addition, when the DNS cache is full, this issue could also impact first packet routing based on business policies.

    Clearing the Edge DNS cache will temporarily relieve the issue.

  • Fixed Issue 127127: A VMware SD-WAN Edge does not learn routes from Hub Edges when the VMware SD-WAN Gateways are upgraded to Release 5.1.x or higher.

    When an Edge is configured with Branch to Branch via Hubs with only the same Edge in the list and the Gateway is running 5.1.x and above versions, the routes are not advertised from the Gateway to the Edge.

    The only workaround is to remove the Edge from the Branch to Branch via Hubs configuration.

  • Fixed Issue 127254: In some scenarios a default-route is not originated by OSPFv3.

    Initially the redistribution of overlay routes into OSPFv3 is disabled and "default-information originate always" is configured at the profile level to advertise a IPv6 default-route. Also an IPv6 overlay default-route is received from a remote Edge. Now if the metric type (OE1/OE2) for the cmd default-information originate always" is changed on the Orchestrator, since the redistribution of overlay routes is disabled in OSPFv3 the LSA created for that cmd is also removed unexpectedly.

    If experiencing this issue where the fix is not yet installed, the workaround is to disable and then enable default-originate for OSPFv3 at the profile level.

  • Fixed Issue 127403: On the Test & Troubleshoot > Remote Diagnostics page of the Orchestrator UI, when running the remote diagnostic Troubleshoot OSPF - List OSPF Redistributed Routes or TroubleshootBGP - List BGP Redistributed Routes, the test returns an error with no data.

    After running either diagnostic the user observes the error: Error reading data for test on the Orchestrator UI.

  • Fixed Issue 127443: A VMware SD-WAN Virtual Edge deployed with a 2 core configuration may experience a Dataplane Service failure, generate a core with a SIGXCPU, and restart to recover.

    On a 2 core Virtual Edge platform, when there is a bulk of events processed (redistributed) in the BGP/OSPF worker thread, events are sent to the Edge routing process clients (BGP/OSPF) and these are running as a non-RR thread and may not get enough CPU cycles to read the data at the same rate. As a result the Edge remains in a permanent retry send loop and starves the other threads, triggering failure logged as a SIGXCPU exit.

  • Fixed Issue 127799: An Operator or Partner user attempting to debug a VMware SD-WAN Gateway with the vcdbgdump command may observe the command fails.

    This issue can be experienced on a Gateway that has AppArmor rule enforcement turned on.

  • Fixed Issue 128132: For a customer enterprise site deployed with a High Availability topology, if user tries to downgrade the HA Edge pair from 5.2.2 to 5.2.1 or lower, the Standby Edge fails to downgrade.

    When this issue is encountered, the site would have a mismatched set of Edges running different software versions. The issue is triggered by an exception in the downgrade path on the Standby Edge.

  • Fixed Issue 128377: For a customer enterprise using BGP for routing, the customer may observe that user traffic is disrupted due to a failure of BGP.

    The BGP process can fail during clean up due to an invalid memory access of self_mac_hash.

    As part of cleaning up the bgp_master, the self_mac_hash is already cleaned up. However, while processing the message after the clean up has already occurred, BGP accesses the deleted hash causing the failure.

  • Fixed Issue 128741: A VMware SD-WAN Gateway may experience a Dataplane Service failure, generate a core file, and restart to recover.

    When a valid management packet arrives unexpectedly on a Non SD-WAN Destination IPsec tunnel or a GENEVE interface, the Gateway may run into an error while processing that packet and this triggers the Gateway service process to fail.

  • Fixed Issue 129359: For a customer enterprise site deployed with a High Availability topology, in rare instances the customer may observe that both the Active and Standby Edges are sending out heartbeat packets on their WAN links with the same MAC address.

    When encountered, this issue impacts the upstream switch which may send alerts related to two different flows with the same MAC address coming into the switch. The issue can result from a timing condition that results in the Standby Edge's MAC file not being created and then the Standby Edge tries to recover from this by mistakenly updating its interface MAC addresses to match the Active Edges.

    If encountering this issue, a user must deactivate the Standby Edge to default factory settings and then reactivate it in the HA setup to restore the MAC file.

  • Fixed Issue 129923: Client users may observe traffic dropping for a flow which uses a recursive static route for a destination prefix.

    Initial traffic flow uses a static route for a destination prefix after a recursive resolution. However, later traffic using the outgoing interface of that recursive route goes down. Then even if a recursive route resolution is possible by some other route using the outgoing interface, the recursive resolution is not updated. This leads to traffic drop though the path is available.

    On an Edge without a fix for this issue, remove and re-add the static route used by the flow.

  • Fixed Issue 130011: A Operator or Partner user attempting to debug a VMware SD-WAN Gateway using tcpdump.sh with find the command works when outputted to a file but files for stdout.

    This is the result of Apparmor blocking access to /dev/pts/* terminals in enforce mode. This caused the vctcpdump process to terminate and tcpdump.sh was not capturing any information on stdout.

    On a Gateway without a fix for this issue, the user can user run tcpdump.sh to a file instead of stdout, or set the /etc/apparmor.d/opt.vc.bin.vctcpdump profile to complain mode.

  • Fixed Issue 130284: For a customer enterprise site deployed with a High Availability topology, a user may observe that memory utilization increases over time and does not come down.

    This is a memory leak and when enough memory is accumulated, it can trigger a memory allocation failure in the Edge's Dataplane Service which causes a defensive restart/failover to recover the memory. This issue can happen due to duplicate address detection (DAD) being triggered in quick succession when it is already in progress or due to an HA synchronization.

    On HA Edges without a fix for this issue, if the user observes an increasingly high level of memory utilization on the Edge's Monitor > System page, they can initiate an Edge Service Restart in a maintenance window to temporarily recover the Edge's memory.

  • Fixed Issue 130368: Direct traffic that matches a Business Policy rule where "Available" link steering with "Transport Group" is configured does not work as expected if the desired link is in an Unstable state.

    WAN link's are marked as Unstable usually for high loss, but any factor that trigger an Unstable state is equally valid for this issue. The issue is the result of the Edge's link selection code for "Available" with "transport group" steering traffic to a better quality link even though the "Available" option only requires that the link be up, not that it also be of good quality.

  • Fixed Issue 130573: For customer sites using VMware SD-WAN Edge models 520/540 or 610/610-LTE, VLAN-configured switched ports may not pass traffic to each other.

    All of the listed Edge models use a switched port board from the same manufacturer. If a user configures a switched port with one or more VLANs on the Edge, the ports cannot send ARP packets and this results in a failure to pass traffic amongst these ports.

    The fix for issue restores correct VLAN switching for the Edge model 610/610-LTE.

    The Edge model 520/540 use two switched port boards, one for LAN1-LAN4 and one for LAN5-LAN8. For these Edge models, the fix restores VLAN switching among ports of the same switched port group. However, VLAN switching between switched ports of 2 different switched port groups still does not work properly. For example, VLAN switching between LAN2 and LAN6 does not work as expected, while switching between LAN2 and LAN4, or LAN6 and LAN8 does work as expected.

  • Fixed Issue 130833: For a customer enterprise where a Non SD-WAN Destination via Gateway with a route based type is configured, client users may find traffic destined for the peer site is dropped during the IKE SA re-key.

    In some cases when Phase1 rekey happens, VCG doesn’t reply on new created Phase1 Dead Peer Detection (DPD) message. This can cause the tunnel to break and BGP tunnel flapping. If the peer (VMware Cloud on AWS) is configured with on-demand DPD, after a Phase1 SA re-key it sends aggressive DPD requests on the newly created Phase1 SA and the Gateway does not reply to these requests. This causes newly created Phase1 SA deletion on the peer side and the initiation of a new Phase1 SA. This will also fail since the Gateway is always the initiator.

    The workaround is to manually delete the Phase1 SA, or wait until new a Phase1 is initiated from the Gateway.

  • Fixed Issue 130901: User traffic may drop for flows which try to switch from "backhaul via CSS" to "Direct".

    The Edge does not allow switching the traffic path if the flow matches a different business policy rule which steers it to via some other path/route. This handling was missing for the flows which start with "backhaul via CSS".

    On an Edge without a fix for this issue, the user can configure the business policy to steer traffic with a destination IP address.

  • Fixed Issue 130907: SNMP walk on ipAddrTable does not fetch the entire table.

    An SNMP walk on the ipAddrTable only fetches the IP Address, the rest of the fields are not fetched due to a corruption of the data of physical LAN interfaces.

  • Fixed Issue 130931: For a customer enterprise site deployed with a High Availability topology, an IPv6 address can be removed on an IPv6 enabled interface with DHCP addressing and is not restored until a Routing Advertisement (RA) is received on the Edge.

    If the RA packet is not received on the Edge before the lifetime is expired, the IPv6 address is deleted.

  • Fixed Issue 130998: In rare instances, a VMware SD-WAN Edge may experience a Dataplane Service failure and restart to recover.

    The Edge Service can fail in the relatively rare situation that it is processing and classifying packets that match the Surfshark VPN application.

  • Fixed Issue 131029: For a customer enterprise site that uses one or more PPPoE links, when the VMware SD-WAN Edge is either deployed or upgraded to Release 5.2.0.x, the PPPoE links may fail to come back up.

    Post-upgrade the Edge process that handles PPPoE interfaces fails because a key configuration file is missing and this results in the failure of any PPPoE links on the Edge.

    On an Edge without a fix for this issue, it the Edge encounters this issue, there is no workaround beyond:

    1. Upgrading the Edge to any 5.2.2.x build. This version includes a fix for this issue and resolves the condition and prevents it from recurring. It is strongly recommended to use this method.

    2. If a customer wants to continue using a 5.2.0.x build, they can contact VMware SD-WAN Support who can correct the issue for the customer on a per-Edge basis.

  • Fixed Issue 131085: A VMware SD-WAN Gateway may experience a Dataplane Service failure, generate a core, and restart to recover.

    In cases particularly involving the Gateway sending out locally generated packets (in exact terms, malloc'd skb) of its interface, the Gateway service tries to access the mbuf member pointer inside of skb (which happens to be null) and this triggers an exception and a service failure. The fix ensures that access to mbuf happens only after mbuf is updated correctly by converting malloc'd skb to dpdk skb.

  • Fixed Issue 131302: SD-WAN tunnels over MPLS backhaul will not come up when multihop External BGP is configured on the Partner Gateway handoff section.

    When multi-hop EBGP is configured on the Partner Gateway, the SD-WAN tunnels response to the tunnel initiation request fails while trying to resolve the ARP. This is because ARP is initiated incorrectly with the BGP peer IP address instead of the next hop IP address. Since, multihop BGP is configured, the peer IP address is not the immediate next hop and ARP fails. As a result, SD-WAN tunnel packets to the peer via the MPLS tunnel never go out of the Gateway.

    The workaround is to use a single hop External BGP peer, if possible.

  • Fixed Issue 131891: For a customer enterprise site where a VMware SD-WAN Edge is deployed either as a standalone Edge or with a High Availability topology, the Edge may experience a Dataplane Service failure and restart to recover, which would result in an HA failover with HA sites.

    The issue can be encountered when the Edge (Active Edge in HA deployments) is handling a large traffic load that keeps the Deep Packet Inspection (DPI) engine busy handling HTTPs traffic featuring Server Name Identifier name strings greater than 32 bytes.

  • Fixed Issue 137279: For a customer enterprise site deployed with a High Availability topology where the HA Edges are using a 5.2.0.x software version, either upgrading the HA Edge pair to any other Edge software version, or a standard fail-over of the Active Edge to the Standby Edge may result in the site going offline on the VMware SASE Orchestrator, though customer traffic would continue to pass.

    The issue is the result of the Standby Edge's certificate not being renewed and ultimately expiring. The Active Edge's certificate is properly updated so the issue only manifests upon an HA fail-over, when the promoted Standby Edge with the expired certificate causes the HA pair to go offline on the Orchestrator. This means the site cannot be reached, monitored, or managed through the Orchestrator.

    When a customer has Certificate Enabled for their enterprise, the Active HA Edge includes the Standby Edge's certificate digest as part of every heartbeat it sends to the Orchestrator. The Orchestrator uses this certificate digest to renew the Standby Edge's certificate. The cause of the issue is a defect in this certificate digest generation process which results in a certificate digest that consists of an empty string and this results in the Standby Edge's certificate never being renewed unless a manual renewal is done from the Orchestrator prior to an HA fail-over or before the Standby Edge certificate expires.

    On an HA site where the Edge pair use a version without a fix for this issue, should a customer HA site experience this issue, the only way to restore the site is to get physical access to the HA Edge pair and reboot the current Active Edge (the former Standby Edge) to trigger an HA fail-over. This can be done either through the Local UI (if so enabled) or by power cycling the Active Edge.

    Once back online with the Orchestrator, a manual renewal can be performed to renew the certificates both HA Edges.

    Caution:

    A manual certificate renewal results in tunnel flaps since the Active Edge's certificate is also renewed.

Orchestrator Resolved Issues

Resolved in Orchestrator Version R5220-20231214-GA

Orchestrator version R5220-20231214-GA was released on 12-14-2023 and resolves the following issues since Orchestrator version R5204-20230831-GA and UI Build R5204-20231201-GA. This means that a fix for an Orchestrator issue listed in the 5.2.0 Release Notes up to these listed builds is included in all Release 5.2.2 builds.

  • Fixed Issue 48230: When on the Monitor > Edges page, searching for Edges by using "Model" is not fully accurate.

    When a user filters by Model number, the Orchestrator returns extra Edges.

  • Fixed Issue 48609: On the Monitor > Routing page, a user cannot sort the segment name in a multicast route table.

    In Release 5.4.0, the API is enhanced to accept the segment name as a sorting parameter and helps in sorting the segment name.

  • Fixed Issue 82095: User can configure invalid device settings for Edge VLANs that will result in significant connectivity issues for the Edge.

    The Orchestrator is not attempting to validate device configurations. In particular, a VLAN configuration for a switched port with an empty table. Some configurations can be so full of errors that the Edge's management process will fail.

    On an Orchestrator without a fix for this issue, the customer should review all VLAN Device settings and ensure they are valid as the Orchestrator is not checking.

  • Fixed Issue 92766: On the Monitor > Routing page, pagination information is incorrect.

    If a user clicks the Next button, the UI page does not stop even after reaching the last page due to the filter logic not being applied correctly with the result that the page number shown is not correct.

  • Fixed Issue 102121: When an Edge's Secure Access configuration is updated multiple times without any updates to the Edge's Firewall configuration while using the Orchestrator UI, the Secure Access configuration updates may stop being sent to the Edges.

    The issue has been encountered most often in testing where Engineering deliberately forces a large number of Secure Access updates without any Firewall updates. However it can, in rare instances, be triggered by a customer in the field.

    If experiencing this issue on an Orchestrator without a fix, a user can manually update the Edge's Firewall configuration from the UI just once. After the manual Firewall update, the user can redo the Edge Secure Access configuration change from the UI and the Orchestrator would push the Edge Secure Access configuration change from the UI to the Edges.

  • Fixed Issue 104843: When on the Configure > Edges page, a user cannot locate all the search options.

    This includes searching by Profile, Operator Profile, and Analytics.

  • Fixed Issue 108285: When configuring a profile on the Orchestrator UI, if a user changes an Edge interface from switched to routed, the Orchestrator appends the new routed interface to the end of the routed instead of inserting it at the appropriate index.

    The wrong order of Routed Interfaces in a Profile Configuration causes validation problems when a user adds a static route using those references for that routed interface.

  • Fixed Issue 113466: On the Configure > Network Services page of the UI, is a user selects a Cisco ASA for the Non SD-WAN Destination via Gateway, the UI returns "o.fragmentationAvoidance is undefined" error.

    The Orchestrator UI should display a template for a sample IKE/IPsec configuration but instead throws the above error.

  • Fixed Issue 114444: When a VMware SASE Orchestrator is upgraded to version 5.1.x or higher, a user cannot save their changes of a configuration for a Non SD-WAN Destination via Gateway where redundant tunnels are already configured.

    User will not be able to save the configuration on the NSD via Gateway page and could cause packet drops on Gateways for redundant NSDs.

    The workaround for this issue is to update the max hop to 2 in the NSD via gateway UI for the redundant BGP neighbors and save the changes.

  • Fixed Issue 115441: On the Global Settings > Customer Configuration page, when configuring the SD-WAN tile, software and firmware images are only visible when they are activated.

    Software and firmware images should always be visible to the user regardless of their status.

  • Fixed Issue 115695: On the Monitor > Edges page, a customer cannot search for an Edge by Description in the Edge List table.

    Certain words in an Edge's Description (for example, Activated or the Edge's location) are useful for searching for all Edges with a similar description.

  • Fixed Issue 115981: For a customer enterprise using VMware SASE Orchestrator's APIv2, when running the API for getting Enterprise Events, the Orchestrator returns only a limited set.

    The specific call is https://\{api_host}//api/sdwan/v2/enterprises/{enterpriseLogicalId}/events. When invoked it returns only the top level hierarchy and does include details like the enterpriseName, EdgeName, segmentName, or edgeID. In addition, APIv2 does not support graph traversal.

    On an Orchestrator without a fix for this issue the only workaround is to use APIv1, which forces a customer to maintain two sets of API families. In addition APIv1 does not support Cloud Web Security.

  • Fixed Issue 116524: For customers who subscribe to Edge Network Intelligence and have Analytics activated, on the Monitor page when looking at the left side list of links, a user observes two links for Analytics that have different names but are exactly the same and redundant.

    There are two links for Edge Network Intelligence: Application Analytics and Branch Analytics, and both go to the same place in ENI. This is corrected with a single link on the left side called Edge Network Intelligence.

  • Fixed Issue 116531: If a user attempts to generate a report on the VMware SASE Orchestrator where at least one Edge description or application includes a comma (,) the report may not format properly.

    When an Edge description or application name includes a comma (as shown in the screenshot below) the Orchestrator's report service confuses this and breaks out the text after each comma into the report's next column, versus containing the entire text string in the Edge Description column as expected.

    So instead of Bytes Transmitted having the values associated with it, it shows the text after the first comma, Bytes Received would have the text after the second comma (if there was one), and so forth. The report would still include the data for Bytes Transmitted and Bytes Received, it would just be pushed farther to the right and not be aligned to the correct columns.

    On an Orchestrator without a fix for this issue, the user would need to ensure the Edge description or application used no commas.

  • Fixed Issue 116633: When logging into the VMware SASE Orchestrator using either Safari or Firefox, if a user configures an invalid VLAN at the profile level (for example, ""), and then configures a valid VLAN value at the VMware SD-WAN Edge, the call still goes through but shows an error.

    The interface value where no value is set should be null but is instead vlanID = "". If a user logs into the Orchestrator with a Chromium based browser, this issue is not observed.

  • Fixed Issue 116790: When a VMware SASE Orchestrator is upgraded to Release 5.1.x or later, Customer VMware SD-WAN Edges may be inadvertently downgraded to an older Edge version than the Edge is configured to use.

     The issue is triggered when a customer enterprise is deleted from the Orchestrator where the deleted enterprise was associated with an Operator Profile by its logical ID in the Orchestrator database. When the enterprise is deleted, the Operator Profile is also deleted. Customers who have Edge Image Management configured and multiple Operator Profiles available and who had this deleted Operator Profile assigned as their default are then assigned an Operator Profile that remains available in their Edge Image Management menu. As a result the customer could be assigned an Operator Profile containing a much older Edge Release, and the Edges assigned this Operator Profile can have their software changed and possibly downgraded with potentially disruptive effects including network failure if the Edge is now running an older release that does not support features the customer is using.

  • Fixed Issue 117138: When using Zscaler Automation to create a Zscaler Cloud Security Service (CSS), Edge to Zscaler CSS IPsec tunnels may not be created.

    The Orchestrator ensures enqueuing only one action per Edge. However, if one action is stuck at the NOTIFIED state, all subsequent actions are not executed and the IPsec tunnels do not get built.

    On an Orchestrator without a fix for this issue, an Operator user needs to manually delete the stale Edge action from the Orchestrator database.

  • Fixed Issue 117573: On the Configure > Device > VPN > Cloud VPN page, when a user tries to configure a Branch to Hub site, Hubs cannot be filtered using the filtering icon.

    Where a customer enterprise has multiple Hubs configured, the lack of filtering makes is difficult to configure a Branch to Hub VPN.

  • Fixed Issue 117614: The Configure > Profile page of the Orchestrator UI is missing several actions in the Shortcuts box.

    The missing shortcuts include: Migrate Profile; Duplicate Profile; Modify Profile; and Delete Profile. These actions can be found on the Profile's list page, but the customer should have an easier access to these actions.

  • Fixed Issue 117988: The "Inbound Route Learning" with the "Exact Match" checkbox configured for OSPF under a VMware SD-WAN Interface does not match what is configured on the Edge when comparing the values on the Classic UI and the New UI of the VMware SASE Orchestrator.

    Exact Match option does not display the correct value even if though it is correctly stored in the Edge's database when looking at the New UI.An

  • Fixed Issue 117993: When a Partner User managing customer enterprises which use native authentication (in other words, username/password) or an Enterprise User attempts to reset a password for an Enterprise User, the attempt fails.

    The user would observe the error: user does not have privileges required to access [enterpriseUser/sendEnterpriseUserPasswordResetEMail]. This issue is only experienced on the New UI which is the default UI for 5.3.0 and is the result of missing request parameters.

  • Fixed Issue 118074: A user may not be able to open some device settings on the Configure > Edge > Device page of the VMware SASE Orchestrator's New UI.

    Settings that may not be accessible include Interfaces, IPv6, Cloud VPN, Non SD-WAN Destination (NSD), and Cloud Security Service (CSS). The issue is traced to the WAN Settings requiring a Public IP Address and if this address is absent, an error is thrown on the New UI and blocks access to those settings.

  • Fixed Issue 118297: On the Configure > Edge page, several sorting capabilities are missing from the Orchestrator UI.

    These include sorting Edges by Profile, Operator Profile, Analytics, Logical ID, HA Edge, and License.

  • Fixed Issue 118302: On the Monitor > Edge page, a user cannot filter Edges by Link Status on the Orchestrator UI.

    Customers with large enterprises need to be able to filter Edges that have down links to see where to direct their troubleshooting resources.

  • Fixed Issue 118501: On the Monitor > Edges page, if a user clicks on CSV to download their list of all SD-WAN Edges, the Orchestrator will only download a maximum of 2048 Edges

    This impacts customers with greater than 2048 Edges in their enterprise who need to download a complete list of all Edges.

  • Fixed Issue 118757: When a user is on the Provision an Edge screen of the Orchestrator UI, selecting an Edge License may be difficult because of the use of a dropdown menu where the user cannot filter between the many possible license options.

    When the dropdown menu for the Edge License field contains a substantial number of license options, users encounter challenges efficiently selecting their desired option because the list does not allow users to filter items. Beginning with Orchestrator Release 5.3.0, the dropdown menu is replaced with a complete list of all licenses to ensure the user can efficiently select the correct one.

  • Fixed Issue 118761: Users cannot filter items while updating the Assign Software Images on the Edge listing page.

    Assign Software Image dropdowns do not allow users to filter items, making the selection process more difficult when there are many items in the dropdown.

  • Fixed Issue 118770: When a user is on the Configure > Edges screen, there is no option to Assign Inventory to Edge.

    This option is not present on the New UI and when restored should not use a drop down menu where there can be 100's or 1000's of possible Edge options. Instead the UI should use a list view where Edges can be filtered.

  • Fixed Issue 119938: For a customer who uses automation for Zscalar tunnels, it may take a long time to create an automated IPsec tunnel from a VMware SD-WAN Edge to Zscaler.

    When customers configure Zscaler sub-locations at Edge > Device Settings, it can take a long time for these configurations to sync with the Zscaler cloud. This is because the Zscaler cloud needs to update its records for each sub-location, which can be a time-consuming process.

    This issue is caused by the automation framework on the Orchestrator, which enqueues IPsec tunnel creation and sub-location creation actions in the automation queue. It also enqueues update actions in the automation queue when the Edge WAN IP changes. However, the wait time for items in the queue increases due to the large number of update actions. In some customer deployment environments, the Edge WAN IP can change up to 4000 times in one day (for example, mobile WAN links).

  • Fixed Issue 120398: A Partner User cannot create a new configuration profile for a customer enterprise they are managing.

    When the Partner User tries to create a configuration profile for their customer, the Orchestrator throws an error that reads 'invalid proxy enterprise context for operator profile'. This issue is encountered for any Partner role with configuration privileges.

  • Fixed Issue 121085: If a Partner Administrator is on the Global Settings > User Management > Service Permissions page, the option "Reset to System Default" does not work as expected.

    Reset to System Default is selected is for all custom role permission packages to revert to an Unpublished state as the default settings for user role permissions are restored. However, the Partner user would observe that one or more custom packages remain in a Published state and those that remain Published cannot be modified or deleted.The expected behavior when

  • Fixed Issue 121118: An Enterprise User does not have the option to generate a Diagnostic Bundle or generate and download a Packet Capture on the VMware SASE Orchestrator.

    This is true of all Enterprise User roles (including Superuser). The expectation is that an enterprise user under SD-WAN > Diagnostics should see the option to:

    1. Generate (but not download) a diagnostic bundle.

    2. Generate and download a packet capture.

    However, all they see are options for Remote Diagnostics and Remote Actions.

    If experiencing this issue on an Orchestrator without a fix for this issue, contact SD-WAN Support for assistance and they can generate the diagnostic bundle for you.

  • Fixed Issue 121336: When an Edge or Profile has "Partner Gateway Assignment" configured on the Configure > Device page and changes are made via an API call, the "Profile Update" Event always shows the Gateways as deleted.

    This is purely cosmetic and has no functional effect. The issue occurs because the Gateways field is deprecated in the configuration.

  • Fixed Issue 121993: A user may not have the option to edit the VLAN properties for a VMware SD-WAN Edge on the VMware SASE Orchestrator UI.

    The issue does not affect all VLANs in use by an Edge but when the issue is encountered the user would click on a VLAN in the UI and the result is nothing happens.

    If a customer is experiencing this issue on an Orchestrator without a fix for this issue, contact SD-WAN Support for assistance.

  • Fixed Issue 121995: A customer receives HA Failover alerts even though they turned those alerts off on the Configure > Alerts page of the Orchestrator.

    When enterprise alerts are enabled but HA Failover alerts are not enabled, the HA Failover alert is being sent to enterprise users.

  • Fixed Issue 122193: On the Configure > Edges > Firewall page, when configuring a Firewall rule for an Edge, for Firewall Action a user can only select 'Allow'.

    The user would click on the Firewall Action and the dropdown menu appears for a split second and then goes away immediately, and the user cannot change from the default Allow to any other option.

  • Fixed Issue 122347: The Service Permissions feature on the VMware SASE Orchestrator's New UI is not operating as designed. The privileges that were removed as part of Service Permissions for enterprise users were not functioning as expected. Additionally, when a user attempts to create a new service, privileges that are not associated with the module are visible.

    The privileges that were removed as part of Service Permissions for enterprise users were not functioning as expected. For example, Remote Diagnostics > Flush Flows still functions even if the privileges are removed from an enterprise user.

    The second issue is that when a user attempts to create a new service, privileges that do not belong to the module are visible in the UI. For example, when selecting SD-WAN > Global Settings, the user receives almost the same options.

    For the second issue, the user can manually select the permissions they need from the UI.

  • Fixed Issue 122519: On the Configure > Edge/Profile > Device > Connectivity page, a user may not be able to edit a VLAN.

    The VLAN editor does not open because the segment name is null in the configuration. This is the result of allowing the segment name to be null in the API to avoid backward compatibility issues. However, the New UI expects this segment name and as a result throws an error. The fix resolves the segment name issue by fetching the actual segment name from segment data.

  • Fixed Issue 123867: The link metrics and series APIs return an error.

    When the link metrics or series API is called without a list of metrics, the API mistakenly returns an error instead of adding all applicable metrics to the response.

    On an Orchestrator without a fix for this issue, a user must submit an API request with a list of metric fields to return. 

  • Fixed Issue 124073: If a user configures a Non SD-WAN Destination via Gateway using redundant Gateway tunnels with AES-256 encryption, the standby redundant Gateway tunnel continues to use AES-128 encryption.

    A user would go to Configure > Network Services on the Orchestrator UI and change the encryption algorithm to AES-256 for an NSD with redundant tunnels. Based on the API response the user would observe that the redundant tunnel continues to use AES-128 and this is the result of a defect with the API which handles the tunnel encryption change.

  • Fixed Issue 124092: On the Overlay Flow Control screen, the user cannot see the Segment name.

    When trying to filter for the Segment name, the Segment name does not be populated as part of the results.

  • Fixed Issue 124129: If the System Property that controls the availability of the Enhanced Firewall Service (EFS) is set to True, any new customer enterprise added to the VMware SASE Orchestrator has EFS activated by default.

    The System Property enterprise.capability.enableATP is set to True by default on previous 5.2.0 builds. When this property is set to True, a user would observe that when checking the Global Customer settings for a new customer enterprise that EFS is enabled by default. The expected behavior is for every new customer enterprise to have EFS not activated by default, where they would need to have this feature activated as an explicit action.

    This is corrected by having the enterprise.capability.enableATP property set to False by default.

  • Fixed Issue 124568: For a VMware SASE Orchestrator deployed with a Disaster Recovery (DR) topology, in rare instances the Operator User may observe that DR halts, leading to a temporary lack of replication between the active and standby Orchestrators.

    The user experience is not affected as this is isolated to DR only. The DR page will indicate an error state instead of STANDBY_RUNNING. This is an intermittent error and will auto-recover.

  • Fixed Issue 125082: If a user configures a VMware SD-WAN Edge with an overridden DNS Server IP address on a VLAN, and then changes an interface setting for the Profile that Edge is using, the DNS Server IP address is no longer present for the Edge VLAN.

    The UI does not send the override flag inside of the DHCP section and this causes any Profile changes to trigger an override of the DHCP section.

  • Fixed Issue 125456: For a customer enterprise where VNF's are used, if a user attempts to modify an Edge Device configuration, the VMware SASE Orchestrator rejects the changes when trying to save them.

    The configuration change can be as minor as adding a comment to an existing static route and the Orchestrator UI does not save the change where a VNF is activated. The user may observe a "Script injection error" banner at the bottom of the UI screen.

    The workaround on an Orchestrator without a fix for this issue is to temporarily deactivate the VNF's and then save the configuration changes. Once these are successfully saved, the user can then reactivate the VNF's.

  • Fixed Issue 127281: Under the Configure > Overlay Flow Control page of the Orchestrator UI, static configurations on routed subinterfaces do not show up on the OFC page as connected routes.

    When a static configuration is done on a routed non-wan interface or subinterface, they are expected to show up on the OFC page as connected routes. But this is not working for subinterfaces using IPv6 configurations where IPv6 is not also activated on the parent interface of the subinterface.

  • Fixed Issue 127727: When creating a new Cloud Security Service (CSS), a user cannot configure the Domestic Preference option.

    If a user activates the Domestic Preference check box and saves the configuration, the Orchestrator verifies the credentials and displays a message stating that "Changes saved successfully!". But post-save, when the CSS profile is opened again, the user would observe that the Domestic Preference check box is not selected.

  • Fixed Issue 128310: VMware SASE Orchestrator users may experience overall slowness and some API failures due to issues with the Orchestrator's database service. Other side effects include SD-WAN Gateways/Edges appearing offline in the UI, configuration changes made through the Orchestrator UI not being pushed to the target SD-WAN Edges, and a loss of reporting capabilities.

    The issues are all caused by the Orchestrator's database service failing with the error: too many open files. This error can be observed by an Operator user on the VMware SASE Orchestrator via logs. An enterprise or partner user accessing VMware SASE Orchestrator via the UI would experience slowness and intermittent API failures, causing error messages on the UI.

  • Fixed Issue 128652: For a VMware SASE Orchestrator deployed with a Disaster Recovery (DR) topology, when the Orchestrator is upgraded, DR immediately fails.

    The Standby Orchestrator replication configuration is missing multiple databases critical to synchronizing with the Active Orchestrator and so DR fails after the upgrade.

  • Fixed Issue 128753: When a customer configures a subinterface that uses DHCP for addressing and creates a user-defined WAN overlay, the user is unable to save the configuration without configuring the Source and Next-Hop IP addresses.

    This blocks a customer from adding another WAN overlay unless they also configure the Source and Next-Hop IP address, which should not be required.

  • Fixed Issue 129232: A user cannot perform a password self-reset on the VMware SASE Orchestrator.

    When the user tries to access the self-password reset link directly in the browser, they are redirected to the login page instead of the password reset page.

  • Fixed Issue 129412: On the Configure > Edge > Device > Connectivity > VLAN page of the UI, on the IPv4 DHCP Server section, the Number of Addresses displayed may be inaccurate.

    The behavior is seen if a user edits the original DHCP value later to increase the number of addresses and the UI continues to display the older, lower number of addresses.

  • Fixed Issue 129494: On the Customer > Global Settings > Customer Configuration > Service Configuration > SD-WAN page, when a user is editing the service configuration, the user is required to add the domain name every time, even if Single Sign-On (SSO) authentication or Edge Network Intelligence (ENI) is not configured for the customer.

    The user is required to do this even if Single Sign-On (SSO) authentication or Edge Network Intelligence (ENI) is not configured for the customer. A domain name is not mandatory if a customer is not using ENI or SSO.

  • Fixed Issue 129522: On the Global Settings > Customer Configuration > Configure BGP and BFD page, addresses with 31 bit prefixes cannot be used as a Local IP value for the Handoff Interface.

    The issue is caused by the Orchestrator not supporting RFC 3021 networks (/31 networks).

  • Fixed Issue 129584: On the Configure > Edges > Business Policy page, when a user edits an existing Business Policy rule, the Orchestrator UI does not update the reconfigured value for the Destination field even after saving the changes.

    For example, for an existing Business Policy rule with Destination set as “IP Address”, if the user changes the value of Destination to “Any” and saves change, the changes made to the Destination field for the rule is not reflected in the UI. The user would still see the Destination field set to “IP Address” instead of “Any” in the rule.

  • Fixed Issue 129662: A user cannot tell if a VMware SD-WAN interface is enabled or disabled when looking at the Configure > Edge > Device > Interfaces page of the Orchestrator UI.

    The VMware SASE Orchestrator UI does not have color differentiation between enabled and disabled interfaces, preventing customers from identifying a sub-interface that is deactivated.

  • Fixed Issue 129926: When a user provisions an Edge with no serial number, the Edge activation fails.

    The user should be able to provision an Edge without a serial number and simply allow the Orchestrator to autodetect it.

  • Fixed Issue 129958: On the Configure > Edge > Device > Connectivity > VLAN page of the UI, on the IPv4 DHCP Server section, the user cannot change the Number of Addresses unless they first override the general VLAN.

    Previously, a user could do an override a subsection of an Edge VLAN like DHCP or OSPF while accepting the remainder of the configuration from the profile. In this instance the user must override the entire VLAN so that they can edit the DHCP portion of it.

  • Fixed Issue 130153: For Enterprise users with a Support role, the Restart Service option is not available under Monitor > Edges > Select Edge > Shortcuts > Remote Actions menu.

    This prevents Support role users from performing an Edge service restart from the Shortcuts menu on the Monitor > Edges page.

  • Fixed Issue 130732: Under the SD-WAN > Overlay Flow Control on the Routes List, when the Subnet "contains address" filter is applied, there may be an error thrown.

    The error reads: "Error occurred while applying filters". This error frequently occurs when the "contains address" filter is applied and no other filter is selected.

    The workaround is to also select the "segmentId" filter along with "subnet contains address" filter.

  • Fixed Issue 131299: On the Configure > Overlay Flow Control page of the UI, when the user changes the segment for a Non SD-WAN Destination via Gateway which uses static routes, the OFC table does not display the updated static routes.

    Instead the OFC table shows the results using the previously selected segment which will mislead the user into thinking their configuration change was not applied when it actually was.

  • Fixed Issue 131544: When looking at the Monitor > Edge > Links for an Edge using an LTE modem, the user may observe gaps for this link type.

    When an LTE modem goes down, the LTE link statistics signal strength field generates an "error: couldn't find modem" error and the Edge raises an exception as this string is converted for transmission to the Orchestrator. The Edge management service tries to send this to the Orchestrator but the Orchestrator cannot interpret this string and also throws an error. The result is that the Orchestrator does not show link statistics for this period of time.

  • Fixed Issue 131789: When configuring Single Sign On (SSO) for an organization, even though the role information is present in the Identity Provider (IdP) response, the users cannot login to the Orchestrator.

    The Orchestrator cannot match the role of a user logging in via Single Sign On (SSO), if the IdP sends role information in a nested JSON structure. Starting in version 5.2.2.0, the Orchestrator can reference and match the role of a SSO user even if it is present in a nested JSON structure.

    If experiencing this issue on an Orchestrator without a fix for this issue, the workaround is to configure the IdP to send the role detail in the immediate level, and not in the nested structure.

  • Fixed Issue 133201: On the Global Settings > User Management page of the Orchestrator UI, If a user is creating or editing a custom role and attempts to change the 'Delete' parameter for PCAP Bundle privileges, the UI also changes all other parameters for PCAP Bundle.

    For example, if a user elects to disable PCAP Bundle Deletion, the UI also disables the Read, Update, and Create parameters as well, even if the user wanted those parameters to remain enabled.

  • Fixed Issue 133642: On the Configure > Edge > Device > Connectivity > VLAN page of the UI, the Edge's VLAN is automatically set to Override by default, and a user cannot deactivate this setting.

    The expected behavior is that if a user creates a VLAN at the Profile level, that the Edge Override setting is deactivated by default and a user must elect to activate Edge Override on a per Edge basis.

Known Issues

Open Issues in Release 5.2.2.

Edge/Gateway Known Issues

  • Issue 14655:

    Plugging or unplugging an SFP adapter may cause the device to stop responding on the Edge 540, Edge 840, and Edge 1000 and require a physical reboot.

    Workaround: The Edge must be physically rebooted. This may be done either on the Orchestrator using Remote Actions > Reboot Edge, or by power-cycling the Edge.

  • Issue 25504:

    Static route costs greater than 255 may result in unpredictable route ordering.

    Workaround: Use a route cost between 0 and 255.

  • Issue 25595:

    A restart may be required for changes to static SLA on a WAN overlay to work properly.

    Workaround: Restart Edge after adding and removing Static SLA from WAN overlay.

  • Issue 25742:

    Underlay accounted traffic is capped at a maximum of the capacity towards the VMware SD-WAN Gateway, even if that is less than the capacity of a private WAN link which is not connected to the Gateway.

  • Issue 25758:

    USB WAN links may not update properly when switched from one USB port to another until the VMware SD-WAN Edge is rebooted.

    Workaround: Reboot the Edge after moving USB WAN links from one port to another.

  • Issue 25885:

    A large configuration update on the Partner Gateway (e.g. 200 BGP-configured VRFs) may cause latency to increase for approximately 2-3 seconds for some traffic via the VMware SD-WAN Gateway.

    Workaround: No workaround available.

  • Issue 25921:

    VMware SD-WAN Hub High Availability failover takes longer than expected (up to 15 seconds) when there are three thousand branch Edges connected to the Hub.

    Workaround: No workaround available.

  • Issue 25997:

    The VMware SD-WAN Edge may require a reboot to properly pass traffic on a routed interface that has been converted to a switched port.

    Workaround: Reboot the Edge after making the configuration change.

  • Issue 26421:

    The primary Partner Gateway for any branch site must also be assigned to a VMware SD-WAN Hub cluster for tunnels to the cluster to be established.

    Workaround: No workaround available.

  • Issue 28175:

    Business Policy NAT fails when the NAT IP overlaps with the VMware SD-WAN Gateway interface IP.

    Workaround: No workaround available.

  • Issue 31210:

    VRRP: ARP is not resolved in the LAN client for the VRRP virtual IP address when the VMware SD-WAN Edge is primary with a non-global CDE segment running on the LAN interface.

    Workaround: No workaround available.

  • Issue 32731:

    Conditional default routes advertised via OSPF may not be withdrawn properly when the route is deactivated.

    Workaround: Reactivating the route, followed by deactivating it again will retract it successfully.

  • Issue 32960:

    Interface “Autonegotiation” and “Speed” status might be displayed incorrectly on the Local Web UI for activated VMware SD-WAN Edges.

    Workaround: Refer to the Orchestrator UI under Remote Diagnostics > Interface Status.

  • Issue 32981:

    Hard-coding speed and duplex on a DPDK-configured port may require a VMware SD-WAN Edge reboot for the configurations to take effect as it requires turning DPDK off.

    Workaround: There is no workaround for this issue.

  • Issue 34254:

    When a Zscaler CSS is created and the Global Segment has FQDN/PSK settings configured, these settings are copied to Non-Global Segments to form IPsec tunnels to a Zscaler CSS.

  • Issue 35778:

    When there are multiple user-defined WAN links on a single interface, only one of those WAN links can have a GRE tunnel to Zscaler.

    Workaround: Use a different interface for each WAN link that needs to build GRE tunnels to Zscaler.

  • Issue 36923:

    Cluster name may not be updated properly in the NetFlow interface description for a VMware SD-WAN Edge which is connected to that Cluster as its Hub.

  • Issue 38682:

    VMware SD-WAN Edge acting as a DHCP server on a DPDK-configured interface may not properly generate “New Client Device" events for all connected clients.

  • Issue 38767:

    When a WAN overlay that has GRE tunnels to Zscaler configured is changed from auto-detect to user-defined, stale tunnels may remain until the next restart.

    Workaround: Restart the Edge to clear the stale tunnel.

  • Issue 39374:

    Changing the order of VMware SD-WAN Partner Gateways assigned to a VMware SD-WAN Edge may not properly set Gateway 1 as the local Gateway to be used for bandwidth testing.

  • Issue 39608:

    The output of the Remote Diagnostic “Ping Test” may display invalid content briefly before showing the correct results.

    Workaround: There is not workaround for this issue.

  • Issue 39624:

    Ping through a subinterface may fail when the parent interface is configured with PPPoE.

    Workaround: There is no workaround for this issue.

  • Issue 39753:

    Toggling Dynamic Branch-to-Branch VPN to off may cause existing flows currently being sent using Dynamic Branch-to-Branch to stall.

    Workaround: Only deactivate Dynamic Branch-to-Branch VPN in a maintenance window.

  • Issue 40421:

    Traceroute is not showing the path when passing through a VMware SD-WAN Edge with an interface configured as a switched port.

  • Issue 40096:

    If an activated VMware SD-WAN Edge 840 is rebooted, there is a chance an SFP module plugged into the Edge will stop passing traffic even though the link lights and the VMware SD-WAN Orchestrator will show the port as 'UP'.

    Workaround: Unplug the SFP module and then replug it back into the port.

  • Issue 42278:

    For a specific type of peer misconfiguration, the VMware SD-WAN Gateway may continuously send IKE init messages to a Non-SD-WAN peer. This issue does not disrupt user traffic to the Gateway; however, the Gateway logs will be filled with IKE errors and this may obscure useful log entries.

  • Issue 42872:

    Activating Profile Isolation on a Hub profile where a Hub Cluster is associated does not revoke the Hub routes from the routing information base (RIB).

  • Issue 43373:

    When the same BGP route is learnt from multiple VMware SD-WAN Edges, if this route is moved from preferred to eligible exit in the Overlay Flow Control, the Edge is not removed from the advertising list and continues to be advertised.

    Workaround: Activate Distributed Cost Calculation (DCC) on the VMware SD-WAN Orchestrator.

  • Issue 44995:

    OSPF routes are not revoked from VMware SD-WAN Gateways and VMware SD-WAN Spoke Edges when the routes are withdrawn from the Hub Cluster.

  • Issue 45189:

    With source LAN side NAT is configured, the traffic from a VMware SD-WAN Spoke Edge to a Hub Edge is allowed even without the static route configuration for the NAT subnet.

  • Issue 45302:

    In a VMware SD-WAN Hub Cluster, if one Hub loses connectivity for more than 5 minutes to all of the VMware SD-WAN Gateways common between itself and its assigned Spoke Edges, the Spokes may in rare conditions be unable to retain the hub routes after 5 minutes. The issue resolves itself when the Hub regains contact with the Gateways.

  • Issue 46053:

    BGP preference does not get auto-corrected for overlay routes when its neighbor is changed to an uplink neighbor.

    Workaround: An Edge Service Restart will correct this issue.

  • Issue 46137:

    A VMware SD-WAN Edge running 3.4.x software does not initiate a tunnel with AES-GCM encryption even if the Edge is configured for GCM.

    Workaround: If a customer is using AES-256, they must explicitly deactivate GCM from the Orchestrator prior to upgrading their Edges to a 4.x Release. Once all their Edges are running a 4.x release, the customer may choose between AES-256-GCM and AES-256-CBC.

  • Issue 46216:

    On a Non SD-WAN Destinations via Gateway or Edge where the peer is an AWS instance, when the peer initiates Phase-2 re-key, the Phase-1 IKE is also deleted and forces a rekey. This means the tunnel is torn down and rebuilt, causing packet loss during the tunnel rebuild.

    Workaround: To avoid tunnel destruction, configure the Non SD-WAN Destinations via Gateway/Edge or CSS IPsec rekey timer to less than 60 minutes. This prevents AWS from initiating the re-key.

  • Issue 46391:

    For a VMware SD-WAN Edge 3800, the SFP1 and SFP2 interfaces each have issues with Multi-Rate SFPs (i.e. 1/10G) and should not be used in those ports.

    Workaround: Please use single rate SFP's per the KB article VMware SD-WAN SupportedSFP Module List (79270). Multi-Rate SFPs may be used with SFP3 and SFP4.

  • Issue 47664:

    In a Hub and Spoke configuration where Branch-to-Branch via Hub VPN is not configured, trying to U-turn Branch-to-Branch traffic using a summary route on an L3 switch/router will cause routing loops.

    Workaround: Configure Cloud VPN to activate Branch-to-Branch VPN and select “Use Hubs for VPN”.

  • Issue 47681:

    When a host on the LAN side of a VMware SD-WAN Edge uses the same IP as that Edge’s WAN interface, the connection from the LAN host to the WAN does not work.

  • Issue 48530:

    VMware SD-WAN Edge 6x0 models do not perform autonegotiation for triple speed (10/100/1000 Mbps) copper SFP's.

    Workaround: Edge 520/540 supports triple speed copper SFPs but this model has been marked for End-of-Sale by Q1 2021.

  • Issue 48597: Multihop BGP neighborship does not stay up if one of the two paths to the peer goes down.

    If there is a Multihop BGP neighborship with a peer to which there are multiple paths and one of them goes down, user will notice that the BGP neighborship goes down and does not come up using the other available path(s). This includes the Local IP-loopback neighborship case too.

    Workaround: There is no workaround for this issue.

  • Issue 50518:

    On a VMware SD-WAN Gateway where PKI is configured, if >6000 PKI tunnels attempt to connect to the Gateway, the tunnels may not all come up because inbound SAs do not get deleted.

    Note:

    Tunnels using pre-shared key (PSK) authentication do not have this issue.

  • Issue 51436: For a site using an Enhanced High-Availability topology while deploying a VMware SD-WAN Edge using an LTE modem, if the site gets into a "split-brain" state, the HA failover takes ~5-6 minutes.

    As part of the recovery from a split-brain state, the LAN ports are brought down on the Active Edge and this impacts LAN traffic during the time the ports are down and until the site can recover.

    Workaround: There is no workaround for this issue.

  • Issue 52955: DHCP decline is not sent from Edge and DHCP rebinding is not restarted after DAD failure in Stateful DHCP.

    If DHCPv6 server allocates an address which is detected as duplicate by the kernel during a DAD check then the DHCPv6 client does not send a decline. This will lead to traffic dropping as the interface address will be marked as DAD check failed and will not be used. This will not lead to any traffic looping in the network but traffic blackholing will be seen.

    Workaround: There is no workaround for this Issue.

  • Issue 53219: After a VMware SD-WAN Hub Cluster rebalances, a few Spoke Edges may not have their RPF interface/IIF set properly.

    On the affected Spoke Edges, multicast traffic will be impacted. What happens is that after a cluster rebalance, some of the Spoke Edge fail to send a PIM join.

    Workaround: This issue will persist until the affected Spoke Edge has an Edge Service restart.

  • Issue 53934: In an enterprise where a VMware SD-WAN Hub Cluster is configured, if the primary Hub has Multihop BGP neighborships on the LAN side, the customer may experience traffic drops on a Spoke Edge when there is a LAN side failure or when BGP is not configured on all segments.

    In a Hub cluster, the primary Hub has Multihop BGP neighborship with a peer device to learn routes. If the physical interface on the Hub by which BGP neighborship is established, goes down, then BGP LAN routes may not become zero despite BGP view being empty. This may cause Hub Cluster rebalancing to not happen. The issue may also be observed when BGP is not configured for all segments and when there are one or more Multihop BGP neighborships.

    Workaround: Restart the Hub which had the LAN-side failure (or BGP not activated).

  • Issue 57210: Even when a VMware SD-WAN Edge is working normally and is able to reach the internet, the LED in the Local UI's Overview page shows as "Red".

    The Edge's Local UI determines the Edge's connectivity by whether it can resolve a well known name via Google's DNS resolver (8.8.8.8). If it cannot do so for any reason, then it thinks it is offline and shows the LED as red.

    Workaround: There is no workaround for this issue, except to ensure that DNS traffic to 8.8.8.8 can reach the destination and be resolved successfully.

  • Issue 61543: If more than one 1:1 NAT rule is configured on different interfaces with the same Inside IP, the inbound traffic can be received on one interface and the outbound packets of the same flow can be routed via different interface.

    For the NAT flows from Outside to Inside, the 1:1 NAT rules will be matched against the Outside IP and the interface where the packets are received. For the outbound packets of the same flow, the VMWare SD-WAN Edge will try to match the NAT rules again comparing the Inside IP and the outbound traffic can be routed via the interface configured in the first matching rule with "Outbound Traffic" configured.

    Workaround: There is no workaround for this issue outside of ensuring no more than one 1:1 NAT rule is configured with a particular Inside IP address.

  • Issue 65560: Traffic from a customer to PE (Provider Edge) device fails.

    BGP neighborship between a Partner Gateway and Provider Edge does not get established when tag-type is selected as "none" on the handoff configuration. This is because ctag, stag values get picked from /etc/config/gatewayd instead of the handoff configuration on the Orchestrator when tag-type is "none".

    Workaround: Update the ctag, stag values to 0 each under vrf_vlan->tag_info in /etc/config/ gatewayd. Do a vc_procmon restart.

  • Issue 67879: A Cloud Security Service (CSS) tunnel is deleted after a user changes a WAN Overlay setting from auto-detect to user-defined on a WAN interface setting.

    After saving the changes, the CSS tunnels do not come back up until the customer takes down and then puts back up the tunnel. Changing the WAN configuration will bring down the CSS tunnel and parse the CSS setup again. However, in some corner cases, the nvs_config>num_gre_links is 0 and the CSS tunnel fails to come up.

    Workaround: Deactivate the CSS setup, and then reactivate it and this will bring the CSS tunnel up.

  • Issue 68057: DHCPv6 release packet is not sent from the VMware SD-WAN Edge on the changing of a WAN interface address mode from DHCP stateful to static IPv6 address and the lease remains active till reaching its valid time.

    The DHCPv6 client possesses a lease which it does not release when the configuration change is done. The lease remains valid till its lifetime expires in the DHCPv6 server and is deleted.

    Workaround: There is no way of remediating this issue as the lease would remain active till valid lifetime.

  • Issue 68851: If a VMware SD-WAN Edge and VMware SD-WAN Gateway each have the same TCP syslog server configured, the TCP connection is not established from the Edge to the syslog server.

    If the Edge and Gateway each have the same TCP server and if the syslog packets from the Edge are routed via the Gateway, the syslog server sends a TCP reset to the Edge.

    Workaround: Send the syslog packets direct from the Edge instead of routing via a Gateway or configure a different syslog server for the Edge and Gateway.

  • Issue 69284: For a site using a High-Availability topology where the Edges deploy VNF's in an HA configuration and are using Release 4.x, if these HA Edges are downgraded to a 3.4.x Release where HA VNF's are not supported, and then upgraded to 4.5.0, when the HA VNF's are reactivated, the Standby Edge VNF will not come up.

    The VNF state on the Standby Edge is communicated as down via SNMP. If the HA VNF pair is downgraded from a version supporting VNF-HA (release 4.0+) to a release which does not support it with VNF configured on the Orchestrator. This issue will be seen when the Edge is upgraded back to a version supporting VNF-HA and it is configured on the Orchestrator again.

    Workaround: VNF should first be deactivated in the case of an HA configuration if the Edge is being downgraded to a version which does not support it.

  • Issue 72358: If the IP address of a VMware SD-WAN Orchestrator DNS name changes, the VMware SD-WAN Gateway's management plane process fails to resolve it properly and the Gateway will be unable to connect to the Orchestrator.

    The Gateway's management process periodically checks the DNS resolution of the

    Orchestrator's DNS name, to see if it has changed recently so that the Gateway can connect to the right host. The DNS resolution code has an issue in it so that all of these resolution checks fail, and the Gateway will keep using the old address and thus no longer be able to connect to the Orchestrator.

    Workaround: Until this issue is resolved, an Operator User should not change the IP address of the Orchestrator. If the Orchestrator's IP address must be changed, all Gateways connecting to that Orchestrator will have to be reactivated.

  • Issue 72762: For a customer enterprise site deployed with an Enhanced High Availability topology, the Remote Diagnostic > Traceroute does not work when run for the a WAN link connected to the Standby Edge.

    The Orchestrator UI shows the Standby Edge proxy WAN link when running the Remote Diagnostic > Interface Status, so the interface is up and link is recognized. In addition, a traceroute run for a Standby Edge proxy WAN link does not work when run from the Active Edge.

    Workaround: There is no workaround for this issue.

  • Issue 77541: When a USB modem that supports IPv6 is unplugged and then replugged into a VMware SD-WAN Edge USB interface, an IPv6 address may not provisioned to the USB interface.

    This affects USB modems that are IP-based, versus being managed by the ModemManager application. Most Inseego modems are IP-based and this is important because Inseego is the modem manufacturer VMware SASE recommends. USB modems supporting IPv6 which use ModemManager versus being IP-based will be fine in a plug out and plug in scenario.

    Workaround: The Edge needs to be rebooted (or power-cycled) after the USB modem is replugged into the Edge's USB port. Post reboot, the Edge will retrieve the IPv6 address for the modem.

  • Issue 81852: For a VMware SD-WAN Edge that is using a Zscaler type Cloud Security Service (CSS) which uses GRE tunnels that has turned on L7 Health Check, when that Edge is upgraded to Release 5.0.0, in some instances the customer may observe L7 Health Check errors.

    This is typically seen during software upgrade or during startup time. When L7 Health check for a CSS using GRE tunnels is turned on, error messages related to socket getaddress error may be seen. The observed error is intermittently seen, and not consistent. Because of this, L7 Health Check probe messages are not sent out.

    Workaround: Without the fix, to remediate the issue, a user needs to turn off and then turn back on the L7 Health Check configuration, and this feature would then work as expected.

  • Issue 82184: On a VMware SD-WAN Edge which is running Edge Release 5.0.0, when a traceroute or traceroute6 is run to the Edge's br-network IPv4/IPv6 address, the traceroute will not properly terminate when a UDP probe used.

    Traceroute or traceroute6 to the Edge's br-network IPv4/IPv6 address will not properly when Default Mode (in other words, UDP probe) is used.

    Workaround: Use -I option in traceroute and traceroute6 to use ICMP probe and then traceroute to br-network IPv4/IPv6 address will work as expected.

  • Issue 85402: For a customer enterprise using BGP with Partner Gateways configured, a user may obseve that some BGP neighborships are down and this causes customer traffic issues.

    If a customer has maximum-prefix configured on a router which has BGP peering with the Edge and Gateway, the BGP session may be dropped by the router.

    For example, if the router has BGP configured to only receive max 'n' number of prefixes, but the Edge and Gateway have more than 'n' number of prefixes to be advertised in the absence of any filters. Now if the BGP filter configuration is changed on the Orchestrator, even if the total number of prefixes allowed in the outbound direction is less than 'n', the issue will be encountered where more than 'n' prefixes are sent to the peer before any filters are applied. This causes the router to tear down the session.

    Workaround: If BGP goes down due to this issue (Maximum Number of Prefixes Reached), flap BGP on the peer using CLI (For FRR/Cisco, "neighbor x shut" followed by "no neighbor x shut"), and the BGP will produce only the desired number of prefixes advertised to the peer.

  • Issue 98136: For customer enterprises using a Hub/Spoke topology where Dynamic Branch To Branch VPN is configured, client users behind a SD-WAN Spoke Edge may observe that some traffic has unexpected latency resulting from the traffic using a sub-optimal path.

    Spoke Edge traffic that experiences this issue uses a route that was initially a non-uplink route for a Hub Edge not included in the Profile the Spoke Edge was using. A Dynamic Branch-to-Branch VPN tunnel can be formed from the Spoke Edge to the Hub Edge because of traffic being sent towards some other unrelated prefix and in this instance the non-uplink route is installed in the Spoke Edge.

    As a result of this non-uplink route, all traffic towards this prefix starts going through the

    Hub Edge and the non-uplink route becomes uplink (community change to uplink community) but the non-uplink route installed previously is not revoked and the traffic takes the Hub Edge path as long as the Dynamic Branch-to-Branch VPN tunnel remains up.

    Workaround: Wait for the Dynamic Branch-to-Branch VPN tunnel to tear down, after which the uplink route will not be installed in the Spoke Edge when a new Dynamic Branch-to-Branch VPN tunnel is formed towards the Hub Edge.

  • Issue 109771: A VMware SD-WAN Edge may not be able to establish a tunnel with a Cisco CSR/ASE type Non SD-WAN Destination via Edge when NAT66 is applied in between.

    If between two peers a source IPv6 address NAT is involved with a Cisco CSR/ASA, then a tunnel is not established.

    Workaround: On an Edge without a fix for this issue, the only thing a user can do is upgrade the Cisco CSR/ASA to a version of Cisco software that supports NAT66 over IPsec.

  • Issue 110561: Dynamic tunnels may not come up between the same set of VMware SD-WAN Edges with bidirectional traffic when traffic stops and then restarts.

    Issue is observed in a test environment where there are 6000 dynamic tunnels with high bidirectional traffic being sent between the Edges. Even in lower scale testing at 1000 dynamic tunnels, not all the tunnels come up.

    Workaround: There is no workaround for this issue.

  • Issue 111085: When a VMware SD-WAN Edge's WAN link is configured with an IP address in the same network as the Edge's loopback IP, the Edge uses the MAC address of the WAN interface while responding to an ARP request for the Edge's loopback IP address.

    This can cause ARP spoof and results in the Management IP being deprecated and network disruptions as a result.

    Workaround: There is no workaround for this issue.

  • Issue 113877: For customers who configure BGP/GRE LAN, those using a TGW GRE will experience BGP flaps and traffic interruption on the TGW secondary tunnel in all segments when the BGP configuration for TGW GRE is modified on the Global segment.

    When customer changes a BGP configuration of TGW GRE on the global segment then the secondary tunnel in the global and other segments flap, leading to a BGP connection reset and reconvergence and traffic interruption. The BGP connection will form again, and traffic will restore.

    Workaround: There is no workaround for this issue.

  • Issue 117876: In a customer site using a High Availability topology, if a user activates or deactivates the Enhanced Firewall Services, a VMware SD-WAN HA Edge may experience multiple restarts.

    When Enhanced Firewall Services is activated or deactivated, only the Active Edge's Device Settings configuration is synchronized immediately with the Standby Edge, with the remainder of the configuration synchronization is only in response to a Standby Edge heartbeat. When the Active Edge is restarted to apply the latest configuration prior to receiving a heartbeat from the Standby Edge it will result in a configuration mismatch between the two HA Edges and they will undergo multiple restarts to complete the configuration synchronization.

    Workaround: The only workaround is to turn on or off Enhanced Firewall Services during a maintenance window for HA Edges.

  • Issue 125274: When a customer runs an SNMP walk, the loopback interface of the VMware SD-WAN Edge is not discovered.

    The Edge loopback interface is a unique interface category that the Edge does not classify as either WAN or LAN. As a result, the loopback interface is not in the allow list of interfaces to process for the snmp-request.

    Workaround: There is no workaround for this issue. The loopback interface status would have to be individually monitored through the Orchestrator UI.

  • Issue 139476: When there is another route along with a static route for a destination prefix and the next hop interface of the static route goes down, traffic flow for that destination may be dropped.

    The flow route look-up does not select an alternative route when the static route is unreachable in a few scenarios. As a result, the flow tries to use the unreachable static route itself and that leads to the traffic being dropped.

    Workaround: Remove and then re-add static route.

  • Issue 143828: A customer may observe that an Edge has an unexpectedly high level of memory usage that may get sufficiently high to reach a critical level and trigger a defensive Edge service restart to recover the memory.

    One of the factors that contributes to this memory leak is extensive use of CLI commands on the Edge by either a Partner or Operator as part of troubleshooting or monitoring the Edge. These commands are accumulated and never cleared from the relevant Edge process. The use of Remote Diagnostics on the Orchestrator UI does not contribute to this issue.

    As with any Edge memory usage issue, entry level Edge models with lower RAM specifications (in other words, the Edge models 510, 610, 710, or 520) would be more likely to experience the issue, but it can happen on any Edge model with sufficiently high memory usage.

    Workaround: Extensive use of CLI commands on the Edge by either an Operator or Partner should be avoided, and if troubleshooting work is done, the customer should check the Edge's memory usage on the Edge > Monitor > System page.

  • Issue 145393: A customer enterprise site deployed with an Edge model 620, 640, or 680 where firewall logging is configured may observe that the Edge no longer stores new firewall or standard debugging logs.

    When this issue is encountered, a 6x0 Edge's eMMC storage experiences an excessive level of wear due to the high volume of writes and rewrites that can be triggered by enabling logging for firewall rules which are matched by a large number of new connections per second in a high traffic customer environment. This issue results in the Edge defensively moving the file partition which hosts logging to a read-only state, and no additional logs are stored.

    Workaround: If a customer has an Edge 620, 640, or 640 Edge model and is also using firewall logging, they should avoid enabling logging for firewall rules which can potentially match a large number of new connections in a high traffic environment. The excessive logging frequency that would result can cause undue wear on the Edge's storage and trigger this issue.

Orchestrator Known Issues

  • Issue 21342:

    When assigning Partner Gateways per-segment, the proper list of Gateway Assignments may not show under the Operator option "View" Gateways on the VMware SD-WAN Edge monitoring list.

  • Issue 24269:

    Monitor > Transport > Loss not graphing observed WAN link loss while QoE graphs do reflect this loss. 

  • Issue 25932:

    The VMware SD-WAN Orchestrator allows VMware SD-WAN Gateways to be removed from the Gateway Pool even when they are in use.

  • Issue 32335:

    The ‘End User Service Agreement’ (EUSA) page throws an error when a user is trying to accept the agreement.

    Workaround: Ensure no leading or trailing spaces are found in Enterprise Name.

  • Issue 32435:

    A VMware SD-WAN Edge override for a policy-based NAT configuration is permitted for tuples which are already configured at the profile level and vice versa.

  • Issue 32856:

    Though a business policy is configured to use the Hub cluster to backhaul internet traffic, the user can unselect the Hub cluster from a profile on a VMware SD-WAN Orchestrator that has been upgraded from Release 3.2.1 to Release 3.3.x.

  • Issue 35658:

    When a VMware SD-WAN Edge is moved from one profile to another which has a different CSS setting (e.g. IPsec in profile1 to GRE in profile2), the Edge level CSS settings will continue to use the previous CSS settings (e.g. IPsec versus GRE). 

    Workaround: At the Edge level, deactivate GRE, and then reactivate GRE to resolve the issue.

  • Issue 35667:

    When a VMware SD-WAN Edge is moved from one profile to another profile which has the same CSS setting but a different GRE CSS name (the same endpoints), some GRE tunnels will not show in monitoring.

    Workaround: At the Edge level, deactivate GRE and then reactivate GRE to resolve the issue.

  • Issue 36665:

    If the VMware SD-WAN Orchestrator cannot reach the internet, user interface pages that require accessing the Google Maps API may fail to load entirely.

  • Issue 32913:

    After activating High Availability, multicast details for the VMware SD-WAN Edge are not displayed on the Monitoring Page. A failover resolves the issue.

  • Issue 33026:

    The ‘End User Service Agreement’ (EUSA) page does not reload properly after deleting the agreement.

  • Issue 38056:

    The Edge-Licensing export.csv file not show region data.

  • Issue 38843:

    When pushing an application map, there is no Operator event, and the Edge event is of limited utility.

  • Issue 39633:

    The Super Gateway hyperlink does not work after a user assigns the Alternate Gateway as the Super Gateway.

  • Issue 39790:

    The VMware SD-WAN Orchestrator allows a user to configure a VMware SD-WAN Edge’s routed interface to have greater than the supported 32 subinterfaces, creating the risk that a user can configure 33 or more subinterfaces on an interface which would cause a Dataplane Service Failure for the Edge.

  • Issue 41691:

    User cannot change the 'Number of addresses' field although the DHCP pool is not exhausted on the Configure > Edge > Device page.

  • Issue 43276:

    User cannot change the Segment type when a VMware SD-WAN Edge or Profile has a Partner Gateway configured.

    Workaround: Temporarily remove the Partner Gateway configuration from the Profile or Edge so that the Segment can be changed between private and regular. Alternatively, the user can remove the Segment from the profile and make the change from there.

  • Issue 47713:

     If a Business Policy Rule is configured while Cloud VPN is toggled off, the NAT configuration must be reconfigured upon turning on Cloud VPN.

  • Issue 47820:

    If a VLAN is configured with DHCP toggled off at the Profile level, while also having an Edge Override for this VLAN on that Edge with DHCP activated, and there is an entry for the DNS server field set to none (no IP configured), the user will be unable to make any changed on the Configure > Edge > Device page and will get an error message of ‘invalid IP address []’ that does not explain or point to the actual problem.

  • Issue 48085: The VMware SD-WAN Orchestrator allows a user to delete a VLAN which is associated with an interface.

    When encountering this issue, the user would see an error message similar to "VLAN ID [xx] cannot be removed, in use by edge [b1-edge1 (GEx-disabled]".

  • Issue 51722: On the VMware SASE Orchestrator, the time range selector is no greater than two weeks for any statistic in the Monitor > Edge tabs.

    The time range selector does not show options greater than "Past 2 Weeks" in Monitor > Edge tabs even if the retention period for a set of statistics is much longer than 2 weeks. For example, flow and link statistics are retained for 365 days by default (which is configurable), while path statistics are retained only for 2 weeks by default (also configurable). This issue is making all monitor tabs conform to the lowest retained type of statistic versus allowing a user to select a time period that is consistent with the retention period for that statistic.

    Workaround: A user may use the "Custom" option in the time range selector to see data for more than 2 weeks.

  • Issue 60522: On the VMware SD-WAN Orchestrator UI, the user observes a large number of error messages when they try to remove a segment.

    The issue can be observed when adding a segment to a profile and the associating the segment with multiple VMware SD-WAN Edges. When the user attempts to remove the added segment from the profile, they will see a large number of error messages.

    Workaround: There is no workaround for this issue.

  • Issue 82095: User can configure invalid device settings for Edge VLANs that will result in significant connectivity issues for the Edge.

    The Orchestrator is not attempting to validate device configurations. In particular, a VLAN configuration for a switched port with an empty table. Some configurations can be so full of errors that the Edge's management process will fail.

    Workaround: Review all VLAN Device settings and ensure they are valid as the Orchestrator is not checking.

  • Issue 82680: For customer using MT-GRE Tunnel Automation, when a user turns off the Cloud-to-Cloud Interconnect (CCI) flag on a VMware SD-WAN Gateway which is configured to use CCI, the Zscaler MT-GRE entries may not get deleted from the Zscaler portal consistently.

    After a CCI site has been deleted from the Gateway, the entries for this site should also be removed. This issue has only been seen during test automation and has not been reproduced manually, but remains a risk.

    Workaround: Manually delete the resource from Zscaler before retrying.

  • Issue 82681: For customer using MT-GRE Tunnel Automation, when a user turns off the Cloud-to-Cloud Interconnect (CCI) flag on a VMware SD-WAN Gateway which is configured to use CCI, and the user deactivates the CCI flag from a VMware SD-WAN Edge with CCI configured which is using a Zscaler Cloud Security Service, the Zscaler MT-GRE entries may not get deleted from the Edge or from the Zscaler portal.

    After a CCI site has been deleted from the Gateway, the entries for this site should also be removed. This issue has only been seen during test automation and has not been reproduced manually, but remains a risk.

    Workaround: Manually delete the resource from Zscaler before retrying.

  • Issue 103769: An Operator may observe that a VMware SASE Orchestrator in a large scale deployment is experiencing performance issues which include 100% disk utilization and the Orchestrator no longer accumulating logs.

    This issue arises a change in logging behavior for the 5.1.0 Orchestrator that may result in the folders that store logs becoming full and also causing the Orchestrator CPU to reach 100% utilization. This issue arises a change in logging behavior for the 5.1.0 Orchestrator that may result in the folders that store logs becoming full and also causing the Orchestrator CPU to reach 100% utilization.

    Workaround: A Superuser Operator needs to log into the Orchestrator and clean up the pending logs.

  • Issue 117699: An Operator attempting to upgrade a 4.2.x VMware SD-WAN Orchestrator to become a Release 5.2.0 SASE Orchestrator may observe that the upgrade fails.

    The upgrade does not succeed, effectively stuck at the "Waiting for the CWS service up...". This issue is limited to 4.2.x Orchestrators.

    Workaround: The workaround for this issue is to upgrade the 4.2.x Orchestrator to 4.5.1 first, and then to Release 5.2.0.0.

  • Issue 125082: If a user configures a VMware SD-WAN Edge with an overridden DNS Server IP address on a VLAN, and then changes an interface setting for the Profile that Edge is using, the DNS Server IP address is no longer present for the Edge VLAN.

    The New UI does not send the override flag inside of the DHCP section and this causes any Profile changes to trigger an override of the DHCP section.

    Workaround: There is no workaround for this issue.

  • Issue 125504: If a static route is configured with next hop as a VLAN with IPv4/IPv6 address at the Profile level and then overridden at the Edge level and add an IPv4/IPv6 address to the VLAN, the static route is not marked as N/A and the VMware SASE Orchestrator asks for the interface in a dropdown menu.

    The expected behavior is where a static route configured with a next hop as a VLAN with IPv4/IPv6 address, the Orchestrator does not ask for the interface and the route is marked as N/A.

    Workaround: There is no workaround for this issue.

  • Issue 125663: A user can configure the same IPv4/IPv6 IP address for multiple Edge interfaces.

    The VMware SASE Orchestrator is allowing a user to configure the same IP on multiple WAN, LAN, or Sub Interfaces.

    Workaround: There is no workaround for this issue beyond ensuring you are not configuring the same IP Address for multiple interfaces.

  • Issue 126421: For Partners using a Partner Gateway, when configuring the Hand Off Details, the "Use for Private Tunnels" option is always checked no matter what a user does.

    This is not a cosmetic issue as the Orchestrator will apply the Use for Private Tunnels configuration to the Partner Gateway handoff and can impact customer traffic using the Partner Gateway.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 126425: When looking at Configure > Device > Routing & NAT page at the Profile level, the OSPF On/Off toggle button is missing.

    The OSPF On/Off toggle button was not migrated to the New UI at the Profile level and only shows at the Edge level.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 126465: The VMware SASE Orchestrator UI is not applying changes a user makes to create an Edge Cluster.

    If a user goes to the Configure > Edge > High Availability section of the UI and turns on HA with a Cluster type and creates a Hub Cluster with name xxxx, and saves changes, the user would observe that post-save the Cluster option is not selected under HA section and the created Hub Cluster with name xxxx is not present.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 126695: If a user is configuring webhooks for Alerts, when they click on the "Configure Payload Template" button the menu is not displayed.

    This issue occurs when configuring webhooks on the SD-WAN > Settings > Alerts > Webhooks page of the UI. When looking at the browser console, a user would also observe the message: ERROR TypeError: Cannot read properties of undefined (reading 'invalid').

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 127152: Users cannot save modified Interfaces with OSPF configurations on the VMware SASE Orchestrator UI.

    At the Profile level, when configuring either OSPFv2/OSPFv3, the Edit Interface dialog becomes invalid after changing any OSPF data.

    Workaround: On an Orchestrator without a fix for this issue, a user would need to activate MD5 Authentication and change the Key ID to any number from 1 to 255, and then deactivate MD5 Authentication.

  • Issue 128070: When a user is configuring OSPFv3 for a VLAN at the Edge level and attempts to add IPv6 Settings to the VLAN, the VMware SASE Orchestrator UI does not save the changes.

    The option to Save is grayed out and not available when attempting to add IPv6 Settings to a VLAN with OSPF3 at the Edge level.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 131997: An ICMP probe may be erroneously marked as down when it is configured for one segment but not for another.

    The Orchestrator fails to send the NULL ICMP probe configuration when it is not configured in a segment. As a result, the configuration of a different segment is reused and this causes the probes to fail.

    Workaround: On an Orchestrator without a fix for this issue, configure a dummy ICMP probe for the other segments.

  • Issue 133198: A user who logs in using RADIUS authentication cannot create a custom role on the Global Settings > User Management section of the Orchestrator UI.

    A RADIUS-authenticated user can go through the steps of creating a customer role but when they attempt to save the configuration, the UI throws the error "Error occurred while creating composite role" and the configuration is not saved.

  • Issue 134378: A VMware SD-WAN Edge upgraded to a 5.2.x software may experience continuous Dataplane Service failures and restart each time to recover.

    The issue can be encountered in a scenario where the Edge interface used to activate the Edge is configured with a VLAN that also includes a VLAN on the subinterface with the same VLAN ID for both. This can occur if the user adds the VLAN to the Edge through the Local UI while already having the Edge configured for a subinterface VLAN with the same ID through the Orchestrator for that Edge. The Orchestrator does not harmonize the Local UI configuration with the configuration it controls and the Edge is provided with a corrupted configuration that triggers the repeated service failures.

    Workaround: If a customer wishes to upgrade their Edges to 5.2.x the workaround is to ensure there are not duplicate VLAN IDs for the interface and subinterface for the Edge as outlined in the description.

check-circle-line exclamation-circle-line close-line
Scroll to top icon