This site will be decommissioned on December 31st 2024. After that date content will be available at techdocs.broadcom.com.

 

VMware SASE 5.4.0 | 30 September 2024

  • VMware SASE™ Orchestrator Version R5409-20240918-GA

  • VMware SD-WAN™ Gateway Version R5401-20240314-GA-141087

  • VMware SD-WAN™ Edge Version R5401-20240718-GA

Check for additions and updates to these release notes.

What Is in The Release Notes

The release notes cover the following topics:

This release is recommended for all customers who require the features and functionality first made available in Release 5.4.0.

Important:

Release 5.4.0 contains all Edge and Gateway fixes listed in the 5.2.0 Release Notes up to the 5.2.0.2 version, and all Orchestrator fixes listed in the 5.2.0 Release Notes up to the 5.2.0.3 version, and in the 5.3.0 Release Notes up to the 5.3.0.2 version.

Compatibility

Release 5.4.0 Orchestrators, Gateways, and Hub Edges support all previous VMware SD-WAN Edge versions greater than or equal to Release 4.2.0.

The following SD-WAN interoperability combinations were explicitly tested:

Orchestrator

Gateway

Edge

Hub

Branch/Spoke

5.4.0

4.2.2

4.2.2

4.2.2

5.4.0

5.4.0

4.2.2

4.2.2

5.4.0

5.4.0

5.4.0

4.2.2

5.4.0

5.4.0

4.2.2

5.4.0

5.4.0

4.3.2

4.3.2

4.3.2

5.4.0

5.4.0

4.3.2

4.3.2

5.4.0

5.4.0

5.4.0

4.3.2

5.4.0

5.4.0

4.3.2

5.4.0

5.4.0

4.5.2

4.5.2

4.5.2

5.4.0

5.4.0

4.5.2

4.5.2

5.4.0

5.4.0

5.4.0

4.5.2

5.4.0

5.4.0

4.5.2

5.4.0

5.4.0

5.0.1

5.0.1

5.0.1

5.4.0

5.4.0

5.0.1

5.0.1

5.4.0

5.4.0

5.4.0

5.0.1

5.4.0

5.4.0

5.0.1

5.4.0

5.4.0

5.1.0

5.1.0

5.1.0

5.4.0

5.4.0

5.1.0

5.1.0

5.4.0

5.4.0

5.4.0

5.1.0

5.4.0

5.4.0

5.1.0

5.4.0

5.4.0

5.2.0

5.2.0

5.2.0

5.4.0

5.4.0

5.2.0

5.2.0

5.4.0

5.4.0

5.4.0

5.2.0

5.4.0

5.4.0

5.2.0

5.4.0

5.4.0

5.4.0

5.4.0

5.4.0
Important:

VMware SD-WAN Release 4.0.x has reached End of Support; Releases 4.2.x, 4.3.x, and 4.5.x have reached End of Support for Gateways and Orchestrators.

  • Release 4.0.x reached End of General Support (EOGS) on September 30, 2022, and End of Technical Guidance (EOTG) December 31, 2022. 

  • Release 4.2.x Orchestrators and Gateways reached End of General Support (EOGS) on December 30, 2022, and End of Technical Guidance on (EOTG) March 30, 2023.   

  • Release 4.2.x Edges reached End of General Support (EOGS) on June 30, 2023, and will reach End of Technical Guidance (EOTG) September 30, 2025.

  • Release 4.3.x Orchestrators and Gateways reached End of General Support (EOGS) on June 30, 2023, and End of Technical Guidance (EOTG) September 30, 2023.

  • Release 4.3.x Edges reached End of General Support (EOGS) on June 30, 2023, and will reach End of Technical Guidance (EOTG) September 30, 2025.

  • Release 4.5.x Orchestrators and Gateways reached End of General Support (EOGS) on September 30, 2023, and End of Technical Guidance on (EOTG) December 31, 2023.

  • For more information please consult the Knowledge Base article: Announcement: End of Support Life for VMware SD-WAN Release 4.x (88319).

Upgrade Paths for Orchestrator, Gateway, and Edge

The following lists the paths for customers wishing to upgrade their Orchestrator, Gateway, or Edge from an older release to Release 5.4.0.

Orchestrator

Orchestrators using Release 4.3.0 or later can be upgraded to Release 5.4.0. 

Gateway

Upgrading a Gateway using Release 4.3.0 or later to Release 5.4.0 is fully supported for all Gateway types.

Important:

When deploying a new Gateway using 5.4.0 the VMware ESXi instance must be at least version 6.7, Update 3 up to version 7.0. Using an earlier ESXi instance will result in the Gateway's Dataplane Service failing when trying to run Release 5.4.0 or later.

Important:

Prior to upgrading a Gateway to 5.4.0, the ESXi instance must be upgraded to at least version 6.7, Update 3 up to version 7.0. Using an earlier ESXi instance will result in the Gateway's Dataplane Service failing when trying to run Release 5.4.0 or later.

Edge

An Edge can be upgraded directly to Release 5.4.0 from any Release 4.x or later.

New SASE Features and Enhancements

Bastion Orchestrator, Phase 2

First introduced in Release 4.3.0, VMware SASE delivers major improvements to the Bastion Orchestrator in Release 5.4.0. Phase 2 improvements include:

  • SD-WAN Edge software update when the Edge is in activated state.

  • Reversion to the last known good configuration in the event of an Edge promotion failure.

  • Events from an unpromoted Edge are sent to a private VMware SASE Orchestrator.

  • Generation of a diagnostic bundle for an unpromoted Edge.

  • Enterprise Firmware updates on the Bastion VMware SASE Orchestrator.

SD-WAN Client Integrated with the VMware SASE Orchestrator

VMware SD-WAN Client can now be managed via the VMware SASE Orchestrator, delivering a unified management console for managing networking, security, and remote access solutions.

New SD-WAN Features and Enhancements

Security Service Edge (SSE)

Release 5.4.0 supports the Security Service Edge (SSE) feature, which allows VMware SD-WAN to easily integrate with a third party SSE vendor using seamless automation through the Orchestrator. VMware SD-WAN offers this feature with support for two vendors: Symantec SSE and Prisma Access by Palo Alto Networks. You can configure multiple SSE integrations with the same vendor and automatically setup IPsec tunnels from all participating SD-WAN Edge devices to either Symantec SSE or Prisma Access, significantly reducing the time spent in manual provisioning and the complexity of using two separate consoles during deployment.

For more information including how to configure this feature, see the Security Service Edge (SSE) documentation in the SD-WAN Administrative Guide.

Edge Firewall Support for FTPv6

Release 5.4.0 includes enhanced application identification for both FTPv4/FTPv6 Active and Passive modes when using VMware SD-WAN Deep Packet Inspection (DPI). This improved DPI is especially helpful for customers who use Passive FTPv6 mode, which assigns random port numbers for data transfer and makes FTP traffic difficult to identify because it does not use the standard ports 20 and 21.

Enhanced Firewall Service Viewing and Searching Enhancements

The Enhanced Firewall Service now includes a firewall table view with a comments field and a new search capability for firewall rules and objects to deliver an optimal user experience.

Enhanced Firewall Service Signature View (IDS/IPS)

The Enhanced Firewall Service includes an improved Signature View (IDS/IPS), which provides a simplified view with visibility of Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS) signatures installed on a VMware SD-WAN Edge along with the installed version, data, and time.

High Availability Enhancements, Phase 2

Release 5.4.0 includes additional improvements for a site deployed using a High Availability topology with a pair of Edges. These include:

  • Alerts: A new alert is added to the list on the Service Settings > Alerts & Notifications page, Edge HA Failed. The Edge HA Failed alert is triggered when the Standby Edge fails to heartbeat to the Active Edge and is marked by the Active Edge as down. This alert is especially useful in Enhanced HA deployments where the Standby Edge also passes customer traffic.

  • Compatibility between Wi-Fi and Non Wi-Fi Capable Edges: Prior to Edge Release 5.4.0, Edge models which did not include a Wi-Fi module (510N, 610N, 620N, 640N, and 680N) could not be used with a Wi-Fi capable counterpart in an HA deployment. For example, an Edge 640 and an Edge 640N were not supported as a High Availability pair. For Release 5.4.0 and forward, this pairing is now supported.

Note:

In a scenario with mismatched Wi-Fi and Non Wi-Fi Edges, the Orchestrator detects the Edge mismatch and automatically deactivates Wi-Fi capability on the Edge that is Wi-Fi capable. The mismatch log is shown in the customer's Events:

  • HA Wi-Fi capability mismatch identified, disabled Wi-Fi (an Edge Wi-Fi mismatch is identified and Wi-Fi is deactivated on the Wi-Fi capable Edge).

  • HA Wi-Fi capability mismatch no longer seen, reverted Wi-Fi (both Edges are detected as the same Wi-Fi type, and Wi-Fi functionality is restored on a Wi-Fi Edge where it was previously deactivated).

Hub or Cluster Interconnect is GA

First introduced as an Early Access Feature in SD-WAN Release 5.1.0, Hub or Cluster Interconnect is fully GA in Release 5.4.0. This solution empowers customers to build a hierarchical and scalable architecture with full SD-WAN overlay connectivity across cloud, regional, and data center hubs, providing full DMPO protection (Dynamic Multipath Optimization), end-to-end visibility, and reliability.

In addition to full support for this feature, the previous hop count restriction of two is raised to a qualified number of four hops.

IPv6 DHCPv6 Prefix Delegation

Release 5.4.0 adds DHCPv6 Prefix Delegation support for customers where their delegating router is either part of cloud hosting or at a remote location, or in scenarios where the customer's ISP allocates IP addresses. DHCPv6 Prefix Delegation includes new address types for IPv6, and supports two ISP prefix delegation servers to allow a primary and backup topology.

Important:

To use the Prefix Delegation feature, the Orchestrator, Gateway, and Edge must all be using a 5.4.0 software version or later. Prefix Delegation is not supported for use on an Edge using 5.2.0 or earlier and if an Edge using Release 5.2.0 or earlier is configured for Prefix Delegation, the feature does not work as expected.

In addition, an Edge using 5.2.0 or earlier configured with Prefix Delegation cannot be upgraded to 5.4.0. Thus, if upgrading a 5.2.0 or earlier Edge to 5.4.0 or later, ensure that Prefix Delegation is not being used on that Edge.

Mellanox Bifurcated Driver Support for the Microsoft Azure Virtual Edge

VMware SD-WAN provides Accelerated Networking (SR-IOV) support for the Microsoft Azure Virtual Edge and includes support for Mellanox ConnectX-4 and ConnectX-5 NICs. When activating SR-IOV on a Azure Virtual Edge interface, the enhancement is automatically activated on the Edge.​

Accelerated Networking can be activated or deactivated through the Azure portal, the Azure CLI, or the Azure PowerShell.​

Note:

This enhancement does not include support for Mellanox ConnectX-3 NICs​

Run-To-Completion FastPath on the Edge, Phase 3

Run-to-completion is a software optimization which increases the performance on Edges and Gateways which results in an improvement in the overall efficiency of the SD-WAN solution.

Important Notes

Change in Behavior for the BGP MED Attribute for Hub Edges

A change introduced in Release 5.2.0 and carried in all later builds includes a change in the BGP MED Attribute as follows: previously, the BGP MED value advertised by Hub Edges started from value 9, 10, 11, and so forth. As part of a fix for Issue #111840, the BGP MED value now starts from 33, 34, 35, and so forth. The description for Issue #111840 can be found in the 5.2.0 Release Notes in the Resolved in Edge/Gateway Version R5200-20230530-GA section.

VMware Security Advisory 2024-0008

VMware SASE Global Settings Guide

Beginning in Release 5.4.0 and forward, documentation for features that fall under the Global Settings section of the Orchestrator are moved from the SD-WAN Administration Guide to a new guide: VMware SASE Global Settings Guide. This was done because the features configured in Global Settings apply to all of the VMware SASE services (SD-WAN, Cloud Web Security, Edge Network Intelligence, SD-WAN Client, or Secure Access). The topics moved from the SD-WAN Administration Guide to the Global Settings Guide include:

  • User Management for Enterprises (Adding a User, User Roles, Service Permissions, Authentication, and so forth)

  • Enterprise Management

  • Customer Configuration (Partner Handoff, Distributed Cost Calculation, Path Calculation)

LAN-Side NAT Behavioral Change

Beginning in Release 4.5 and forward, when a LAN-side NAT is configured for many-to-one translations using Port Address Translation (PAT), traffic initiated from the opposite direction can allow unexpected access to fixed addresses based on the outside mask and original IP address. This new behavior applies to Destination NAT (DNAT), Source NAT (SNAT), and Source and Destination NAT (S+D NAT) rules.

For example, a SNAT rule with an inside network of 192.168.1.0/24 and an outside address of 10.1.1.100/32 permits outside-to-inside translation to 192.168.1.100.

To address this new behavior, SD-WAN now blocks traffic when a connection is initiated in the reverse PAT direction.

To restore the original behavior, a user needs to configure two rules of the same type as the original rule (SNAT, DNAT, S+D NAT) in a particular order. For example, using the earlier SNAT scenario a user needs to configure the following:

  1. SNAT rule with an inside network of 192.168.1.100/32 and an outside address of 10.1.1.100/32

  2. SNAT rule with an inside network of 192.168.1.0/24 and an outside address of 10.1.1.100/32

If the original rule is a DNAT or S+D NAT, then the user would need two DNAT or S+D NAT rules with the same structure and order.

In Release 4.5.0 through 5.2.0, a user can determine if flows are dropped for this type of traffic in the dispcnt logs of a diagnostic bundle by searching for the counter lan_side_nat_reverse_pat_drop.

Beginning in Release 5.4.0 and forward, a user can find six separate counters in the logs:

  • lan_side_nat_rev_pat_drop_snat1

  • lan_side_nat_rev_pat_drop_snat2

  • lan_side_nat_rev_pat_drop_dnat1

  • lan_side_nat_rev_pat_drop_dnat2

  • lan_side_nat_rev_pat_drop_sdnat1

  • lan_side_nat_rev_pat_drop_sdnat2

Limitation When Deactivating Autonegotiation on VMware SD-WAN Edge Models 520, 540, 620, 640, 680, 3400, 3800, and 3810

When a user deactivates autonegotiation to hardcode speed and duplex on ports GE1 - GE4 on a VMware SD-WAN Edge model 620, 640 or 680; on ports GE3 or GE4 on an Edge 3400, 3800, or 3810; or on an Edge 520/540 when an SFP with a copper interface is used on ports SFP1 or SFP2, the user may find that even after a reboot the link does not come up.

This is caused by each of the listed Edge models using the Intel Ethernet Controller i350, which has a limitation that when autonegotiation is not used on both sides of the link, it is not able to dynamically detect the appropriate wires to transmit and receive on (auto-MDIX). If both sides of the connection are transmitting and receiving on the same wires, the link will not be detected. If the peer side also does not support auto-MDIX without autonegotiation, and the link does not come up with a straight cable, then a crossover Ethernet cable will be needed to bring the link up.

For more information please see the KB article Limitation When Deactivating Autonegotiation on VMware SD-WAN Edge Models 520, 540, 620, 640, 680, 3400, 3800, and 3810 (87208).

Orchestrator API Changes

Orchestrator API Changes since 5.3.0

Changes to the VMware SASE Orchestrator Portal API ("API v1")

The following are issues that impact API v1 users:

Ticket

Symptom/Description

Known Issue #123867

The link metrics and series APIs return an error when called without a list of metrics. This is an issue that will be fixed in a future release. To work around this issue, you can submit your API request with a list of the metric fields that you want to return. This will ensure that you receive the data you need, even though the API will return an error message.

This issue does not affect the functionality of the APIs, so you can still use them to retrieve data.

Known Issue #127994

The Swagger spec is reporting a schema error due to an additionalProperty: title in the response schema. This is a cosmetic issue and does not affect the functionality of the API. This issue will be resolved in a future release and the API can still be used to retrieve data, even if the Orchestrator receives this schema error.

Note:

This issue is resolved in Orchestrator version 5.4.0.3.

Fixed Issue #127995

The Swagger spec is reporting a schema error due to an incorrect type in the items field of the response schema. This is a cosmetic issue and does not affect the API's ability to retrieve data, and the error can be safely ignored. This issue is fixed in Release 5.4.0.

For reference, the complete API Changelog is available for download at developer.vmware.com (see "VMware SD-WAN Orchestrator API v1").

Changes to the VMware SASE Orchestrator API v2

There are no new API v2 APIs since release 5.3.0.

The following are changes in the API v2 from 5.3.0 to 5.4.0.

Ticket

Symptom/Description

Fixed Issue #116610

Users cannot add a new loopback interface via APIv2. The schema structure for the loopback interface in APIv2 did not allow interfaces named by the interface ID. So, the update of the loopback interface fails.

Fixed Issue #129679

Customers will not be able to update the Edge's device settings module using the APIv2 PATCH Edge device settings when the Edge's device settings module has subinterfaces configured. When subinterfaces are configured on the Edge's device settings module, an API call to v2 PATCH edge device settings API will result in the request pending in an "Accepted" status forever and eventually times out. As a result, customers will not see any changes reflected in the edge's device settings module on the Orchestrator.

Developer Documentation

All VMware SASE/SD-WAN API documentation resides on the Developer Documentation Portal at https://developer.vmware.com/apis.

Available Languages

The VMware SASE Orchestrator using version 5.4.0 is localized into the following languages: Czech, English, European Portuguese, French, German, Greek, Italian, Spanish, Japanese, Korean, Simplified Chinese, and Traditional Chinese.

Document Revision History

September 30th, 2024. Twenty-Third Edition.

  • Added a new Orchestrator rollup build R5409-20240918-GA to the Orchestrator Resolved Issues section. This is the ninth Orchestrator rollup build and is the new default Orchestrator GA build for Release 5.4.0.

  • Orchestrator build R5409-20240918-GA includes the fixes for issues #139581, #143506, #148179, #148274, and #148873, each of which is documented in this section.

  • Orchestrator build 5.4.0.9 adds support for provisioning and configuring Edge models 710-5G, 720, and 740 on the UI.

Note:

These new Edge models must be deployed with a 5.2.4.x Edge build as they do not have 5.4.0.x Edge software support.

  • Added Fixed Issue #139369 to Orchestrator build R5404-20240228-GA. This was omitted in error from the seventh edition of these notes.

September 3rd, 2024. Twenty-Second Edition.

August 24th, 2024. Twenty-First Edition.

  • Changed Fixed Issue #121606 back to an Open Issue and moved it from the Edge/Gateway Resolved Issues section of Edge/Gateway Version R5400-20231009-GA to the Edge/Gateway Known Issues section of the Release Notes.

July 29th, 2024. Twentieth Edition.

  • Added a new Orchestrator rollup build R5408-20240725-GA to the Orchestrator Resolved Issues section. This is the eighth Orchestrator rollup build and is the new default Orchestrator GA build for Release 5.4.0.

  • Orchestrator build R5408-20240725-GA includes the fixes for issues #118684, #133642, #145134, #145783, and #147155, each of which is documented in this section.

July 22nd, 2024. Nineteenth Edition.

  • Added a new Edge hotfix build R5401-20240718-GA to the Edge/Gateway Resolved Issues section. This is the new default Edge GA build for Release 5.4.0.

  • This Edge build remediates CVE-2024-6387, a critical vulnerability in OpenSSH. For more information on this OpenSSH vulnerability, please consult the article Broadcom Software Defined Edge Division response to CVE-2024-6387.

  • Note:

    CVE-2024-6387 could potentially impact a 5.4.0 Edge, but does not impact a 5.4.0 Gateway. This is why there is an Edge build only.

June 25th, 2024. Eighteenth Edition.

  • Added the feature Security Service Edge (SSE) to the New SD-WAN Features and Enhancements section. This feature was first listed in the 5.3.0 Release Notes, and is now listed here as the 5.4.0 Orchestrator is the earliest available version which includes the SSE feature. This section is also revised to include Symantec SSE as one of the two vendors supported for the SSE configuration along with Prisma Access.

June 20th, 2024. Seventeenth Edition.

May 15th, 2024. Sixteenth Edition.

  • Added a new Orchestrator rollup build R5407-20240510-GA to the Orchestrator Resolved Issues section. This is the seventh Orchestrator rollup build and is the new default Orchestrator GA build for Release 5.4.0.

  • Orchestrator build R5407-20240510-GA includes the fixes for issues #63453, #123962, #138712, #139409, #139548, #141617, #142230, and #144759, each of which is documented in this section.

  • Added Open Issue #143828 to the Edge/Gateway Known Issues section.

May 2nd, 2024. Fifteenth Edition.

  • Added a new Important Notes: Change in Behavior for the BGP MED Attribute for Hub Edges, where the BGP MED value for Hub Edges no longer starts from 9, 10, 11, and so forth, but now starts from 33, 34, 34, and so forth. This change was introduced in Release 5.2.0 as part of Fixed Issue #111840, and applies to all 5.4.0 Edge software.

April 25th, 2024. Fourteenth Edition.

  • Added Open Issue #143551 to the Edge/Gateway Known Issues section.

April 19th, 2024. Thirteenth Edition.

  • Added Open Issue #143450 to the Edge/Gateway Known Issues section.

April 12th, 2024. Twelfth Edition.

  • Added Open Issues #138773 and #142366 to the Edge/Gateway Known Issues section.

  • Added Fixed Issue #141886 to the Orchestrator Resolved Issues section for rollup build R5406-20240323-GA. This ticket should have been included in the Tenth Edition of these Release Notes.

  • Corrected the wording for Open Issue #118704 to change the workaround from a CLI action to an action on the Orchestrator UI to restart the Edge service to remediate the issue.

April 2nd, 2024. Eleventh Edition.

  • Added an Important Note regarding CVE-2024-22247, which details a missing authentication and protection mechanism vulnerability that impacts an SD-WAN Edge. VMware's response to this vulnerability is documented in VMSA-2024-0008. More information on mitigating this vulnerability is found in the KB article: VMware Response to CVE-2024-22247 (VMSA-2024-0008) (97391).

March 28th, 2024. Tenth Edition.

  • Added a new Orchestrator rollup build R5406-20240323-GA to the Orchestrator Resolved Issues section. This is the sixth Orchestrator rollup build and is the new default Orchestrator GA build for Release 5.4.0.

  • Orchestrator build R5406-20240323-GA includes the fixes for issues #115570, #130943, #136085, #136210, #138417, #139467, #140063, #140322, #140390, and #141535 each of which is documented in this section.

  • Added Open Issue #118704 to the Edge/Gateway Known Issues section.

March 15th, 2024. Ninth Edition.

  • Added a new Gateway Hotfix build R5401-20240314-GA-141087 to the Edge/Gateway Resolved section. This is the new Gateway GA build for Release 5.4.0.

  • Gateway Hotfix build R5401-20240314-GA-141087 includes the fix for issue #141087, which is documented in this section.

  • Added Fixed Issue #108583 to the Orchestrator Resolved Issues section for the original GA build R5400-20231020-GA.

  • Added Open Issue #139409 to the Orchestrator Known Issues section.

March 8th, 2024. Eighth Edition.

  • Added a new Orchestrator rollup build R5405-20240307-GA to the Orchestrator Resolved Issues section. This is the fifth Orchestrator rollup build and is the new default Orchestrator GA build for Release 5.4.0.

  • Orchestrator build R5405-20240307-GA includes the fixes for issues #129239, #134498, #135804, #138932, and #139966, each of which is documented in this section.

  • Added Open Issues #133092 and #136085 to the Orchestrator Known Issues section.

  • Added Open Issue #72762 to the Edge/Gateway Known Issues section.

March 1st, 2024. Seventh Edition.

  • Added a new Orchestrator rollup build R5404-20240228-GA to the Orchestrator Resolved Issues section. This is the fourth Orchestrator rollup build and is the new default Orchestrator GA build for Release 5.4.0.

  • Orchestrator build R5404-20240228-GA includes the fixes for issues #112612, #128753, #129694, #131496, #132789, #132827, #133180, #133240, #133851, #133983, #134078, #134911, #134940, #135517, #136247, #136404, #136454, #136622, #136930, #136937, #137281, #137282, #137447, #137699, and #139437, each of which is documented in this section.

  • Added a new Edge/Gateway rollup build R5401-20240224-GA to the Edge/Gateway Resolved section. This is the first Edge/Gateway rollup build and is new default Edge/Gateway GA build for Release 5.4.0.

  • Edge/Gateway build R5401-20240224-GA includes the fixes for issues #89332, #123594, #125336, #127443, #129311, #130833, #131029, #131122, #131891, #135544, #135810, #136992, and #138006, each of which is documented in this section.

  • Changed Fixed Issue #129239 back to an Open Issue and moved it from the Orchestrator Resolved Issues section of Orchestrator rollup build R5403-20240126-GA to the Orchestrator Known Issues section of the Release Notes.

January 29th, 2024. Sixth Edition.

  • Added a new Orchestrator rollup build R5403-20240126-GA to the Orchestrator Resolved Issues section. This is the third Orchestrator rollup build and is the new default Orchestrator GA build for Release 5.4.0.

  • Orchestrator build R5403-20240126-GA includes the fixes for issues #118501, #122193, #124849, #125006, #125082, #125456, #125837, #126695, #127268, #127994, #128330, #128368, #128888, #128921, #129088, #129232, #129239, #129662, #129958, #131224, #131464, #131631, #132047, #132598, #133008, #133102, #133198, #133199, #133201, #133664, #134435, and #135644, each of which is documented in this section.

January 2nd, 2024. Fifth Edition.

  • Added a new Edge/Gateway Hotfix build R5400-20231230-GA-134893 to the Edge/Gateway Resolved section. This is the new Edge/Gateway GA build for Release 5.4.0.

  • Edge/Gateway Hotfix build R5400-20231230-GA-134893 includes the fix for issue #134893, which is documented in this section.

November 27th, 2023. Fourth Edition.

  • Added Fixed Issues #129695, #130810, and 132524 to the Orchestrator Resolved Issues section for Orchestrator build R5402-20231122-GA. These issues were omitted in error from the Third Edition of the Release Notes.

November 22nd, 2023. Third Edition.

  • Added a new Orchestrator rollup build R5402-20231122-GA to the Orchestrator Resolved Issues section. This is the first Orchestrator rollup build and is the new default Orchestrator GA build for Release 5.4.0.

  • Orchestrator build R5402-20231122-GA includes the fixes for issues #128330, #131789, and #132449, each of which is documented in this section.

  • Added Fixed Issue #131138 to the Orchestrator Resolved Issues section for Orchestrator build R5401-20231115-GA. This issue was omitted in error from the Second Edition of the Release Notes.

  • Moved Fixed Issue #131846 to the Orchestrator Known Issues section as it is not fixed as previously documented in the Second Edition under Orchestrator Build R5401-20231115-GA.

  • Added a new Important Note: VMware SASE Global Settings Guide to notify users of a new guide that documents all the features that fall under the Global Settings section of the Orchestrator and which were previously documented in the SD-WAN Adminstration Guide. This includes all user management features and customer configurations like partner handoff.

November 16th, 2023. Second Edition.

  • Added a new Orchestrator rollup build R5401-20231115-GA to the Orchestrator Resolved Issues section. This is the first Orchestrator rollup build and is the new default Orchestrator GA build for Release 5.4.0.

  • Orchestrator build R5401-20231115-GA includes the fixes for issues #117138, #123078, #123387, #125309, #125964, #126602, #127727, #127774, #127843, #127904, #128017, #128277, #128279, #128357, #128765, #129061, #129494, #129584, #129756, #129765, #129894, #130153, #130877, and #131846, each of which is documented in this section.

  • Added updated Edge build R5400-20231108-GA-125647 to the Edge/Gateway Resolved section. This is an update to the original Edge GA build R5400-20231009-GA and is the new default Edge/Gateway GA build for Release 5.4.0.

    Edge build R5400-20231108-GA-125647 includes the fix for issue #125647, which is documented in this section.

    Important:

    • The previous Release 5.4.0 Edge build is deprecated in favor of R5400-20231108-GA-125647.

    • Customers upgrading to 5.4.0 should only upgrade to R5202-20231107-GA-125647.

    • Customers already successfully using the original 5.4.0 Edge build do not need to upgrade their Edges to R5400-20231108-GA-125647.

October 19, 2023. First Edition.

Edge and Gateway Resolved Issues

Edge Build R5401-20240718-GA

Edge version R5401-20240718-GA was released on 07-22-2024 and is a hotfix build for Release 5.4.0.

This Edge build includes a remediation for CVE-2024-6387, a critical vulnerability in OpenSSH. For more information on this OpenSSH vulnerability, please consult the article Broadcom Software Defined Edge Division response to CVE-2024-6387.

Edge build R5401-20240718-GA adds no additional software fixes to the 1st Edge rollup build, version R5401-20240224-GA.

Resolved in Gateway Version R5401-20240314-GA-141087

Gateway build R5401-20240314-GA-141087 was released on 03-15-2024 and is a Hotfix build for Gateway Release 5.4.0.

This Gateway Hotfix build addresses the below critical issue since the 1st Gateway rollup build, R5401-20240224-GA.

  • Fixed Issue 141087: If one or more customer enterprises connected to a VMware SD-WAN Gateway use Dynamic Branch-to-Branch, the Gateway may experience a Dataplane Service failure, generate a core, and restart to restore functionality.

    When an Edge asks the Gateway to provide the endpoint information for another Edge to form a Dynamic Branch-to-Branch tunnel, the Gateway adds this information to a hash table. Once the information is delivered to the requesting Edge, the entry is removed. The Gateway service runs a timer where, at 60 second intervals, the Gateway removes the entries which have not been delivered to the requesting Edge.

    In this context, the Gateway service can experience a mutex induced service failure caused by the hash table iterating for over 90 seconds (which triggers an exception). An Operator running a backtrace would observe that the failure points to vc_de2e_entry_timeout_iter_cb.

Resolved in Edge/Gateway Version R5401-20240224-GA

Edge/Gateway build R5401-20240224-GA was released on 03-01-2024 and is the 1st Edge/Gateway rollup build for Release 5.4.0.

This Edge/Gateway rollup build addresses the below critical issues since the previous hotfix build, R5400-20231230-GA-134893.

  • Fixed Issue 89332: If the VMware SD-WAN Edge's /velocloud directory is full or cannot be written into, the customer is unaware of this status in Events.

    The customer would only notice that the Edge was not posting additional Events and that could be days before they did. Reporting Events to the Orchestrator happens by writing a file into the /velocloud directory. If either the directory is full, or if there is an issue with the disk such that it goes into read-only mode, the customer and VMware Engineering/Support has no way of knowing this since there is no Event or logging to warn of this.

    The fix for this issue adds a new log entry written to a different folder to indicate clearly what is happening that would read similar to the following:

    • edge:b1-edge1:~# cat /var/log/log_eventgen.log

    • 2023-05-19 13:03:46,628 Unable to create event save file: [Errno 30] Read-only file system: '/velocloud/events/client/event.1684501426626.mgp7kfvg'

    • 2023-05-19 13:03:46,634 events.EventPackage failure, queing to MGD:

  • Fixed Issue 123594: For a customer enterprise site configured with a High Availability topology, when the VMware SD-WAN HA Edges are downgraded to a lower build, it may take up to an hour before the Standby Edge synchronizes with the Active Edge.

    The issue stems from the Edge's PIC watchdog timer not being rearmed during the Edge reboot process and having too short of a timeout which prevents it from triggering an additional Edge restart if the interface is stalled.

  • Fixed Issue 125336: A customer enterprise deployed using port forwarding rules may observe that traffic matching these rules is dropped by the SD-WAN Edge.

    Once the issue is encountered the behavior persists until a user toggles the Firewall status under Configure > Edge > Firewall.

    The issue is triggered by an Edge interface being down and then coming up later. The Edge marks this interface as pending, but when the interface comes up, the port designation for this interface is not changed for the inbound policy index and this results in a port mismatch which results in the Edge dropping the traffic matching this rule.

  • Fixed Issue 127443: A VMware SD-WAN Virtual Edge deployed with a 2 core configuration may experience a Dataplane Service failure, generate a core with a SIGXCPU, and restart to recover.

    On a 2 core Virtual Edge platform, when there is a bulk of events processed (redistributed) in the BGP/OSPF worker thread, events are sent to the Edge routing process clients (BGP/OSPF) and these are running as a non-RR thread and may not get enough CPU cycles to read the data at the same rate. As a result the Edge remains in a permanent retry send loop and starves the other threads, triggering failure logged as a SIGXCPU exit.

  • Fixed Issue 129311: An Operator or Partner user may observe that a VMware SD-WAN Gateway has an increasing number of stale tunnel entries and stale peer entries and in some cases that the Gateway restarts its service.

    The Gateway restarts if the number of stale tunnels and/or stale peer entries reach a critical level as a way of clearing the entries.This information can be found when looking at the Gateways page for a particular Gateway under the System Information > General screen of the Orchestrator UI.

  • Fixed Issue 130833: For a customer enterprise where a Non SD-WAN Destination via Gateway with a route based type is configured, client users may find traffic destined for the peer site is dropped during the IKE SA re-key.

    In some cases when Phase1 rekey happens, VCG doesn’t reply on new created Phase1 Dead Peer Detection (DPD) message. This can cause the tunnel to break and BGP tunnel flapping. If the peer (VMware Cloud on AWS) is configured with on-demand DPD, after a Phase1 SA re-key it sends aggressive DPD requests on the newly created Phase1 SA and the Gateway does not reply to these requests. This causes newly created Phase1 SA deletion on the peer side and the initiation of a new Phase1 SA. This will also fail since the Gateway is always the initiator.

    The workaround is to manually delete the Phase1 SA, or wait until new a Phase1 is initiated from the Gateway.

  • Fixed Issue 131029: For a customer enterprise site that uses one or more PPPoE links, when the VMware SD-WAN Edge is either deployed or upgraded to Release 5.2.0.x, the PPPoE links may fail to come back up.

    Post-upgrade the Edge process that handles PPPoE interfaces fails because a key configuration file is missing and this results in the failure of any PPPoE links on the Edge.

    On an Edge without a fix for this issue, it the Edge encounters this issue, there is no workaround beyond:

    1. Upgrading the Edge to any 5.2.2.x build. This version includes a fix for this issue and resolves the condition and prevents it from recurring. It is strongly recommended to use this method.

    2. If a customer wants to continue using a 5.2.0.x build, they can contact VMware SD-WAN Support who can correct the issue for the customer on a per-Edge basis.

  • Fixed Issue 131122: In some cases, UDP traffic matching a Business Policy rule which includes a Policy-Based NAT (PBNAT) may not be steered as expected.

    The SD-WAN Gateway may not see the QOS synchronization for certain UDP flows (for example, DNS traffic). This results in UDP packets potentially not getting the expected routing and business policy rules applied. When the particular flow consists of a single packet (like a DNS flow) the impact can be significant as the entire flow is incorrectly steered.

  • Fixed Issue 131891: For a customer enterprise site where a VMware SD-WAN Edge is deployed either as a standalone Edge or with a High Availability topology, the Edge may experience a Dataplane Service failure and restart to recover, which would result in an HA failover with HA sites.

    The issue can be encountered when the Edge (Active Edge in HA deployments) is handling a large traffic load that keeps the Deep Packet Inspection (DPI) engine busy handling HTTPs traffic featuring Server Name Identifier name strings greater than 32 bytes.

  • Fixed Issue 135544: When a customer activates an SD-WAN Edge with a factory image of 5.0.0 or newer, in some cases the Edge may become unbootable.

    When an Edge with a 5.0.0 or greater factory image is connected to the internet, the Edge checks with a VMware service (Maestro) to check if it is using the latest version. If it is not on the latest version, it initiates a factory image update. The issue arises if the user attempts to activate the Edge while it is updating its factory image and both processes concurrently running can result in the Edge becoming unbootable.

    As a workaround the user activating the Edge should first connect the Edge to the internet and leave it that way for several minutes to ensure both processes do not conflict.

  • Fixed Issue 135810: When an SD-WAN Gateway certificate is renewed the Gateway brings down and then reestablishes all IPsec tunnels that connect to SD-WAN Edges.

    This action can cause a brief disruption in customer traffic that uses the Gateway (for example, all Multi-Path traffic).

  • Fixed Issue 136992: A user can factory reset an SD-WAN Edge, even though the last factory image update attempt failed.

    An Edge factory image update can be interrupted for various reasons, (for example: losing power, or rebooting). An Edge factory reset followed by an unsuccessful factory image update will leave the Edge non-bootable.

  • Fixed Issue 138006: For a customer site deployed with a High Availability topology, the customer may observe the Standby Edge restart as a result of having high memory usage.

    If memory usage is sufficiently high, the Standby Edge defensively restarts its Dataplane Service to clear the issue. The restart can impact traffic on Enhanced HA topologies where the Standby is passing traffic. In a Standard HA topology, there is no functional impact.

    The high memory usage is the result of a memory leak which is caused by the Edge refreshing all IPsec tunnel certificates which triggers a tear down and rebuilding of all IPsec tunnels even if there is no change in the certificate and this high number of revoked certificates are not cleared from the Standby Edge's memory.

Resolved in Edge/Gateway Version R5400-20231230-GA-134893

Edge/Gateway build R5400-20231230-GA-134893 was released on 01-02-2024 and is a Hotfix build for Release 5.4.0.

This Edge/Gateway Hotfix build addresses the below critical issue since the previous updated GA build, R5400-20231108-GA-125647.

  • Fixed Issue 134893: Customers may observe that cloud/internet traffic which traverses a Gateway is failing when the connected Gateway uses version 5.4.0.

    In this situation, Edge-to-Edge traffic continues to work properly even while cloud/internet traffic fails when using the Gateway (in other words, multi-path traffic).

    SD-WAN software includes flow compatibility processing, which handles differences in ICMP processing between SD-WAN software versions prior to and after version 5.2.2. With this issue, when one or more Edges use version 5.2.1 or earlier, send ICMP traffic, and are connected to the Gateway using 5.4.0, the ICMP processing allows SD-WAN Gateway and Edge flows to be released without proper cleanup. This results in stale NAT entries on the Gateway that are never removed and causes NAT tables to reach capacity with no free entries available. Lacking free NAT entries, Edge cloud/internet traffic using an affected Gateway deployed as their primary would fail.

    As noted above this issue is only encountered if there are Edges using 5.2.1 or earlier which also send ICMP traffic while connected to a Gateway using 5.4.0. If all the Edges are using version 5.2.2 or later or do not send ICMP traffic, the issue is not encountered.

    Note:

    The fix is primarily for the SD-WAN Gateway as the field-found issues have been on this component. However, the Edge could potentially experience this issue and impact traffic for local users. As a result customers can also upgrade their Edges to the hotfix build to mitigate the risk of this issue for their sites.

Resolved in Edge Version R5400-20231108-GA-125647

Edge build R5400-20231108-GA-125647 was released on 11-14-2023 and is an update to the Edge GA build for Release 5.4.0.

This updated Edge build addresses the below critical issue since the original GA build, R5400-20231009-GA.

Important:

  • All previous Release 5.2.0 Edge builds are deprecated in favor of R5202-20231107-GA-125647.

  • Customers upgrading to 5.2.0 should only upgrade to R5202-20231107-GA-125647.

  • Customers already successfully using a 5.2.0.x Edge build do not need to upgrade their Edges to R5202-20231107-GA-125647.

  • Fixed Issue 125647: For a site deployed with an Edge model 520 or 540, when an Edge is upgraded to a 5.2.0 version, client users connected to the Edge via a LAN port may experience a total loss of connectivity.

    Rebooting the Edge 520/540 does not recover the issue nor does downgrading the Edge to an older software version once it has been upgraded to a 5.2.0 version. When the Edge's console is deactivated in the Edge Security > Console Access settings on the Firewall configuration page of the Orchestrator (which is the default configuration for any Edge), the driver that manages the LAN1 through LAN8 ports of the Edge 520 or 540 does not configure itself properly, leading to those ports not being created at all.

    On an Edge without a fix for this issue, a customer can prevent this issue from occurring and/or restore connectivity on LAN ports on an affected Edge, by doing the following: navigate to Configure > Edge/Profile > Firewall > Edge Security, and under Console Access click on Enable and Save Changes.

    Note:

    Changing this configuration requires an Edge reboot which takes ~2-3 minutes to complete. If possible, perform this change in a maintenance window.

Resolved in Edge/Gateway Version R5400-20231009-GA

Edge/Gateway build R5400-20231009-GA was released on 10-23-2023 and resolves the following issues since Edge/Gateway build R5202-20230725-GA.

Note:

Release 5.4.0 contains all Edge and Gateway fixes that are documented in the 5.2.0 Release Notes up to the 5.2.0.2 version (R5202-20230725-GA).

  • Fixed Issue 69641: For a customer enterprise that uses one or more Business Policies which include rate limits, the customer may observe packet drops on all flows, even those unrelated to the rate limited Business Policy flows and on other segments and peers.

    Setting a rate limited Business Policy and sending higher demand traffic (more than the limit) with a high number of flows results in packets from other flows being dropped (even from other segments and peers) due to the net scheduler buffer limit being hit.

    On an Enterprise whose Edges do not have a fix for this issue, the workaround would be to remove the rate limit configuration and instead reclassify the traffic matching the rule with the lowest possible value (Low, Bulk).

  • Fixed Issue 74422: In cases of High Availability, the Edge may go offline if only the Standby Edge has a WAN link which is up and has a valid IP address.

    This issue occurs when a WAN link has DHCP enabled where only the Standby Edge has a WAN link available. When the Standby WAN link receives an IP address from the DHCP server, it sends the interface details to the Active Edge. The Active Edge makes a call to add the IP address as a route, however this function does not add the route to the Linux kernel. The Edge function only adds the route to the FIB (forwarding information base). As a result, the Edge's management process throws an error as there is no route present in Linux kernel route table for the packet to exit and the site is effectively offline.

  • Fixed Issue 79598: For a customer enterprise using Zscaler with redundant tunnels, on a failover of the primary to the secondary tunnel, traffic switches back to the primary tunnel sooner than expected.

    The expected behavior is for the primary Zscaler tunnel to come up and for traffic to continue using the secondary tunnel for another 30 minutes before traffic is then steered back to the primary tunnel. This ensures that the primary tunnel is stable and less likely to go down again forcing another traffic failover. With this issue, traffic is steered to the primary tunnel in less than 30 minutes.

  • Fixed Issue 91164: A Hub Edge Hub with High Availability configured may not forward internet backhaul traffic after an HA failover.

    The Standby Edge does not set the destination route for internet backhaul flows when the backhaul flow is configured to route via a static route using a routed interface where WAN overlay is deactivated.

  • Fixed Issue 92421: When a public and private overlay are configured on the same Edge interface with different custom VLAN tags, there is a chance that the underlay routed traffic may get dropped.

    When a public and private overlay are configured on the same interface with different custom VLAN tags, the Edge may learn the ARP entries with the wrong VLAN tags, resulting in the traffic being dropped.

  • Fixed Issue 96334: On a VMware SASE Orchestrator where the IP address changes frequently, the VMware SD-WAN Edges may lose contact with the Orchestrator and be reported as offline.

    In this scenario where the Orchestrator IP address changes frequently, the Edges are responding to each Orchestrator IP address change by changing the management traffic source from a loopback interface to the GE1 port IP address, even when the Orchestrator is configured to exclusively use the loopback interface as the source. As result the Edges lose contact with the Orchestrator and while this does not impact customer traffic, this issue prevents configuration updates and monitoring from working as expected.

  • Fixed Issue 99162: If a VMware SD-WAN Edge has frequent tunnel flaps, this can result in an increase in memory consumption and potentially lead to the Edge defensively restarting to recover the memory.

    The Edge memory affected is the vc_edge_route object, which the Edge service does not clear after each instance of a tunnel being torn down and then rebuilt. Like any memory leak, if enough Edge memory is consumed by this leak, the Edge will trigger a service restart to clear the memory and this can disrupt user traffic for 10-15 seconds.

  • Fixed Issue 104776: For a customer configuring a VMware SD-WAN Edge interface for PPPoE, when the WAN Overlay settings for that interface include 802.1P Setting, the Edge does include the 802.1P PRI bits on outgoing traffic.

    The Edge does not include the configurable option for netifd to set 'EGRESS priority mappings' for PPPoE interfaces and the result is the Edge does not mark those packets with PRI.

  • Fixed Issue 104786: For a customer enterprise deployed using a Hub or Cluster Interconnect topology, the auto-rebalancing of tunnels between two interconnecting clusters does not work.

    Rebalance of interconnected clusters does not happen on an increasing cluster score or BGP flap, which should occur and can cause traffic issues because of the imbalance in tunnel usage.

  • Fixed Issue 105034: SNMP polling for a VMware SD-WAN Edge's CPU and memory always gets a zero as a response value.

    The SNMP polling for the CPU and memory as part of the Edge health stats always gets a response value as zero. As a resolution, CPU utilization is now renamed as CPU load average and memory utilization is also populated as the response.

  • Fixed Issue 106160: With an Edge configured DNS server and a next hop Gateway defined for an Edge interface for which clients query the DNS server, there is no response.

    The DNS request packet is received by the Edge DNS server as expected. However, the reply packet does a route table lookup based on an iptables connection tracking and finds the next hop Gateway IP address and resolves the MAC address. The result is the DNS reply packet will use the MAC address of the Gateway, not the sender.

  • Fixed Issue 106992: For a customer enterprise deployed using a Hub or Cluster Interconnect topology, the customer may observe that the Hub Clusters cannot form overlays between them.

    Hub Cluster members with interconnect enabled may not be able to establish overlays, because of stale Hub Cluster mapping. The only way to remediate this issue is to deactivate and then reactivated interconnect on the Hub Clusters.

  • Fixed Issue 107550: For customer enterprises where Non SD-WAN Destinations via Gateways are deployed, a user may observe some IPsec encrypted packets drop in the path.

    The current implementation uses an inner IP header Time-to-live (TTL) value and this does not match the RFC requirement. As a result, the TTL value must be constructed and if the packet originator uses a low TTL value then there is a chance that this packet will not reach its destination.

  • Fixed Issue 108111: When an Operator or Partner user tries to debug a VMware SD-WAN Gateway using the command debug.py --bgp_agg_dump, the command does not have a proper help string.

    The help string explains what arguments are taken (for example: [[v4 | v6 | all] ...]]), and with this issue that string is missing for the debug command.

  • Fixed Issue 109830: Combinations of certain PPTP (Point-to-Point Tunneling Protocol) VPN Clients and Servers may not be able to immediately re-establish a connection after it is interrupted when using 1:1 NAT with the PPTP server behind the Edge and the client in the Internet.

    This issue has been confirmed to occur with Windows Server 2016 and Windows 10 client but may occur with other versions as well. The issue is caused by the server reusing the same PPTP call-id for the new connection while the client uses a new call-id. When the server call-id is reused for the new connection, the Firewall does not identify it as such.

    Without a fix for this issue, a user can flush the stale PPTP connection from the NAT table to restore connectivity.

  • Fixed Issue 112115: A VMware SD-WAN Edge under a high CPU load may experience a Dataplane Service failure and restart to recover.

    Under high CPU conditions, multiple service failures triggered by a mutex monitor can occur due to a lower priority thread acquiring the debug ring lock. The resolution to this issue is an enhancement to the Dataplane that makes that thread both lock-free and wait-free.

  • Fixed Issue 112509: A VMware SD-WAN Edge configured to use a VNF may experience a Dataplane Service failure and restart to recover.

    The issue is traced to SKB (network buffer) handling. In some instances the SKB allocation check is missing and this can trigger the Edge service failure.

  • Fixed Issue 113877: For customers who configure BGP/GRE LAN, those using a TGW GRE will experience BGP flaps and traffic interruption on the TGW secondary tunnel in all segments when the BGP configuration for TGW GRE is modified on the Global segment.

    When customer changes a BGP configuration of TGW GRE on the global segment then the secondary tunnel in the global and other segments flap, leading to a BGP connection reset and reconvergence and traffic interruption. The BGP connection will form again, and traffic will restore.

  • Fixed Issue 114529: For LTE Edge models (510-LTE, 610-LTE), when a user does not set the CELL configurations in the Orchestrator even after activating CELL1/CELL2 and there is no default carrier and configurations selected, the Edge throws a configuration error.

    If a user activates an LTE Edge CELL1/CELL2 interface from the Configure > Edge > Device > Interface page and does not select any carrier/network from the list, this field goes to the Edge as empty and when it is read, the Edge throws an error that it is not from one of the available SPNs. This error can be seen in Events section on the Orchestrator.

  • Fixed Issue 114659: The debugging command tcpdump.sh does not work for a VMware SD-WAN Gateway.

    The Gateway's AppArmor security process blocks access to /dev/pts/* terminals. This causes the vctcpdump process to terminate and tcpdump.sh does not capture any info on stdout.

    On a Gateway without a fix for this issue, the workaround is to run the command tcpdump.sh-w to save the output to a file, as this still works.

  • Fixed Issue 114854: For a user troubleshooting a VMware SD-WAN Edge model 610 with DPDK activated, running a packet capture from the Orchestrator, or using tcpdump.sh or vctcpdump shows that the VLAN tag is missing for return traffic.

    The lack of VLAN tags on return traffic impacts a user's ability to successfully troubleshoot a network issue with any version of an Edge 610.

  • Fixed Issue 114938: When looking at Monitor > Edges > Destinations for a Customer Enterprise, a user may observe an incorrect domain name for a Destination.

    These invalid host names can fill up the Edge's DNS cache fully and may lead to Max DNS reached events, and the valid host names cannot be added after this occurs. The issue is caused by the Edge's Deep Packet Inspection (DPI) engine adding invalid host names (for example, IP address or IP address:Port) into the Edge's DNS cache.

  • Fixed Issue 114988: ICMPv6 message 'Packet Too Big' is not received either from or via a VMware SD-WAN Gateway.

    The Gateway data path consumes all ICMPv6 'Packet Too Big' messages locally. The fix ensures the Gateway sends the appropriate destination.

  • Fixed Issue 115148: When a customer deploys a Cloud Security Service with redundant tunnels (in other words, active and standby) and L7 Health Check is toggled on, if the primary CSS tunnel goes and then comes back up, the standby CSS tunnel may remain up.

    If the standby tunnel is up when L7 Health Check is toggled on and then this feature is toggled off prior to the primary CSS tunnel coming back up, the standby tunnel can remain in an up state indefinitely. The cause of the issue is that even though L7 Health Check is toggled off, the Gateway will check the state of L7 for the primary tunnel and its state will read as down, and as a result the Gateway concludes the primary tunnel is down and will keep the standby tunnel up.

    On an Edge without a fix for this issue, if a user performs a Remote Actions > Edge Service Restart, this will resolve the issue for that location.

  • Fixed Issue 115225: When a VMware SD-WAN Edge experiences a large number of tunnel flaps, this may result in an increased memory usage due to a memory leak.

    When looking in the logs, a user would observe an increase in the vc_edge_route memory object where there are numerous tunnel flaps as a result of Edge route-related memory leaks.

  • Fixed Issue 115262: Where a customer enterprise uses BGP for routing, BGP neighborship with a secondary IP address may not come up on a VMware SD-WAN Edge.

    When a user configures BGP neighbor first and then configures the corresponding secondary IP address on a VLAN interface, the BGP session does not come up because the BGP interface is not updated with the secondary IP address removal/addition on a VLAN interface.

  • Fixed Issue 115604: A VMware SD-WAN Edge or Gateway may experience a Dataplane Service failure and generate a core with an Assert in the logging.

    When an Edge or Gateway processes a corrupted packet, the software can hit an assert where actual user packet length is more than internal packet buffer. The Gateway is expected to drop this kind of packet and prevent it from being sent to the Edge, but instead processes it and this results in the service failure and restart.

  • Fixed Issue 115869: Tunnels re-establish to a VMware SD-WAN Gateway in the middle of its upgrade process.

    In a scaled environment where there are 1000s of tunnels and peers connecting to a Gateway, when a Gateway is upgraded, the traffic switches to the secondary Gateway by design to ensure a short downtime and resume traffic flow quickly. When the upgrade script is running on the Gateway being upgraded (from 4.5.1 to 5.2.0 or from 5.0.1 to 5.2.0 or 5.1.0 to 5.2.0), in the middle of the upgrade script execution, the tunnels start coming back up on the Gateway being upgraded and traffic again switches from the Secondary Gateway to the Gateway being upgraded. And then when the script ends, again the Gateway requires a reboot and once again the traffic is switched to a secondary Gateway. This can cause major customer traffic disruptions for Multipath type traffic using the Gateway.

    On a Gateway without a fix for this issue, the workaround is to move vc_proc_mon restart from the post-installation script to after the upgrade completes.

  • Fixed Issue 115904: When a user triggers a diagnostic bundle for a VMware SD-WAN Edge using the VMware SASE Orchestrator, the Edge may experience a Dataplane Service failure, generate a core, and restart to recover.

    A user can generate an Edge diagnostic bundle on the SD-WAN > Diagnostics > Diagnostic Bundle page. When this action is taken, a race condition between dns_name_cache (addition and/or delete) and the DNS name cache can occur which causes the Edge service to try and access an in use or deleted element, which triggers a service failure with a SIGSEGV or SIGBUS reason.

  • Fixed Issue 116049: The VPN State of a WAN link configured as a backup can go to a DEAD state instead of the expected STANDBY state.

    The impact to the customer is lessened because the Backup WAN link functionality (link becoming ACTIVE when other paths go down) remains unaffected. However the displayed UI status of the WAN link as DEAD can cause confusion for the customer, and if the backup WAN link is in fact down, the customer would not be able to tell through the Edge > Monitor page when this issue is experienced.

    This issue can occur where the enterprise is connected to a Partner Gateway and the BGP handoff IP address configured is not an IP address of any Edge interface in that segment. In that scenario, the Edge's backup link check messages can be dropped. As a result, the WAN links configured as backups for Edges can be marked as DEAD instead of STANDBY even though the link is up.

  • Fixed Issue 116059: For a customer enterprise site deployed with a High Availability topology where VNF's are used, connectivity to a Standby Edge VNF fails from the VNF manager present on the VNF management VLAN.

    When a VNF manager is deployed on the VNF management VLAN, a wrong MAC address entry can be learned on the Standby VNF management bridge's forwarding database (FDB) and this entry persists even after the Standby bridge ports are set to a disabled state. As a result, the Standby Edge VNF connectivity fails from the VNF manager.

  • Fixed Issue 116199: For a customer enterprise where Hub and Cluster Interconnect is configured, when the tunnel between a Spoke Edge and a Hub or Cluster goes down, the routes may not be revoked from the remote Spoke Edges.

    This issue can impact customer traffic using these stale routes and can only be temporarily resolved by re-initiating the routes.

  • Fixed Issue 116257: For a VMware SD-WAN Edge connected through a Partner Gateway where a NAT handoff is configured for a remote server, return traffic to the Edge may drop from that server.

    If the traffic is initially not encrypted from the Edge to the remote server and then updated with an encrypted flag, once the route is updated, the reverse traffic is dropped on the Edge due to a route lookup failure.

    The issue can be temporarily resolved by flushing flows on the affected Edge.

  • Fixed Issue 116368: The routing logs on a VMware SD-WAN Gateway may reach capacity and not accumulate any additional entries.

    This issue is caused by the Gateway's routing software missing the log rotation configuration, whose purpose is to rotate the routing logs prior to reaching capacity so that new log entries can be added. Without this configuration, the routing logs do not rotate and result in Operators and Partners potentially missing critical log entries for a Gateway.

  • Fixed Issue 116428: On a customer deployment where multiple segments are configured and each non-Global segment has a custom name, when running Remote Diagnostics > Ping Test, the drop down menu to choose an interface does not show the custom name for each segment.

    Instead of the custom name for each non-Global segment, the user sees a generic name with a number: Segment 1, Segment 2, and so forth. The is the result of the Edge hard-coding the segment name for each non-Global segment.

  • Fixed Issue 116827: A VMware SD-WAN Edge may experience a Dataplane Service failure and restart to recover from the condition.

    The Edge can experience this issue because of a race condition during the Edge startup that causes the Edge service to fail due to uninitialized data.

  • Fixed Issue 116894: 1:1 NAT does not work properly when the Outside IP address and Source IP address are in the same subnet.

    With this 1:1 NAT configuration the Edge changes the source port during the NAT translation and the result is traffic dropping that matches this rule for inbound traffic.

  • Fixed Issue 116916: Edge paths to peer Edges and Gateways may go down after the addition or deletion of an IPv6 kernel default route to any destination via an Edge interface which is used by the Edge's management process.

    The Edge paths can go down upon the addition or deletion of an IPv6 kernel default route involving any interface used by the Edge's management process to form paths with other nodes (that is, Edge or Gateways).

    If experiencing this issue without a fix, a user needs to remove the IPv6 route and restart the service.

  • Fixed Issue 116925: FTP traffic may be misclassified as generic TCP by the VMware SD-WAN Edge's deep packet inspection (DPI) engine.

    When encountered this has significant impact for a customer that is using a Business Policy or a Firewall Rule that is intended to match FTP traffic as the rule would not work.

    The FTP Data traffic is classified as APP_TCP instead of APP_FTP_DATA. This is because the FTP control flows are removed from the DPI engine context once the classification is done. But for the FTP data traffic to be classified correctly, flow should persist in the DPI engine.

  • Fixed Issue 117037: For a customer using a Hub/Spoke topology where multiple WAN links are used to send and receive traffic from the Spoke Edge to the Hub Edge, customers may observe lower than expected performance for traffic that is steered by Business Policies because the WAN links are not aggregating the WAN link's bandwidth.

    SD-WAN uses a counter for accounting the number of packets buffered in a resequencing queue. This counter is managed per peer and used to make sure only 4K packets are buffered per peer. Under some conditions, this counter can become negative. Prior to Release 4.2.x, when this counter became negative, the respective counter was immediately reset back to 0 after flushing the packets in the resequencing queue. However, starting in Release 4.3.x, this counter is updated automatically to ensure that the counter stays within expected bounds.

    The result of this change in behavior can cause cases where the counter accounting is incorrect and the resequencing queue can stay at a very high number to which SD-WAN reacts by flushing every single packet. This action not only prevents bandwidth aggregation but can reduce the effectiveness of flows that would otherwise be on a single link.

    On Edges without a fix for this issue, the workaround is to configure business policies that steer matching traffic to a single mandatory link.

  • Fixed Issue 117314: When an ICMP flow already exists between a Source and Destination IP address pair, a Firewall rule that uses an Object Group/Service Group (type and code) to filter ICMP packets may not work.

    As part of a revision to Firewall functionality, the changes introduced for caching ICMP type and code were reverted, and this impacted Firewall rules which used a Service Group with an ICMP type and code (for example, ICMP Redirect Type 5 and Code 0). If a flow is already present between a source and destination IP address, then ICMP traffic that should match the rules for this flow will not be honored, and only the first packets for a session will match the Firewall rules. The issue impacts either IPv4 or IPv6 ICMP flows.

    Flushing flows to create new ICMP flows will temporarily remediate the issue.

  • Fixed Issue 117320: For a customer where Stateful Firewall is activated and Syslog turned on, Syslog messages for traffic that matches a LAN side NAT rule does not include the source IP address.

    A fully complete Syslog message for any traffic is expected to include the source IP address, especially for traffic being NAT'd.

  • Fixed Issue 117831: The VMware SD-WAN Gateway's default firewall rules prevent the Linux Azure agent from communicating with the Azure fabric after SD-WAN service startup.

    This does not affect SD-WAN functionality but some metrics may be missing from the Azure Portal for Gateway's VM.

    The Linux Azure agent requires outbound communication over ports 80/TCP and 32526/TCP with WireServer (168.63.129.16). After the Gateway service starts up, port 32526 is blocked.

  • Fixed Issue 117838: A VMware SD-WAN Edge may experience a Dataplane Service failure, generate a core, and restart to recover.

    The issue can be encountered when the Edge receives an ike packet with a mismatched version when compared to ike_ds. The is sue is caused when the Edge service is processing mismatched version packets and takes a mutex lock for the sake of modifying a cookie value in ike_ds, but if a version mismatch happens the Edge triggers a security association (SA) deletion which again tries to take mutex on the same ike_ds. The result is an Edge deadlock and service failure with resulting restart.

  • Fixed Issue 117943: On a VMware SD-WAN Edge with Wi-Fi capability, the automatic channel selection for some countries can result in the Edge choosing channels which result in either poor Wi-Fi performance or even complete failure of Wi-Fi to come up.

    In some countries like Great Britain, Wi-Fi takes a long time to come up when configured for the 5GHz band (or can even fail to come up). The automatic channel selection for the 5GHz band can end up selecting inappropriate channels in certain countries - either very low-power channels, or channels that require radar detection (which can delay or fail the startup).

    On an Edge without a fix for this issue, when selecting the 5GHz band in a European country, configure the radio channel to either 36, 40, 44 or 48 explicitly (instead of leaving it as "auto").

  • Fixed Issue 118097: An Operator or Partner user when debugging a VMware SD-WAN Gateway may find that the command debug.py --path command does not return a result.

    The issue is caused by an unhandled key when a transient tunnel is present, and this breaks the debug.py --path command while the Gateway is processing transient paths.

  • Fixed Issue 118348: For a customer enterprise site deployed with an Enhanced High Availability topology, if a user activates DHCP on a VMware SD-WAN Edge subinterface, the HA Edge may experience a Dataplane Service player, generate a core, and restart to recover.

    If this occurs on the Active Edge, this will trigger an HA failover and promote the Standby. If this occurs on the Standby, there is no failover but the traffic using the Standby Edge's WAN link(s) will be briefly disrupted.

  • Fixed Issue 118436: DNS traffic to an underlay DNS server might not work.

    The issue can be encountered when a VMware SD-WAN Edge's routed interface is configured without a Gateway and DHCPv6 with interface IPv6 as a DNS server IP address but without a IPv6 default route on a Partner Gateway. In this configuration, the DNS response from the underlay would be dropped by the Edge.

  • Fixed Issue 118591: For a customer enterprise site using an Enhanced High Availability topology, the Standby HA Edge may experience frequent interface flaps.

    In an Enhanced HA deployment, when a high number of flows are sent or a high number of routes installed, the Standby Edge interface state will from UP to DOWN and then back to UP.

  • Fixed Issue 119010: On the VMware SD-WAN Edge models 520 and 540, the Edge may not forward traffic from a VLAN located on LAN ports 1-4 to a VLAN located on LAN ports 5-8, and vice-versa.

    The Edge models 520 and 540 have two LAN NIC cards, each with a bank of four ports for a total of 8 LAN ports. When a LAN is configured for a LAN port on the first bank of ports and a different VLAN configured for a LAN port on the second bank of ports, the Edge does not handle this traffic properly and it is dropped.

  • Fixed Issue 119033: During a startup, the VMware SD-WAN Gateway may experience a Dataplane Service failure and need to restart again to recover.

    Details of NAT port allocations are stored in a shared memory segment outside of the Gateway service process (managed by the NAD process) by design. This is done to ensure the Gateway service can quickly reinitialize its NAT table when the process restarts. However, it is possible for this persistent state to get corrupted, causing the Gateway service to fail immediately after restarting.

    On a Gateway without a fix for this issue, flushing the NAT table or rebooting the OS can prevent this issue from occurring.

  • Fixed Issue 119466: For a customer enterprise site deployed with a High Availability topology, traffic matching a Many:1 NAT rule may fail after an HA failover.

    When this issue is encountered, LAN side NAT entries are not synchronized to the Standby Edge. Since Many:1 NAT also involves port address translation (PAT), LAN side NAT entries cannot be created based on the configuration and need to be synchronized to prevent traffic disruption on an HA failover.

  • Fixed Issue 119544: When ICMP Echo Response is turned off on a VMware SD-WAN Edge's loopback interface, L7 health check will fail with the result that the Edge will trigger a teardown of a CSS tunnel where L7 health check is configured.

    In addition, when the management traffic goes direct, the Edge to Orchestrator communication is also lost.

    When the Edge tries to send a L7 health check request (HTTP SYN packet) it will reach the loopback interface and since ICMP Echo Response is turned off, this results in the dropping of HTTP packets. When L7 health check does not get an ACK for the SYN packet it has sent the L7 health check will fail, and it will lead to a teardown of CSS tunnel.

    Similarly, when the Edge tries to send HTTPS packets towards the Orchestrator, it will reach the loopback interface and since ICMP Echo Response is turned off, it will result in the dropping of HTTPS ACK packets.

  • Fixed Issue 119811: A customer may observe that in their Events list there are multiple MGD_WEBSOCKET_INIT and MGD_WEBSOCKET_CLOSE Edge Events per day.

    There can be multiple MGD_WEBSOCKET_INIT and MGD_WEBSOCKET_CLOSE events shown in the Orchestrator's Event list without any customer actions.

    These event messages are spurious and can be safely ignored as they carry an "Info" level of severity.

  • Fixed Issue 120620: For a customer enterprise site where the SD-WAN Edge is configured as a DHCPv6 relay or server, client users may experience latency for packets that use this process.

    There can be a delay in the SD-WAN Edge sending DHCPv6 Router Advertisement (RA) related information. The Edge does not respond to Router Solicitations (RS) but only sends out periodic RAs and this causes the delay in sending an RA when the Edge is configured as a DHCPv6 relay or DHCPv6 server.

    For a site experiencing this issue without a fix, as a temporary measure, a low advertisement interval can be manually configured on the Edge in /var/etc/radvd.conf. For each interface section, MaxRtrAdvInterval and MinRtrAdvInterval should be set to very small values in seconds. This will ensure that the Edge sends RAs more frequently to compensate for dropped RS messages. However, manual modification of configuration files is not recommended, as any configuration change would be overwritten later and the preferred solution is to upgrade the Edge to a build with the fix included.

  • Fixed Issue 120940: If there are multiple routes for the same prefix in BGP from different neighbors in the same internal VRF (like a VRF created for Non SD-WAN Destinations via Gateway BGP sessions), the same neighbor IP is updated for all the routes.

    The issue can be observed the SD-WAN Gateway using the commands debug.py --routes and debug.py --bgp_view outputs, and is caused by a Gateway's routing process not updating the neighbor IP (source) properly.

  • Fixed Issue 121024: Remote Access Service (RAS) traffic fails when a Business Policy matching internet traffic is configured on a VMware SD-WAN Edge.

    When a Business Policy matching internet traffic is configured on an Edge and the action is to streer this traffic Direct, then for any Remote Access Service traffic that reaches this Edge, the return traffic matches this Business Policy and it is dropped on the Edge.

    The only workaround is to configure a more specific Business Policy matching RAS subnets and set the action to send it Multipath (via the Gateway).

  • Fixed Issue 121338: For a site where a WAN link is configured to be a Hot Standby, the VMware SD-WAN Edge includes that link as part of the available bandwidth.

    A Hot Standby WAN Link is by design idle and should not be included in an Edge's available bandwidth. Because the Hot Standby is included, the Edge makes inaccurate calculations for total available bandwidth and can result in packet loss.

  • Fixed Issue 121513: For a customer enterprise using a Partner Gateway where that Gateway has the "Secure BGP Routes" option activated, client users may observe traffic quality issues.

    Traffic may be dropped at the Edges when traffic is initiated with a BGP peer IP address as Source behind a Partner Gateway when Secure BGP Routes are configured. This is because when traffic is initiated from a BGP endpoint as Source, it creates an unsecured flow in the Gateway, because the source route will be of type BGP-Peer, for which secure setting handling is not done. However, if the source route lookup at the Edges returns a secure route, there will be a mismatch in the secure setting of the incoming traffic and route lookup. This will result in source route lookup failure at the Edges.

  • Fixed Issue 121815: For a customer enterprise site deployed with a High Availability topology and which uses VNF's, when a Standby Edge has a LAN interface up and a VNF down, and the Active Edge has a LAN interface down and a VNF up, there is no HA failover even the Standby Edge has a LAN port up.

    This issue stems from the Edge including the VNF state as part of the LAN count in the HA heartbeat. A VNF up would count as an additional LAN and results in the listed symptoms.

    The issue is resolved by decoupling any VNF states from the LAN count so that HA failover decisions can be taken properly. With this change, the Edge gives higher priority to a VNF when it has basic WAN and LAN connectivity up. In other words, if the Active Edge has a VNF down and the Standby Edge has a VNF up, the Standby Edge would be promoted to Active if it has at least 1 WAN and 1 LAN, and this is irrespective of the LAN/WAN count on the Active Edge.

  • Fixed Issue 121998: For a customer using the Stateful Firewall in a Hub/Spoke topology, traffic that matches a firewall rule configured for Spoke-to-Hub traffic where the rule includes a source VLAN may be dropped.

    When there is an application classification, business policy table, or firewall policy table version change, SD-WAN performs a firewall lookup for flows on its next packet. Due to a timing issue, that packet could be one from the management traffic (VCMP) side. As a result, during a firewall policy lookup key creation, SD-WAN swaps the Spoke Edge VLAN with the Hub Edge VLAN, and this leads to not matching the rule and dropping that traffic.

    For an Edge without a fix for this issue, a customer can change the Source from an Edge VLAN to 'Any'.

  • Fixed Issue 122029: A Cloud Security Service (commonly, Zscaler) deployed with GRE Automation where the VMware SD-WAN Edge uses a 5.2.x release does not work.

    This issue is caused by both the local IP address and public IP addresses not being sent and this information is required for the Orchestrator to send a configuration state of the tunnel down to the Edge. The tunnel needs to be in a pending state for the Orchestrator to send the automated GRE configuration.

    This issue is limited to 5.2.x Edges and a user could work around this only by downgrading the Edge to a 5.1.x or earlier release and bring the tunnel up and then upgrade to 5.2+ release.

  • Fixed Issue 122528: For a customer enterprise which uses WAN static routes with ICMP probes configured, the ICMP probes may stop functioning on multiple VMware SD-WAN Edges at once with all traffic using those routes dropping.

    Each Edge has an ICMP probe sequence counter with a maximum number of 65535 iterations. When this counter rolls over after 65535 iterations, the probes fail.

    On an Edge without a fix for this issue, the workaround is to remove the ICMP probe, restart the Edge service, and then restore the probe.

  • Fixed Issue 123144: When on the Orchestrator UI page Monitor > Edge > Destinations, the page shows invalid Destination domain names.

    The Edge sends invalid Destination names like IP:port that are shown on the Orchestrator's UI due to a validation miss in the DNS hostnames.

  • Fixed Issue 123237: For a VMware SD-WAN Edge running Release 5.2.x and where the interfaces are configured with IPv6 only, the Diagnostics > Remote Diagnostics page does not load.

    When IPv6 only Edge interfaces are configured with IPv4 settings deactivated, an exception is thrown in a key script which causes the Diagnostics > Remote Diagnostics page to not load.

    The workaround is to activate IPv4 settings with dummy configuration.

  • Fixed Issue 123956: A user cannot access the Local UI on a VMware SD-WAN Edge using a 5.2.x Release.

    The browser page does not load with a 404 Error. The issue is the result of exceptions thrown in both Remote Diagnostics and Local UI scripts.

  • Fixed Issue 124106: When LAN side NAT is configured for Many:1 translations where Port Address Translations (PAT) is used, traffic initiated from the opposite direction allows unexpected access to fixed addresses based on the outside mask and original IP address.

    LAN Side NAT rules allow connections to be initiated in both directions for Many:1 rules without a clear way to prevent traffic in the 1:many direction. 1:Many translations will now be blocked and users must create explicit 1:1 NAT rules to enable the traffic.

    This issue is fully covered in Important Notes: LAN-Side NAT Behavioral Change.

  • Fixed Issue 124162: When a user takes a packet capture on a VMware SD-WAN Edge interface, they may see a packet that appears to be corrupted.

    There is no actual packet corruption, the packet only appears corrupted in the PCAP file. This issue is due to a defect in the way the Edge writes packets to the packet capture interface, VLAN-tagged packets may be written incorrectly and will show up as a corrupted packet (invalid ether-type) in the PCAP file.

  • Fixed Issue 124263: A customer may observe high CPU utilization on a VMware SD-WAN Edge while handling IPv6 neighbor discovery (ND) and L2 encapsulation packets.

    Troubleshooting the Edge points to api - vc_ip6_host_addr_to_network_addr as consuming high CPU.

  • Fixed Issue 124357: A VMware SD-WAN Edge may experience a Dataplane Service failure and restart as a result.

    When a business policy is configured with internet backhaul and a 3000 byte or larger is sent from the LAN side the Edge process asserts for big packet sizes which triggers a service failure.

  • Fixed Issue 125035: When a customer deploys a Fortinet 6.4.x VNF, the Edge's Management IP address is not accessible.

    Fortinet made changes in 6.4.x which results in FortiLink and transparent mode not working together. SD-WAN does not use FortiLink and as a result the resolution to this issue is to disable FortiLink before setting the transparent mode while deploying the VNF.

  • Fixed Issue 125421: A customer may observe that the WAN links on a VMware SD-WAN Edge are intermittently showing as down and then up on the Monitoring and Events page of the VMware SASE Orchestrator UI, with the potential the Edge may become unresponsive and fail to pass traffic until it is manually rebooted, or the Edge can experience a Dataplane Service failure and restart.

    This is an Edge memory leak issue that is encountered when the Edge Dataplane Service cannot open shared memory, causing stale PIs. This in turn causes open file descriptor exhaustion which will initially impact WAN links. However, if this issue is sufficiently advanced and results in Edge memory exhaustion the Edge can:

    1. Become unresponsive and unreachable through the Orchestrator, which requires an on-site reboot/power cycle.

    2. Can trigger an Edge service failure with a core file generated, with the Edge restarting to recover.

  • Fixed Issue 125487: Edge-to-Edge traffic flow may be disrupted by an ARP resolution issue.

    When encountering this issue, the Edge is forwarding the ARP request to the next hop IP address using the primary interface IP address instead of the subinterface IP address. The issue is triggered during flow creation when a non-connected route is used to reach the destination, and if the Edge's subinterface is used for that connectivity, the Edge does not properly fill the source IP address for the subinterface case.

  • Fixed Issue 126304: 1:1 NAT rules whose source translation overlaps with Many:1 NAT rules or other 1:1 NAT rules may lead to port collisions or failed translations.

    To correct this behavior, when a 1:1 NAT rule's source translation overlaps another rule, port address translation (PAT) is activated for the 1:1 NAT rule as well.

  • Fixed Issue 126458: For a customer site deployed with a High Availability topology where the HA Edges are Edge models 520/540, the customer may observe multiple HA failovers that are the result of an Active/Active state.

    The condition is triggered on HA configured 520/540 Edges, when the number of concurrent flows exceeds 300K.

    On Edge 520/540 HA Edges without a fix for this issue, the workaround is to increase the HA failover time from 700ms to 7000ms on the Configure > Edge > Device page as this will reduce the change of an Active/Active state.

  • Fixed Issue 127127: A VMware SD-WAN Edge does not learn routes from Hub Edges when the VMware SD-WAN Gateways are upgraded to Release 5.1.x or higher.

    When an Edge is configured with Branch to Branch via Hubs with only the same Edge in the list and the Gateway is running 5.1.x and above versions, the routes are not advertised from the Gateway to the Edge.

    The only workaround is to remove the Edge from the Branch to Branch via Hubs configuration.

  • Fixed Issue 127403: On the Test & Troubleshoot > Remote Diagnostics page of the Orchestrator UI, when running the remote diagnostic Troubleshoot OSPF - List OSPF Redistributed Routes or TroubleshootBGP - List BGP Redistributed Routes, the test returns an error with no data.

    After running either diagnostic the user observes an error: Error reading data for test.

  • Fixed Issue 128377: For a customer enterprise using BGP for routing, the customer may observe that user traffic is disrupted due to a failure of BGP.

    The BGP process can crash during clean up due to an invalid memory access of self_mac_hash.

    As part of cleaning up the bgp_master, the self_mac_hash is already cleaned up. However, while processing the message after the clean up has already occurred, BGP accesses the deleted hash causing the failure.

  • Fixed Issue 128741: A VMware SD-WAN Gateway may experience a Dataplane Service failure, generate a core file, and restart to recover.

    When a valid management packet arrives unexpectedly on a Non SD-WAN Destination IPsec tunnel or a GENEVE interface, the Gateway may run into an error while processing that packet and this triggers the Gateway service process to fail.

Orchestrator Resolved Issues

Resolved in Orchestrator Version R5409-20240918-GA

Orchestrator build R5409-20240918-GA was released on 09-30-2024 and is the 9th Orchestrator rollup for Release 5.4.0.

Orchestrator build 5.4.0.9 adds support for provisioning and configuring Edge models 710-5G, 720, and 740 on the UI.

Note:

These new Edge models must be deployed with a 5.2.4.x Edge build as they do not have 5.4.0.x Edge software support.

This Orchestrator rollup build also addresses the below critical issues since the 8th rollup build, R5408-20240725-GA.

  • Fixed Issue 139581: The Orchestrator allows a user to create or update a Business Policy rule or a BGP route filter and use a name already used by another rule or filter.

    The Orchestrator lacks a validation check for duplicate Business Policy rule and BGP route filter names. This validation check would be expected to throw an error that the name the user configured is already in use.

  • Fixed Issue 143506: For a Partner or Operator user on the Customer & Partners > Manager Partner Customers > Manage Customers screen of the Orchestrator UI, the customer settings "Edge Config Updates Enabled" and "Edge Config Updates Enabled on Upgrade" may display an incorrect value.

    When encountering the error, a user would observe that both settings always show "Not Enabled" when they are in fact both "Enabled" as can be observed in the Service Settings > Edge Management for that same customer.

  • Fixed Issue 148179: For an Edge where the Firewall is enabled, if navigates to Diagnostics > Remote Diagnostics and runs "List Active Firewall Sessions", they may observe that the TCP State value "SYN_RECEIVED" overlaps the value show for Bytes Sent in the adjoining column.

    Because the TCP State value Syn Received overlaps the numberical value for Bytes Sent, the user cannot see the Bytes Sent value.

  • Fixed Issue 148274: A user may observe gaps in charts on the Monitor > Edge section of the Orchestrator UI.

    When an Orchestrator's database receives a high volume of simultaneous queries, it does not properly queue requests beyond the maximum concurrent query limit.

    As a consequence, queries are rejected, leading to client failures. Clients without proper retry mechanisms or error handling capabilities fail to correctly handle this failure, resulting in gaps in visualizations, like Monitor > Edge charts (QoE, Links, Flows, and so forth).

  • Fixed Issue 148873: When an Operator or Partner upgrades an Orchestrator to 5.4.0.x build, the user may observe that the upgrade fails.

    The failure would show the following symptoms:

    The package python3-update-manager was not upgraded, and its installation failed.
WARNING - =========== VCO upgrade failed ===========

    Starting with Release 4.0.0, a breaking change was introduced in the python3-update-manager which prevents its upgrade in subsequent Orchestrator releases. This issue becomes more prominent with Release 5.x Orchestrator versions due to additional sanity checks performed by the Orchestrator. As a result, users can encounter the warning message: VCO upgrade failed during the upgrade process and point to the python3-update-manager package. Despite this warning, the overall Orchestrator upgrade completes successfully, and there is no functional impact on the system beyond this package issue.

Resolved in Orchestrator Version R5408-20240725-GA

Orchestrator build R5408-20240725-GA was released on 07-29-2024 and is the 8th Orchestrator rollup for Release 5.4.0.

This Orchestrator rollup build R5408-20240725-GA addresses the below critical issues since the 7th rollup build, R5407-20240510-GA.

  • Fixed Issue 118684: For customers using the monitoring/getEnterpriseEdgeStatus API where ClickHouse is enabled to store statistics data, the customer may observe data discrepancies for Edge statistics.

    When this issue is encountered, the API only returns the edgeLogicalId instead of returning both the edgeLogicalId and the edgeId. This behavior results in data discrepancies and user confusion as they rely on the retrieval of edge IDs for various use cases within their environments. The resolution to the issue ensures that the getEnterpriseEdgeStatus API returns the edgeLogicalId and the edgeId, bringing alignment with our documentation and providing a more predictable user experience.

  • Fixed Issue 133642: On the Configure > Edge > Device > Connectivity > VLAN page of the UI, the Edge's VLAN is automatically set to Override by default, and a user cannot deactivate this setting.

    The expected behavior is that if a user creates a VLAN at the Profile level, that the Edge Override setting is deactivated by default and a user must elect to activate Edge Override on a per Edge basis.

  • Fixed Issue 145134: For customer enterprises using Zero Touch Provisioning (ZTP) and who deploy one or more sites with a High Availability topology, users may observe deployed HA Edge serial numbers in the ZTP Available inventory list.

    When an Orchestrator requests the Edge inventory from the Maestro service, it is expected to exclude Edges that are being used in HA pairs. The issue is caused by the Orchestrator not checking if the incoming serial numbers are specifically used as HA pair serial numbers and results in the Orchestrator hiding some of HA serial numbers, but not all of them.

  • Fixed Issue 145783: On some Orchestrators that use customized branding, users may observe that control buttons at the top of the page are difficult to locate.

    Some branded Orchestrators have a header color scheme that matches the colors of the top-right controls (Help, User Information, and Menu) on the Global Settings page, with the result that the buttons blend with the Orchestrator coloring and become difficult to locate.

    This issue does not affect VMware cloud hosted Orchestrators, which use a default color scheme whose header colors contrast with the interface buttons.

  • Fixed Issue 147155: Customers deployed on an Orchestrator using Version 5.4.0.7 may observe that Edge WAN links do not show an ISP name under Monitor > Edge > Network Overview.

    Recently the Orchestrator software migrated to a different method of establishing geolocation for an Edge. The issue is that this new method did not include the ISP information for the WAN link(s) an Edge is using which are needed for ISP mapping under Network Overview > Links. As a result, the WAN links lacked ISP information.

Resolved in Orchestrator Version R5407-20240510-GA

Orchestrator build R5407-20240510-GA was released on 05-14-2024 and is the 7th Orchestrator rollup for Release 5.4.0.

This Orchestrator rollup build addresses the below critical issues since the 6th rollup build, R5406-20240323-GA.

  • Fixed Issue 63453: An Edge WAN link whose speed is manually set to a high value may show a different value when looking at the Edge > Monitor > Overview and Links pages of the Orchestrator UI.

    For example, if a link is manually set with a bandwidth at 10 Gbps, the UI shows a value of 1.41 Gbps on a Monitor page for that Edge. In addition, if the link is set to automatically detected and monitoring is set to Live Mode, the correct value is displayed. The issue is the result of the Orchestrator code not being designed to handle a manually entered bandwidth value above 4.2 Gbps.

  • Fixed Issue 123962: A user may observe that after downloading a custom role package from the Orchestrator, when the user attempts to upload this package using the 'Upload Permission' option to another Orchestrator or under a different name, the upload is not successful.

    When this issue is encountered, the user would also observe the error "The scope specified in the package is not accessible to the user", and the new permissions never appear in the Service Permissions list at the Customer level. The issue is the result of the Orchestrator failing to open the uploaded custom role package.

  • Fixed Issue 138712: On a customer enterprise which uses a Hub/Spoke topology and where Internet Backhaul is configured for the Hub Edges, a user may not observe the option to select Backhaul Hubs in the drop down list of options when configuring a Business Policy rule.

    To show the Backhaul Hub Edges in the drop-down, the Orchestrator uses the name of the object, which is unique to each Backhaul Hub in mapping with the current rule. The issue is triggered if the Hub Edge name is updated in the configuration module.

  • Fixed Issue 139409: For a customer enterprise deployed on an Orchestrator using 5.4.0.x software where the customer deploys one or more Cloud Security Services (CSS), if the customer makes a change to a CSS on the peer side while the Orchestrator is being upgraded, an Edge in their enterprise may experience multiple Dataplane Service failures in succession and, after the third such failure, stop passing traffic until the Edge is manually restarted by the user.

    When this issue is encountered, the Orchestrator creates more than one CSS creation automation action for a single Edge WAN link, and this erroneously creates multiple CSS tunnels. The Edge receives this wrong data, which triggers an exception that causes the Edge service to fail. The nature of the issue means that even after the Edge recovers from the initial failure, the Orchestrator will continue to provide the erroneous tunnel information, causing additional failures and, on the third such failure in succession, the Edge defensively stops trying to recover which results in all customer traffic dropping.

    On an Orchestrator without a fix for this issue, the user must clear the erroneous CSS tunnel information the Orchestrator is generating to remediate the issue, and this is done by disabling all CSS instances for that Edge, wait for the tunnels to be deleted, and then re-enable all CSS instances.

  • Fixed Issue 139548: In rare instances, a customer enterprise may be assigned two Super Gateways after a reboot or power cycle which can result in tunnels not coming up and customer traffic not passing through the Edges in the enterprise.

    Some enterprises unexpectedly have two Gateways assigned the role of Super Gateway in the Orchestrator database. Normally, this inconsistency is not propagated to the Edges, and does not trigger any issues on the Edge dataplane side. However, in some rare cases, when combined with a configuration change that triggers a Super Gateway recalculation as some other changes that also trigger control plane updates, the inconsistency in the Orchestrator database may be propagated to the Edges. This results in different Edges getting different Super Gateways, which prevents tunnels from being created and traffic disruption. The fix ensures that the Orchestrator cannot assign two different Super Gateways to two different Edges of the same enterprise.

  • Fixed Issue 141617: When a customer enterprise Profile is updated, on the Monitor > Events page the "Profile updated" Event Detail has the Principle field showing as "unknown".

    For the "Profile updated" Event, a user should see when clicking on the Event to open the Event Detail box, the user should observe a value for the "principle" field. With this issue that value is always "unknown" and prevents the user from knowing what aspect of the Profile was updated.

  • Fixed Issue 142230: For a customer enterprise which deploys Non SD-WAN Destinations (NSD) via Gateway and uses redundant tunnels, a user may find that they cannot edit the NSD tunnel IP addresses on the Orchestrator UI.

    With this issue, when the user sets a new IP address for the redundant tunnel, the change is accepted by the UI, but then it displays the same IP address configured for the primary tunnel.

  • Fixed Issue 144759: On the Global Settings > Customer Configuration page of the Orchestrator UI, a user may observe that the option to configure Cloud Web Security is missing.

    The Cloud Web Security service should be visible for existing customers on this page.

Resolved in Orchestrator Version R5406-20240323-GA

Orchestrator build R5406-20240323-GA was released on 03-28-2024 and is the 6th Orchestrator rollup for Release 5.4.0.

This Orchestrator rollup build addresses the below critical issues since the 5th rollup build, R5405-20240307-GA.

  • Fixed Issue 115570: For a customer enterprise site configured for a High Availability topology, if the HA Edge's HA Interface is changed from the default location and then later downgrades the Edge software to a version that does not support this feature, the interface remains fixed to the port previously configured and cannot be changed.

    In the scenario the HA Edge was running an Edge version which supported configurable HA Interfaces (5.2.x or later) and then was downgraded to Edge version 5.1.x or earlier. The default HA interface is GE1 for all models except the Edge 5x0 models. Not only is the HA interface now fixed to the changed interface, the user cannot edit that interface.

  • Fixed Issue 130943: A customer may observe that on the Monitor > Links page for an Edge that statistics are missing

    LINK_DEAD QoE events are not inserted when the link's internalId is passed by the Edge instead of logicalId. When this occurs, the pushLinkQualityEvents API schema fails validation. This would be observed in the Orchestrator logs.

  • Fixed Issue 136085: On the Administration > User Management > Privileges page of the Orchestrator UI, if a user creates a Privilege Bundle where the Enterprise Superuser or Standard Administrators do not have the privilege to create Profile Device Authentication Settings, these settings are not applied either user type can continue to create authentication services.

    This impacts a user's ability on the Configure > Profile > Device Settings > Edge Services > Authentication (for example, RADIUS Server settings). The expected result is the Enterprise Superuser or Standard Administrator should not be able to create or change settings here, but they still can.

  • Fixed Issue 136210: An Operator user with a Standard Administrator role where their privilege to read Profile Cloud VPN status is removed can still read that parameter.

    The Standard Administrator can not only read the status of Cloud VPN in the Configure > Profile > Device section but can also edit the parameter despite having that privilege removed under Administration > User Management > Service Permission.

  • Fixed Issue 138417: Users cannot access the Orchesrator's microservices microfrontend.

    When an Operator Superuser updates the microservices redirection URL System Property: xxxRedirectionUrl twice quickly in succession, the change is not reflected in the Orchesrtrator's nginx configurations.

    Without a fix, the Operator Superuser needs to manually update the nginx configuration file with the correct redirection URL.

  • Fixed Issue 139467: A user may observe that the Diagnostics > Remote Diagnostics page for any SD-WAN Edge does not load after a user clicks "Reconnect".

    The issue is caused when a Remote Diagnostics page for an Edge times out and then the user clicks Reconnect.

    Note:

    While the symptoms for this issue match previous Fixed Issue #134911 (fixed in Orchestrator rollup build 5.4.0.4), the root cause and the matching fix differ for #139467.

  • Fixed Issue 140063: A user may not be able to remove a segment from a profile on the Orchestrator UI.

    The only way to remove the segment from a profile is to switch to another segment in the dropdown menu where the user has the option to rmeove the original segment without an error.

  • Fixed Issue 140322: When attempting to create a new profile on a customer enterprise, the user may find the effort times out on the Orchestrator UI.

    Even though the UI time's out the request for a new enterprise profile, the Orchestrator's backend does complete the API request which would be revealed if refreshing the browser ~120 seconds after the timeout. This issue is most likely encountered for an enterprise which uses a large number of Cloud Security Service site objects or generally where there is a feature that creates network_service enterprise objects.

  • Fixed Issue 140390: If a user clones and edits an existing BGP Filter, this action also modifies the original rule.

    The Orchestrator UI modifies and saves the original BGP filter rule with no notice to the user and this is expected to cause customer traffic disruption until it is corrected.

  • Fixed Issue 141535: On the Configure > Profile or Edge > Device > Telemetry section of the Orchestrator UI, a user whose role has had the Update privilege for Visibility Mode denied can still configure this parameter.

    The Orchestrator UI is not applying the change to the user role privileges under Administration > User Management > Privileges.

  • Fixed Issue 141886: A customer may not be able to view their enterprise's Firewall Logs on the Orchestrator UI.

    If the customer has Hosted Firewall Logging enabled on the Orchestrator and navigates to the Monitor > Firewall Logs page on the Orchestrator UI, the page may be empty due to an issue with the Orchestrator parsing these log types.

Resolved in Orchestrator Version R5405-20240307-GA

Orchestrator build RR5405-20240307-GA was released on 03-08-2024 and is the 5th Orchestrator rollup for Release 5.4.0.

This Orchestrator rollup build addresses the below critical issues since the 4th rollup build, R5404-20240228-GA.

  • Fixed Issue 129239: For a large scale enterprise customer (~1000 or more Edges), on the Configure > Profile page of the Orchestrator UI, when a user edits any setting at the Profile level and attempts to Save, they may observe that the API call times out and is not successful.

    The issue is the result of other Orchestrator database queries taking too long to complete and causing the attempt to save the new Profile settings to time out.When this issue is encountered, the Orchestrator UI displays a red error banner on the browser page that reads "configuration/updateConfigurationModule time out".

  • Fixed Issue 134498: For a customer enterprise where a Cloud Security Service (CSS) or Non SD-WAN Destination via Edge is used, if a user removes a non-global segment from a profile, the CSS/NSD via Edge tunnels will be deleted from all other segments, including the global segment.

    A common example of a CSS or NSD via Edge involves Zscaler and in this issue when all tunnels are deleted by the Orchestrator, the result is all traffic dropping that matches the rule(s) to be backhauled to the CSS.

    On an Orchestrator without a fix for this issue, the user must first disable and then reenable the CSS or NSD via Edge.

  • Fixed Issue 135804: For a customer enterprise where Two Factor Authentication (2FA) is used, if an enterprise user logs in and then logs out, when logging in the second time, the Orchestrator throws an error and prevents the user from logging back in.

    The Orchestrator displays an "Unknown Error" message which offers no information as to why the login attempt failed. The issue is the result of the UI not sending the necessary session cookie for the re-login.

  • Fixed Issue 138932: An Operator Superuser may observe that when they click on the Monitor > Customer page for any customer enterprise, that the page is stuck in a loading state but never actually loads.

    The Manage Customer page is also experiencing the same issues and in either instance a user would observe API errors when looking at the browser's console for that page.

  • Fixed Issue 139966: For a customer enterprise configured to use multiple segments, one or more cloud security services (CSS), and which has Cloud VPN toggled on, if Cloud VPN is disabled on any segment, all CSS tunnels are deleted for all segments with traffic steered to those tunnels being dropped.

    This issue is limited to a scenario where a single Edge is assigned to a profile with multiple segments. In that scenario, when Cloud VPN is toggled off any used segment, the Orchestrator deletes the Pre-Shared Keys (PSK) and FQDN's for all CSS instances, which results in the deletion of all tunnels.

Resolved in Orchestrator Version R5404-20240228-GA

Orchestrator build R5404-20240228-GA was released on 02-29-2024 and is the 4th Orchestrator rollup for Release 5.4.0.

This Orchestrator rollup build addresses the below critical issues since the 3rd rollup build, R5403-20240126-GA.

  • Fixed Issue 112612: The Firewall logs the Orchestrator uploads to the UI are missing several data points.

    The firewall logs are missing data in the following columns on the Firewall Log UI: Action, Duration Seconds, Bytes Sent, Bytes Received, Reason, and Application Protocol.

  • Fixed Issue 128753: When a customer configures a subinterface that uses DHCP for addressing and creates a user-defined WAN overlay, the user is unable to save the configuration without configuring the Source and Next-Hop IP addresses.

    If a user does try create another WAN overlay for the same interface and specifies a different VLAN, the Orchestrator triggers an error message that reads: "Attaching multiple WAN links to interface requires Source IP and Next Hop IP on extra links [Interface Name]".

  • Fixed Issue 129694: If a user configures the BGP keep alive time as blank (no value entered), the Orchestrator UI will not send the configuration.

    As a result, the neighbor status will be configured with a keep alive and hold time 0 and this value means keep alive cannot work. After an Edge restart, the peer device remains up but the Edge would continuously fail to connect via BGP.

  • Fixed Issue 131496: For an Orchestrator configured with a Disaster Recovery (DR) topology, if the Standby Orchestrator is promoted to Active, the UI does not properly load the Operator Events page.

    Even after refreshing the page the Operator Events do not load as expected.

  • Fixed Issue 132789: On the Configure > Edge > Device > Interfaces page of the Orchestrator UI, if a user makes any changes to an Edge routed interface configuration and clicks SAVE, the UI does not provide any feedback for whether the save was successful or that the changes were applied and dialogue box does not close.

    This UI behavior can occur if the DHCP Address value is set outside of expected values for the route interface, but the issue here is the that UI does not communicate why the configuration fails and makes troubleshooting difficult.

  • Fixed Issue 132827: For a customer enterprise, on the Monitor > Routing page of the Orchestrator UI, a user cannot delete a route where its State = Removed.

    The expected behavior is for the UI to have the option to delete a Removed route from this page.

  • Fixed Issue 133180: For a customer enterprise where Branch to Branch VPN is configured along with the Hubs for VPN option, a user cannot later disable Branch to Branch VPN while Hubs for VPN is still configured.

    If the user disables Branch to Branch VPN with Hubs for VPN also selected and tries to Save, the Orchestrator UI throws the error "Cannot save changes. There is one or more errors with your configuration." However the configuration is valid and the change should be allowed.

  • Fixed Issue 133240: On the Global Settings > Customer Configuration page of the Orchestrator UI, a user cannot make SD-WAN service configuration changes if the customer has no licenses selected.

    The issue is that in this instance Edge licensing is disabled for the customer enterprise, which means the Orchestrator should not consider the absence of licenses a consideration when validating the configuration.

  • Fixed Issue 133851: When a user with an Enterprise Support role logs into the Orchestrator, when that user goes to the Service Settings > Alerts & Notifications page of the UI, the SNMP Trap Destination and Webhook parameters can be configured by this user role.

    Enterprise Support is a read-only role and all Alert settings should also be read-only and not configurable by this user role.

  • Fixed Issue 133983: For an Operator with a Super User role, when this role attempts to turn off Two Factor Authentication (TFA) for another Operator user, the Orchestrator throws an error and prevents the action.

    The error reads: "user does not have CREATE:OPERATOR_USER required for this operation", which is not correct as the Operator Super User does have the permissions to make this change.

  • Fixed Issue 134078: On the Configure > Network Services > Authentication Services > RADIUS Service page of the Orchestrator UI, if a user changes the Authentication Port value and saves changes, the UI does not update this value and the older value continues to be used.

    The port number is being changed by the API on the backend but the UI continues to show the older value, leading the user to think their change failed.

  • Fixed Issue 134911: A user may observe that the Diagnostics > Remote Diagnostics page for any SD-WAN Edge does not load for potentially more than an hour.

    The portal logs would include multiple occurrences of the error message: "Enterprise WebSocket connection limit exceeded". The issue is caused when a Remote Diagnostics page for an Edge times out and then the user clicks Reconnect. Upon this action the Orchestrator initiates not one, but numerous reconnect attempts that trigger a rate limit function on the Orchestrator and prevent the page from loading.

  • Fixed Issue 134940: The Orchestrator may assign an Edge a primary and secondary Gateway from the same datacenter location.

    Doing this undermines Gateway resiliency through geographic diversity while also potentially creating imbalances in Gateway usage for a Gateway Pool. The issue arises out of the Orchestrator's use of a geolocation service that relies on the Gateway's IP address to establish location. This service can sometimes place an IP address significantly farther away from the Gateway's actual location in the datacenter and fool the Orchestrator into classifying it as geographically diverse from other Gateways in the same datacenter. This is corrected by adjusting the Orchestrator's geolocation tolerances to allow for this service's behavior so that all Gateways in that location are properly classified and ensure geographic diversity for a customer site.

  • Fixed Issue 135517: On the Monitor > Edges screen of the Orchestrator UI, if a user clicks on a particular Edge the screen for that Edge may not load.

    When this issue is encountered, the screen just remains blank and is the result of a failure of the URL redirection process which can observed if looking at the browser's console.

  • Fixed Issue 136247: A user with an Enterprise Administrator role where the privilege to update LAN-Side NAT is denied can still update these rules

    This issue is found in a user role created using the Role Customization feature and the Orchestrator UI is not applying the LAN-Side NAT parameter as part of this custom role package.

  • Fixed Issue 136404: On the Configure > Edges screen of the Orchestrator UI, a user may observe that when this screen loads there are no Edges listed.

    This issue can be encountered due to an undefined parameter in the Orchestrator's API response for this screen and so the Edge list shows as empty.

  • Fixed Issue 136454: On the Configure > Edges > Firewall > Edge Security page of the Orchestrator UI, the parameter USB Port Access does not include the Edge 7x0 model.

    With this defect, USB Port Access only specifies Edge models 510/510-LTE and 6x0 in their deny list. With the introduction of the Edge 710-W, the deny list should also include this model and all upcoming Edge models in the 7x0 line as they are USB enable/disable capable.

  • Fixed Issue 136622: On an Orchestrator that runs on a large scale (1000+ Edges), users may observe degraded performance during certain periods.

    During peak use periods on a large scale Orchestrator where a high number of configuration changes or monitoring tasks are being performed, Orchestrator services could run out of MySQL connections and trigger API rejections and timeouts. The system does not respond gracefully to these API callers, leading to accumulated unprocessed APIs and increased pressure on system resources. The fix ensures smoother system operation in such cases.

  • Fixed Issue 136930: A user may find their attempt to activate an Edge 710-W is not successful when using the provided email link.

    The link is based on code that is conditional for the device model being activated, and this code does not account for the new Edge 710-W and results in some Edge configurations failing for this model with a resulting overall activation failure.

  • Fixed Issue 136937: On the Configure > Edge > Device > Interfaces page of the Orchestrator UI, the SD-WAN Edge models 710-W and upcoming 710-5G display the wrong interfaces.

    The Edge 710-W shows CELL interfaces, which it does not possess. The Edge 710-5G shows an SFP2 interface, and GE5 and GE6 interfaces, none of which this model includes.

  • Fixed Issue 137281: Customers may observe that their Edges show as offline on the Orchestrator UI though the Edges are in fact up, connected, and passing traffic.

    When an Edge configuration is changed, each Edge is expected to make just 3 database calls, which the Orchestrator can effectively manage even there is a mass Edge configuration change (for example when a Profile that a large number of Edges use is changed). For this issue, the mass configuration change has most of the Edges making from 20 to 30 database calls and this overwhelms the Orchestrator and impacts the Edge heartbeat management and results in the Orchestrator missing heartbeat checks and marking the Edges as offline.

  • Fixed Issue 137282: On the Global Settings > Customer Configuration page of the Orchestrator UI, if the enterprise uses Partner Gateways with hand-offs and BGP filters are configured, a user cannot insert a filter in between two existing filters.

    The expected behavior is to have the option to add a rule above or below an existing rule.

  • Fixed Issue 137447: On the Configure > Edge > Device > Interfaces page of the Orchestrator UI where a user has deployed an Edge 710-W, the Orchestrator is not selecting the correct configuration for the Wi-Fi radio based on what is seen on the UI.

    This is specific to when the Orchestrator autodetects the Wi-Fi type (2.4 Ghz or 5 Ghz). The Orchestrator will select the type based on the location of the Edge, making sure to select 2.4 Ghz when the country where the Edge is located requires it (for example, China and Taiwan). However with this issue, the Orchestrator shows the Edge 710-W is using the 2.4 Ghz but it is in reality using the 5 Ghz type. The issue relates to the 710-W having dual-band capable radio, which is new for any SD-WAN Edge model.

    This can be worked around by overriding the setting and manually configuring the correct Wi-Fi type.

  • Fixed Issue 137699: On an Orchestrator upgraded to 5.4.0.2 build, if a user goes to the Configure > Edge > Device page of the UI they cannot save any changes to either Zscalar or OSPF settings.

    The issue is the result of a UI validation defect when changing the configurations for these Edge settings.

  • Fixed Issue 139369: A user may observe that their attempt to activate an Edge 710-5G fails.

    The Orchestrator erroneously sends the alias name Edge7105G instead of the correct name Edge710-5G and this results in an Edge activation failure.

  • Fixed Issue 139437: The Orchestrator does not accurately record and display the APIv2 connection metrics.

    For every invocation of an APIv2 endpoint, the metrics about total database queries and query execution times are reported to a statistics process. Beginning in Orchestrator Release 5.4.0, the database connections are individually assigned to every API call, and this had the effect of not pushing connection metrics for APIv2 to the Orchestrator's statistics process.

Resolved in Orchestrator Version R5403-20240126-GA

Orchestrator build R5403-20240126-GA was released on 01-29-2024 and is the 3rd Orchestrator rollup for Release 5.4.0.

This Orchestrator rollup build addresses the below critical issues since the 2nd rollup build, R5402-20231122-GA.

  • Fixed Issue 118501: On the Monitor > Edges page, if a user clicks on CSV to download their list of all SD-WAN Edges, the Orchestrator will only download a maximum of 2048 Edges.

    This impacts customers with greater than 2048 Edges in their enterprise who need to download a complete list of all Edges.

  • Fixed Issue 122193: On the Configure > Edges > Firewall page, when configuring a Firewall rule for an Edge, for Firewall Action a user can only select 'Allow'.

    Firewall Action and the dropdown menu appears for a split second and then goes away immediately, and the user cannot change from the default Allow to any other option.The user would click on the

  • Fixed Issue 124849: When configuring a user role, if Update Device Settings is selected as a "Deny Privilege", privilege disables all settings on the Configure > Device page.

    When Update Device Settings = Deny Privilege is set on the Orchestrator UI, the Orchestrator blocks a user from changing any setting on the Configure > Device settings page. The expected behavior is for the Orchestrator to block access to only specific components: Interfaces, Loopback interfaces, Management traffic, VLAN, and WAN links.

  • Fixed Issue 125006: For a VMware SASE Orchestrator configured with a Disaster Recovery (DR) topology, the Orchestrator's database may fail and this leads to the Standby Orchestrator going into an error state and in rare cases this can lead to Edges and Gateways showing as offline on the Orchestrator UI and triggering events and alerts.

    The database state is expected to autorecover within a few minutes and the DR Orchestrators would resynchronize. However, sometimes this state can exceed the tolerance period, and both Edges and Gateways would begin to send their heartbeats to the Standby Orchestrator instead of the Active Orchestrator. As a result, the Active Orchestrator marks both Edges and Gateways not heart beating to it as down and triggers events and alerts. This issue is purely on the management side and does not affect network traffic.

    On an Orchestrator without a fix for this issue, the way to avoid this issue is to increase the Orchestrator System Property that governs tolerance for a failed synchronization: vco.disasterRecovery.transientErrorToleranceSecs by a suitable amount to provide a longer auto-recovery window.

  • Fixed Issue 125082: If a user configures a VMware SD-WAN Edge with an overridden DNS Server IP address on a VLAN, and then changes an interface setting for the Profile that Edge is using, the DNS Server IP address is no longer present for the Edge VLAN.

    The New UI does not send the override flag inside of the DHCP section and this causes any Profile changes to trigger an override of the DHCP section.

  • Fixed Issue 125456: For a customer enterprise where VNF's are used, if a user attempts to modify an Edge Device configuration, the VMware SASE Orchestrator rejects the changes when trying to save them.

    The configuration change can be as minor as adding a comment to an existing static route and the Orchestrator UI does not save the change where a VNF is activated. The user may observe a "Script injection error" banner at the bottom of the UI screen.

    The workaround on an Orchestrator without a fix for this issue is to temporarily deactivate the VNF's and then save the configuration changes. Once these are successfully saved, the user can then reactivate the VNF's.

  • Fixed Issue 125837: On the Monitor > Edges > System tab of the Orchestrator UI, the "Over Capacity Drops" graph does not display the correct data.

    The Over Capacity Drops graph only shows accurate data for last 5 minutes. Any other time period is wrong.

  • Fixed Issue 126695: If a user is configuring webhooks for Alerts, when they click on the "Configure Payload Template" button the menu is not displayed.

    This issue occurs when configuring webhooks on the SD-WAN > Settings > Alerts > Webhooks page of the UI. When looking at the browser console, a user would also observe the message: ERROR TypeError: Cannot read properties of undefined (reading 'invalid').

  • Fixed Issue 127268: User cannot perform a password reset by entering the provided Two Factor Authentication (2FA) pin number.

    The Orchestrator backend API improperly handles the new reset request format included with the New Orchestrator UI and this causes the failure to reset the password.

  • Fixed Issue 127994: For customers who deploy Edges with High Availability topology and are alos using an API with the VMware SASE Orchestrator, the Swagger specification is reporting a schema error due to an additionalProperty: title in the response schema.

    Full error would read:

    Structural error at paths./edge/getHaInterfaces.post.responses.200.schema should NOT have additional properties 
additionalProperty: title

    This is a cosmetic issue and does not affect the functionality of the API. The API can still be used to retrieve data regarding HA interface details, even if the Orchestrator receives this schema error.

  • Fixed Issue 128330: For an enterprise using a Non SD-WAN Destination (NSD) via Gateway network service, the UI permits the user to delete a NSD via Gateway from a Profile which includes a Business Policy rule associated with the global segment.

    As a result, the Business Policy rule becomes invalid and any traffic matching that rule is not steered as expected causing a possibly significant impact to customer traffic matching that rule.

  • Fixed Issue 128368: User cannot manually change the Gateway allocation when it is associated with a Non SD-WAN Destination (NSD) via Gateway configuration

    The issue is caused by the Orchestrator only displaying active the Gateway and not the redundant Gateway. As a result, all the possible Gateways are not available to the user. 

  • Fixed Issue 128888: For an Operator or Partner user, a site name filter in the Monitor > Events tab persists after the switches to a different customer on the Orchestrator UI.

    An Operator or Partner as part of their support of several customers may switch between multiple customer instances on the Orchestrator to check Events for each customer. The issue here is that if a user types in a site name filter while on one customer, the filter persists when switching to a different customer.

  • Fixed Issue 128921: A Partner or Enterprise Administrator with a Superuser role who navigates to the SD-WAN > Enterprise > Service Settings page may observe that there is no option to view Edge licenses.

    The expected behavior when navigating to Service Settings is to see an Edge Licensing listing on the left-hand table of contents as both partner and enterprise superusers have the correct privileges to see Edge licenses.

  • Fixed Issue 129088: The Orchestrator UI blocks a user from switching a configuration profile for an Edge if the replacement profile is missing a segment-specific configuration.

    The UI reports a reason to prevent selecting this profile as there being some configuration (for example, a Non SD-WAN Destination) that is present on a segment in the current profile and missing from the new profile if this is tried from Configure > Edges > Assign Profile. However, this is incorrect and a user should be able to change profiles in this scenario.

  • Fixed Issue 129232: A user cannot perform a password self-reset on the VMware SASE Orchestrator.

    When the user tries to access the self-password reset link directly in the browser, they are redirected to the login page instead of the password reset page.

  • Fixed Issue 129662: A user cannot tell if a VMware SD-WAN interface is enabled or disabled when looking at the Configure > Edge > Device > Interfaces page of the Orchestrator UI.

    The VMware SASE Orchestrator UI does not have color differentiation between enabled and disabled interfaces, preventing customers from identifying a sub-interface that is deactivated.

  • Fixed Issue 129958: On the Configure > Edge > Device > Connectivity > VLAN page of the UI, on the IPv4 DHCP Server section, the user cannot change the Number of Addresses unless they first override the general VLAN.

    Previously, a user could do an override a subsection of an Edge VLAN like DHCP or OSPF while accepting the remainder of the configuration from the profile. In this instance the user must override the entire VLAN so that they can edit the DHCP portion of it.

  • Fixed Issue 131224: On the Configure > Device > VPN Services page of the UI, when configuring a Zscaler service, the Sub-Location Name 'Other' can be edited and results in the Orchestrator marking the configuration as invalid.

    The UI throws the errors "Zscaler sublocations cannot be named as 0" and "Cannot save changes there is one or more errors within your configuration". The 'Other' Sub-Location should never be edited and this issue occurs only if the Zscaler service is deactivated and then reactivated.

  • Fixed Issue 131464: Changing the name of an existing WAN link configured as a backup only also causes the backup value to switch from "true" to "false".

    This results in a WAN link meant to be a backup to become active and pass traffic like any live WAN link which can especially impact mobile links on data plans that are not meant to be active full time.

  • Fixed Issue 131631: On the Monitor > Edge > Destinations page, a user cannot filter by Applications as the option is not present on the UI.

    A user should be able to filter by Applications for improved troubleshooting and traffic analysis.

  • Fixed Issue 132047: On the Configure > Edge > Device > Connectivity > VLAN page of the UI, when the user chooses the VLAN DHCP option 2 (Time Offset), a negative value cannot be entered despite it being an integer.

    The expected behavior is for the UI to allow both positive and negative integers.

  • Fixed Issue 132598: On the Monitor page of the Orchestrator UI, for any tab where there is an option to change the Hostname (for example, Sources), the hostname cannot be changed using Chinese characters.

    The UI is incorrectly using a validation that requires the hostname field to be standardized alphanumeric characters only. This validation is invalid as the hostname has no requirements to adhere to this convention.

  • Fixed Issue 133008: On the Monitor > Network Overview page of the UI, if the Edges are sorted by link status (for example, Links down, Links stable, or Links degraded), the Auto-Refresh option does not work and all monitoring information disappears from the UI.

    The user would observe the screen working correctly initially, and then after approximately 30 seconds, all monitoring information disappears, leaving only a red dot visible on the screen.

  • Fixed Issue 133102: On the Monitor > Sources tab of the Orchestrator UI, if a user edits the hostname for a source and saves changes, the change may not persist.

    When encountering the issue, the change persists for ~20 minutes but if a user came back and check after that time, they would observe the old hostname restored for that source. This issue is caused by a race condition that can occur due to the asynchronous client device uploads and background merge processes.

  • Fixed Issue 133198: A user who logs in using RADIUS authentication cannot create a custom role on the Global Settings > User Management section of the Orchestrator UI.

    A RADIUS-authenticated user can go through the steps of creating a customer role but when they attempt to save the configuration, the UI throws the error "Error occurred while creating composite role" and the configuration is not saved.

  • Fixed Issue 133199: When an enterprise user with read-only access pulls up the Edit Interface dialog, an untagged VLAN is displayed as empty.

    A read-only user should be able to view (though not edit) the untagged VLAN configuration.

  • Fixed Issue 133201: On the Global Settings > User Management page of the Orchestrator UI, If a user is creating or editing a custom role and attempts to change the 'Delete' parameter for PCAP Bundle privileges, the UI also changes all other parameters for PCAP Bundle.

    For example, if a user elects to disable PCAP Bundle Deletion, the UI also disables the Read, Update, and Create parameters as well, even if the user wanted those parameters to remain enabled.

  • Fixed Issue 133664: When the user attempts to save the changes made to a configuration, they may encounter an error stating that the domain name cannot be empty.

    When the user updates certain SD-WAN configurations and clicks on the Save Changes button, an error message can be displayed stating "Domain name cannot be empty as SSO is enabled for the Enterprise".

  • Fixed Issue 134435: When a user adds a new Hub Cluster to their enterprise, the newly added cluster may not appear in the cluster dropdown list.

    The Hub Cluster list is found on Configure > Edge > Device > HA. When the user adds a new Hub Cluster under the HA availability screen, the newly added Hub Cluster does not appear in the Hub Cluster dropdown list. The user needs to refresh the screen to see the newly added cluster in the list.

  • Fixed Issue 135644: After removing the Create Privilege > Non SD-WAN Destination via Gateway for an Operator with a Standard Administrator role, the Operator user cannot update the service.

    The Operator should be able to update an existing NSD via Gateway from the existing list found in the Configure > Network Services page, just not create new ones.

Resolved in Orchestrator Version R5402-20231122-GA

Orchestrator build R5402-20231122-GA was released on 11-22-2023 and is the 2nd Orchestrator rollup for Release 5.4.0.

This Orchestrator rollup build addresses the below critical issues since the 1st Orchestrator build, R5401-20231115-GA.

  • Fixed Issue 128330: For an enterprise using a Non SD-WAN Destination (NSD) via Gateway network service, the Orchestrator UI permits the user to delete a NSD via Gateway from a Profile which includes a Business Policy rule associated with the global segment.

    As a result, the Business Policy rule becomes invalid and any traffic matching that rule is not steered as expected. The correct workflow for this action is to first delete the Business Policy rule associated with the NSD and only then should a user delete the NSD itself.

  • Fixed Issue 129695: If a Partner or Operator user changes an Edge's Wi-Fi password on its WLAN interface and saves changes, they will still see the old Wi-Fi password when they look at the same Edge's WLAN interface after logging back in.

    In fact the password has not changed because an Operator or Partner user cannot change a WLAN Password on an Edge interface if the Global Settings > Enterprise Settings > Operator Support Access setting 'Allow access to sensitive data' is turned off.

  • Fixed Issue 130810: A user cannot insert a BGP filter rule into a list of these rules, either above or between two or more rules.

    A BGP filter rule can only be added at the end of the list. If a user really wanted this new rule to be higher on the list they would need to delete other rules and then re-add them at the end in sequence.

  • Fixed Issue 131789: When configuring Single Sign On (SSO) for an organization, even though the role information is present in the Identity Provider (IdP) response, the users cannot login to the Orchestrator.

    The Orchestrator cannot match the role of a user logging in via Single Sign On (SSO), if the IdP sends role information in a nested JSON structure. Starting in version 5.0.1.7, the Orchestrator can reference and match the role of a SSO user even if it is present in a nested JSON structure.If experiencing this issue on an Orchestrator without a fix for this issue, the workaround is to configure the IdP to send the role detail in the immediate level, and not in the nested structure.

  • Fixed Issue 132449: For a customer or partner using Single Sign On (SSO), when a VMware SASE Orchestrator is upgraded to 5.4.0.1, SSO users may not be able to login to the Orchestrator.

    After the Orchestrator is upgraded to 5.4.0.1, SSO users could observe that their attempts to login fail. In the Orchestrator logs a user would observe the messages "IdP returned empty roles" and "Authentication complete for user undefined".

    On an Orchestrator using 5.4.0.1, the workaround is to have users switch to native authentication (user name + password).

  • Fixed Issue 132524: On the Configure > Edge > Device > Routing & NAT > Static Route Settings page of the UI, when a user adds or modifies a static route to an Edge where the next hop IP address of the static route is within the same subnet of the VLAN network, the UI displays Edge interfaces when the Interface settings for the Local Routes should change to N/A.

    This issue can prevent customers from updating or adding static routes to their enterprise using the UI if their enterprise uses a next hop address within the same subnet of the VLAN network.

Resolved in Orchestrator Version R5401-20231115-GA

Orchestrator build R5401-20231115-GA was released on 11-15-2023 and is the 1st Orchestrator rollup for Release 5.4.0.

This Orchestrator rollup build addresses the below critical issues since the original GA build, R5400-20231020-GA.

  • Fixed Issue 117138: When using Zscaler Automation to create a Zscaler Cloud Security Service (CSS), Edge to Zscaler CSS IPsec tunnels may not be created.

    The Orchestrator ensures enqueuing only one action per Edge. However, if one action is stuck at the NOTIFIED state, all subsequent actions are not executed and the IPsec tunnels do not get built.

    On an Orchestrator without a fix for this issue, an Operator user needs to manually delete the stale Edge action from the Orchestrator database.

  • Fixed Issue 123078: When using the VMware SASE Orchestrator UI and navigating to the Monitor > Edge > Overview page, the columns are not aligned properly and difficult to read.

    The UI does not include any information for the Device Serial Number column, which means there are 11 columns but only 10 headers and this causes the readability issue.

  • Fixed Issue 123387: A user cannot add a Zscaler Non SD-WAN Destination (NSD) via Gateway with an existing IP address.

    The Orchestrator's validation prevents users from adding a Zscaler with a primary or secondary IP address that is already used in another NSD via Gateway.

  • Fixed Issue 125309: When a user deactivates IPv6 at the Edge level under Configure > Device > IPv6 Settings, the OSPF option for IPv6 can still be edited, activated, and saved.

    This causes confusion for a customer user who configures these settings not knowing they should not because the IPv6 version of OSPF is not in effect.

  • Fixed Issue 125964: For a customer deploying Non SD-WAN Destinations (NSD) via Gateway, when navigating to the Configure > Network Services > NSD via Gateway > Generic IKEv2 page and clicking on Save after adding custom site subnets, the NSD configuration changes are not getting saved.

    This issue is the result of invalid fields (IKE SA Lifetime(min) and IPsec SA Lifetime(min) in the Primary VPN Gateway section.

    Customer is using Old UI to complete the task.

  • Fixed Issue 126602: A customer cannot add a Gateway Pool to their existing partner configuration Gateway Pool.

    The attempt to do so returns an error if the Gateway Pool in the Partner configuration has a managed pool because the existing managed pool IDs are not removed by the UI.

  • Fixed Issue 127727: When creating a new Cloud Security Service (CSS), a user cannot configure the Domestic Preference option.

    If a user activates the Domestic Preference check box and saves the configuration, the Orchestrator verifies the credentials and displays a message stating that "Changes saved successfully!". But post-save, when the CSS profile is opened again, the user would observe that the Domestic Preference check box is not selected.

  • Fixed Issue 127774: Under Configure > Edge > Device > Connectivity > Loopback Interfaces, a user attempts to configure a loopback interface may not be successful.

    When the issue is encountered, a user configures a loopback interface for an Edge and Saves Changes and the UI does not throw an error but the configuration is not applied and does not show on the UI page.

  • Fixed Issue 127843: The UI does not display correctly when localized to the Italian language.

    This impacts a user because some navigation tabs overlap with one another.

  • Fixed Issue 128017: A customer may observe that when navigating to the Configure > Edge > Device page, that the page never loads.

    When this issue is encountered, the Edge configuration references are mistakenly deleted by the Orchestrator from its database. Once these reference are removed, they cannot be restored.

    On an Orchestrator without a fix for this issue the only workaround is to force the Edge to use all the configurations from the Profile associated with that Edge. If the Edge has a lot of custom Edge Override configurations, this would be mean creating a special Profile just for that Edge.

  • Fixed Issue 128277: When a Partner or Enterprise native user (one who logs into the Orchestrator with a username and password) attempts to log in with an expired password, the UI enters a loop and displays a blank screen.

    A user would observe the Expired Password screen refreshing over and over indefinitely. The only way to work around this is to have an Operator or Partner with the correct role to reset the password for the user.

  • Fixed Issue 127904: When a user creates a static route and adds an ICMP probe to this route, the Edge does not install the ICMP probe and displays a parsing error.

    This issue is caused by sending Next Hop IP and Source IP value as null instead of an empty string from the Orchestrator to the Edge.

  • Fixed Issue 128279: On the Configure > Overlay Flow Control > Routes List page, a user can see a maximum of 256 routes with no option to click to an additional page of routes.

    The hard 256 route limit and lack of pagination impacts customers with large enterprises which include a number of routes well in excess of the hard 256 limit.

  • Fixed Issue 128357: On a customer Enterprise that uses OSPFv2 or OSPFv3 for routing and selects OE1 from the Default Route list, an invalid option “None” appears in the Advertise list.

    The valid options for Advertise are "Always" and "Conditional". None is not a valid option and should not appear on the UI.

  • Fixed Issue 128765: On the BGP Filters page, the Submit button may remain inaccessible a user changes pages.

    When a user edits the BGP Filters table and navigates to another page with an invalid state on a current one, the controls will be invalid after returning back and the user fills in the correct information.

    On an Orchestrator without a fix for this issue, a user needs to stay on the page of the table where the control is invalid. Alternately, the user can remove this row and add it again.

  • Fixed Issue 129061: For a customer with Partner Hand off activated from the Customer > Global Settings > Customer Configuration > Additional Configuration > Gateway Pool screen, the "Use for Private Tunnels" and "Advertise Local IP Address via BGP" check boxes are not clickable under the IPv6 section of Gateway Hand Off Interface.

    This prevents the user from deactivating the IPv6 private tunnel for Hand off interface. 

  • Fixed Issue 129494: On the Customer > Global Settings > Customer Configuration > Service Configuration > SD-WAN page, when a user is editing the service configuration, the user is required to add the domain name every time.

    The user is required to do this even if Single Sign-On (SSO) authentication or Edge Network Intelligence (ENI) is not configured for the customer. A domain name is not mandatory if a customer is not using ENI or SSO.

  • Fixed Issue 129584: On the Configure > Edges > Business Policy page, when a user edits an existing Business Policy rule, the Orchestrator UI does not update the reconfigured value for the Destination field even after saving the changes.

    For example, for an existing Business Policy rule with Destination set as “IP Address”, if the user changes the value of Destination to “Any” and saves change, the changes made to the Destination field for the rule is not reflected in the UI. The user would still see the Destination field set to “IP Address” instead of “Any” in the rule.

  • Fixed Issue 129756: When an Operator runs the update schema for a cloud-based VMware SASE Orchestrator, the Orchestrator may not be able to connect to the remote database.

    This issue does not impact VM-based Orchestrators.

  • Fixed Issue 129765: When editing a routed interface for a VMware SD-WAN Edge, the Orchestrator UI populates a wrong default value for dhcpServer.options.

    For example, when a user edits the "GE3" routed interface and saves the device settings configuration data, the value of the “options” field under “dhcpServer” is sent as null instead of an empty array.

  • Fixed Issue 129894: In the Operator portal of the Orchestrator UI, when looking at the Gateway Management > Gateways > Overview > Customer Usage screen, a user may observe some Edge client tunnel details are missing.

    This issue can occur if the Edge name, Gateway Pool name, and Gateway type are the same.

  • Fixed Issue 130153: For Enterprise users with a Support role, the Restart Service option is not available under Monitor > Edges > Select Edge > Shortcuts > Remote Actions menu.

    This prevents Support role users from performing an Edge service restart from the Shortcuts menu on the Monitor > Edges page.

  • Fixed Issue 130877: When a user adds a static route to an Edge using the Orchestrator UI, client users for that Edge may observe that traffic fails for some local routes.

    On the UI under Configure > Edge > Device > Routing & NAT > Static Route Settings, the Interface settings for the Local Routes are changed to N/A and cannot be edited. The local_routes logs for that Edge would show "reachable": "False" for the impacted local routes.

    This issue can be encountered when the next hop IP of a static route is within the same subnet of the VLAN network. In this scenario the Orchestrator UI removes the interface of the static route and this results in those local routes showing a reachability of false.

  • Fixed Issue 131138: On the Configure > Device > VPN > Cloud VPN page, the UI allows a user to save a change to the Branch to VPN Hubs option if there are no Hubs configured.

    This results in the UI removing all Cloud VPN configurations because that file is corrupted which will result in client user traffic being disrupted for routes that would otherwise use Cloud VPN.

Resolved in Orchestrator Version R5400-20231020-GA

Orchestrator Version R5400-20231020-GA was released on 10-23-2023 and resolves the following issues since Orchestrator Version R5302-20231011-GA.

Note:

Release 5.4.0 contains all Orchestrator fixes that are listed in the 5.2.0 Release Notes up to version 5.2.0.3 (R5203-20230809-GA), and all Orchestrator fixes in the 5.3.0 Release Notes up to version 5.3.0.2 (R5302-20231011-GA).

  • Fixed Issue 48230: When on the Monitor > Edges page, searching for Edges by using "Model" is not fully accurate.

    When a user filters by Model number, the Orchestrator returns extra Edges.

  • Fixed Issue 48609: On the Monitor > Routing page, a user cannot sort the segment name in a multicast route table.

    In Release 5.4.0, the API is enhanced to accept the segment name as a sorting parameter and helps in sorting the segment name.

  • Fixed Issue 78581: A user can advertise a VLAN when connected routes are deactivated on the Orchestrator UI.

    This action should be rejected by the Orchestrator with an error thrown. In the fixed version, the Orchestrator UI listens to connected routes and deactivates the advertise flag based on the connected routes and vice-versa.

  • Fixed Issue 82095: User can configure invalid device settings for Edge VLANs that will result in significant connectivity issues for the Edge.

    The Orchestrator is not attempting to validate device configurations. In particular, a VLAN configuration for a switched port with an empty table. Some configurations can be so full of errors that the Edge's management process will fail.

    On an Orchestrator without a fix for this issue, the customer should review all VLAN Device settings and ensure they are valid as the Orchestrator is not checking.

  • Fixed Issue 91869: When an Operator user goes to Manage Partners and clicks "Add Operator Profile" button, the Operator Profile description is truncated when it exceeds a certain text length.

    The Orchestrator should not truncate the Operator Profile description regardless of its text length.

  • Fixed Issue 92766: On the Monitor > Routing page, pagination information is incorrect.

    If a user clicks the Next button, the UI page does not stop even after reaching the last page due to the filter logic not being applied correctly with the result that the page number shown is not correct.

  • Fixed Issue 102121: When an Edge's Secure Access configuration is updated multiple times without any updates to the Edge's Firewall configuration while using the Orchestrator UI, the Secure Access configuration updates may stop being sent to the Edges.

    The issue has been encountered most often in testing where Engineering deliberately forces a large number of Secure Access updates without any Firewall updates. However it can, in rare instances, be triggered by a customer in the field.

    If experiencing this issue on an Orchestrator without a fix, a user can manually update the Edge's Firewall configuration from the UI just once. After the manual Firewall update, the user can redo the Edge Secure Access configuration change from the UI and the Orchestrator would push the Edge Secure Access configuration change from the UI to the Edges.

  • Fixed Issue 103828: The Application Map editor's application search filter has poor clarity.

    This makes checking or changing the configuration for an application very difficult, requiring manual paging through numerous pages of applications to find the application of interest.

  • Fixed Issue 104395: The Monitor > Network > Overview page's "Links Down" section shows only the first 10 sites with links down. When a user clicks on the "View All" button, the Orchestrator UI takes the user to the Edges section which lists all Edges regardless of link status even though the user only wants to see sites with Edges that have one or more links down.

    When a user clicks on the “Links down” icon, the list should show all down links, not just 10 and there is no possibility to filter Edges by link or Edge status after the 'Show All' button is clicked.

  • Fixed Issue 108285: When configuring a profile on the Orchestrator UI, if a user changes an Edge interface from switched to routed, the Orchestrator appends the new routed interface to the end of the routed instead of inserting it at the appropriate index.

    The wrong order of Routed Interfaces in a Profile Configuration causes validation problems when a user adds a static route using those references for that routed interface.

  • Fixed Issue 108583: For a VMware SASE Orchestrator deployment where Google Maps is not configured, if a user attempts to manually enter the address for an Edge location, the Orchestrator does not allow the attempt to Save.

    Google Maps requires an API key to work on an Orchestrator and the lack of one means the Orchestrator has no means of establishing geolocation for an Edge. However, the expected workaround is for a user to manually enter an address for the Edge. With this issue the user cannot input an address manually, because the Orchestrator has a validation to check if the address is real and valid, and in the absence of the Google Maps application, this check cannot be performed and the Save button is not exposed.

  • Fixed Issue 108687: A user can configure a Non SD-WAN Destination via Gateway to have multiple Gateways with the same IP address.

    The Orchestrator should reject this configuration and throw an error.

  • Fixed Issue 109125: On the Administration > User Management page of the Orchestrator UI, a user cannot sort SSH Keys by the Duration column.

    This limits a user's ability to find a particular SSH key if trying to locate it by the key's duration.

  • Fixed Issue 109715: A down WAN link disappears from the Monitor > Network Overview page after 24 hours even though the Edge Down Link Limit is set to a value > 24 hours.

    A user can navigate to Customers & Partners > Monitor Customers > Service Settings > Edge Management and for the General Edge Settings section, the Edge Link Down Limit can be set to some value greater than 24 hours and saved. However, despite this new setting, a down WAN link is no longer present on the Monitor page after 24 hours.

  • Fixed Issue 110285: A user under a customer enterprise with greater than 5000 Network Services may observe that the Orchestrator UI loads slowly for Network Overview, Monitor > Edges, Configure > Edges, and Configure > Edges > Device.

    This issue is caused by the getEnterpriseServices API call taking a long time to fetch the results when there are a large number of Network Services to retrieve.

  • Fixed Issue 111407: The VMware SASE Orchestrator does not allow a user to deactivate an Edge's routed interface if it is user-defined and has no WAN link associated with it.

     The Orchestrator validation does not let a user save data without a WAN link added, and the only workaround is to add a WAN link.

  • Fixed Issue 111497: Under the Configure > Overlay Flow Control page of the Orchestrator UI, the Preferred Exit VPN values are showing as "undefined" for Site subnets.

    If the Next Hop configuration is configured for a Non SD-WAN Destination via Gateway, the site subnets in Configure > Overlay Flow Control shows Preferred Exit VPN as "undefined" for those VPNs whose tunnel Type is "ACTIVE_ACTIVE".

  • Fixed Issue 111778: When a user is editing a Non SD-WAN Destination via Gateway with the Checkpoint type the Orchestrator changes the VPN type to Generic IKEv1 Router.

    This issue causes issues for any customer with a Checkpoint type NSD and can cause disruption if the user does not catch what the Orchestrator UI is doing.

  • Fixed Issue 112123: When a VMware SD-WAN Edge 640-N is added, the VMware SASE Orchestrator UI displays the model name incorrectly.

    The Orchestrator UI displays the name as “EdgeModel.edge640-n” and not “Edge 640-N”.

  • Fixed Issue 112188: For a customer enterprise that deploys a Non SD-WAN Destination using GRE, when the NSD tunnel goes down, the Orchestrator displays the incorrect message.

    The Orchestrator displays the message: "Edge Direct IPsec tunnel down", when the NSD tunnel is GRE.

  • Fixed Issue 112535: When configuring a Firewall rule under Configure > Firewall, the in Source > Define > Transport screen displays the Transport Port incorrectly.

    The Orchestrator displays the Transport Port option as just "Transport", which is vague and can cause confusion when configuring a Firewall rule.

  • Fixed Issue 112826: When a customer exports a list of Alerts to a CSV file through the Orchestrator UI, they only retrieve 512 items.

    Doing the same export using an API can deliver up to 2048 Alerts, which is the expected amount for the Orchestrator UI export as well. The 512 limitation impacts customers exporting Alerts where the number of Alerts exceeds that 512 limit for the specified time span.

  • Fixed Issue 113474: User cannot configure a partial host interface address while configuring DHCPv6 Prefix Delegation on IPv6 LAN interfaces or subinterfaces.

    The Orchestrator is missing proper validation to allow a partial IPv6 address.

    What is expected for this behavior: With the fix, the following types of full or partial interface addresses are allowed or blocked for DHCPv6 Prefix Delegation in LAN and VLAN:

    • Allowed IPv6 Addresses:

      • 2001::20; 2222:db8:3333:4444:5555:6666:7777:8888

      • 2001:db8:3333:4444:CCCC:DDDD:EEEE:FFFF

      • ::1:0:0:0:1

      • ::f:1:0:f

      • ::f:1:e;

      • ::c:1

    • UNIQUE_LOCAL_IPV6 - fc00::

    • GLOBAL_UNICAST - 2000:: or 2001:0DB8:C21A:1::

    • Blocked IPv6 Addresses:

      • empty address - ::

      • multicast address - FF01:0:0:0:0:0:0:18C, FF01:0:0:0:0:0:0:BAC0, or FF01:0:0:0:0:DB8::

      • loopback - 0:0:0:0:0:0:0:1 or ::1

      • link-local - fe80::210:5aff:feaa:20a2 or fe80::260:97ff:fe02:6ea5

      • site local - fec0::

    • Only one pair of :: can be used, so the following IP addresses are blocked: ::1:0:0::0:1 or ::f:1:0::f

  • Fixed Issue 113530: User cannot change IPv6 addressing type from DHCPv6 Prefix Delegation to any other type on a Routed LAN interface or subinterface.

    On the Configure > Edge > Device > Interface > LAN IPv6 section where DHCPv6 Prefix Delegation can be configured on the Orchestrator, if the user attempts to select any other addressing type (Static/DHCP Stateful/DHCP Stateless), the Save option remains inaccessible, and the user cannot complete the change. The issue is the result of a defect in a process that checks the uniqueness of the DHCPv6 Prefix Delegation configuration in an Edge.

  • Fixed Issue 114151: When displaying Edges and Gateways on their respective list pages, the certificate assigned to them is an optional column.

    The use of certificates is a default behavior, so the default column sets for either Edge or Gateway lists should always show them without requiring the user to set them.

  • Fixed Issue 114444: When a VMware SASE Orchestrator is upgraded to version 5.1.x or higher, a user cannot save their changes of a configuration for a Non SD-WAN Destination via Gateway where redundant tunnels are already configured.

    User will not be able to save the configuration on the NSD via Gateway page and could cause packet drops on Gateways for redundant NSDs.

    The workaround for this issue is to update the max hop to 2 in the NSD via gateway UI for the redundant BGP neighbors and save the changes.

  • Fixed Issue 114475: When an Operator attempts to upgrade a VMware SASE Orchestrator from Release 4.2.0 to 5.1.0, the Orchestrator may report that the upgrade failed.

    In the logs the Operator would observe this entry: Error while initializing CWS Server service Error: Too many connections. This issue is triggered by MySQL restarting before vco-db-schema is installed which is caused by MySQL not setting the maximum number of connections. In addition, while the Orchestrator reports that the installation failed, in reality the installation did finish, and they can restart the Orchestrator and all services will work as expected.

  • Fixed Issue 114897: When logged into the VMware SASE Orchestrator UI as a Partner or when an Operator looks at the Partner level, the "Customers" tab shows as "Customers & Partners".

    The correct name is Customers, and this is displayed with this Orchestrator version.

  • Fixed Issue 115441: On the Global Settings > Customer Configuration page, when configuring the SD-WAN tile, software and firmware images are only visible when they are activated.

    Software and firmware images should always be visible to the user regardless of their status.

  • Fixed Issue 115695: On the Monitor > Edges page, a customer cannot search for an Edge by Description in the Edge List table.

    Certain words in an Edge's Description (for example, Activated or the Edge's location) are useful for searching for all Edges with a similar description.

  • Fixed Issue 115837: On the Monitor > Edges or Configure > Edges page of the Orchestrator UI, when trying to search for Edges on the main table, the Edges cannot be searched until all Edges are loaded.

    When loading Edges on the main table, the list spinner blocks the Search bar making it impossible to use until request finishes which can take some time, especially if the enterprise includes a large number of Edges.

  • Fixed Issue 116021: On the Configure > Edges > Certificated Detail page of the Orchestrator UI, the dialogue window has poor usability.

    The Certificate Detail dialogue window has 3 embedded scroll bars, which makes it very cramped and difficult to use properly.

  • Fixed Issue 116524: For customers who subscribe to Edge Network Intelligence and have Analytics activated, on the Monitor page when looking at the left side list of links, a user observes two links for Analytics that have different names but are exactly the same and redundant.

    There are two links for Edge Network Intelligence: Application Analytics and Branch Analytics, and both go to the same place in ENI. This is corrected with a single link on the left side called Edge Network Intelligence.

  • Fixed Issue 116610: Users cannot add a new Edge loopback interface via APIv2.

    Unable to create a new loopback interface using the PATCH ADD operation via APIv2. The schema structure for the loopback interface in APIv2 did not allow interfaces named by the interface ID and in this scenario, the update of the loopback interface fails.

  • Fixed Issue 116999: When an Operator user logs into the VMware SASE Orchestrator, on the right side of the Orchestrator the menu reads "Partner Menu".

    This can cause confusion for the Operator thinking they are accessing the wrong menu since it should read "Operator Menu".

  • Fixed Issue 117001: On the Administration > User Management > Roles page of the Orchestrator UI, the user is unable to observe all the role types/users on one page.

    The Roles page has pagination, which means not all roles are found on a single browser page but are instead distributed across multiple browser pages. The issue is that the UI page controls are not easily seen, and the fix eliminates pagination and places all the roles and types on one page.

  • Fixed Issue 117020: On the Configure > Network Services page of the Orchestrator UI, if a user selects the TACACS Services tab and tries to delete a TACACS service, the UI displays the wrong text string to confirm deletion.

    When deleting a TACACS service, the UI displays the text string: configuration.networkServices.deleteConfirm. The UI should display the correct delete dialog “Are you sure you wish to delete this item?”.

  • Fixed Issue 117145: For a customer enterprise which deploys VNFs, when a user changes any configuration in Configure > Device, the Orchestrator deactivates the VNF.

     The Orchestrator does not redeploy the VNF after it is deactivated, causing significant disruption for a customer.

  • Fixed Issue 117152: If a user creates a BGP filter where community has integer values greater than 65535 and assigns it to a neighbor, the Orchestrator allows this integration even though the Edge ignores the filter.

    The Edge supports only community values in the AA:NN format with values (0-65535):(0-65535). If any values greater than 65535 are given as an integer, the Edge ignores these values and the Orchestrator fails to reject a filter with a value that exceeds 65535.

  • Fixed Issue 117385: A user may observe slowness on the VMware SASE Orchestrator when they attempt multiple configuration changes.

    The customer cannot perform multiple configuration changes on the Orchestrator, with the operations taking up to a minute.

  • Fixed Issue 117537: On the Monitor > Edges > Sources page of the Orchestrator UI, the list of Client Devices shows an incorrect number of devices, usually a smaller number than there actually are.

    The issue occurs when "Visibility by IP Address" or "Visibility by MAC Address" mode is not set on the default Edge's specific configuration module, but it is set in an associated module.

  • Fixed Issue 117573: On the Configure > Device > VPN > Cloud VPN page, when a user tries to configure a Branch to Hub site, Hubs cannot be filtered using the filtering icon.

    Where a customer enterprise has multiple Hubs configured, the lack of filtering makes is difficult to configure a Branch to Hub VPN.

  • Fixed Issue 117614: The Configure > Profile page of the Orchestrator UI is missing several actions in the Shortcuts box.

    The missing shortcuts include: Migrate Profile; Duplicate Profile; Modify Profile; and Delete Profile. These actions can be found on the Profile's list page, but the customer should have an easier access to these actions.

  • Fixed Issue 118265: On the User Management page of the Orchestrator UI, an Admin User of the correct privileges cannot delete a user account if they used search to find that account.

    When the user uses the search bar to find the user and uses 4 characters or more, once located the Delete button is grayed out and not functional.

  • Fixed Issue 118302: On the Monitor > Edge page of the Orchestrator UI, the user cannot filter the link status for Edges.

    The Orchestrator API has been enhanced to accept the link status as a filter parameter and helps in filtering the link status on the Edge list screen.

  • Fixed Issue 118527: On a VMware SASE Orchestrator deployed as a Bastian Orchestrator using an External Certificate Authority, customers may observe a VMware SD-WAN going offline when it is successfully promoted.

    In a bastion environment where the private Orchestrator is integrated with an External Certificate Authority, the Edge goes to an offline state soon after its promotion to the private Orchestrator.

    On an Orchestrator without a fix for this issue, the workaround for this issue is to delete the certificate revocation list (CRL) on the Edge device and restart the Edge service.

  • Fixed Issue 118670: The Monitor > Network Services > Edge Clusters page can take over 30 seconds to load.

    When an enterprise has more than 10K Network Services, the Monitor > Network Services > Edge Cluster page take a long time to load.

  • Fixed Issue 118761: On the Edge Listing page, when using the Assign Software Image dropdown menu, the user cannot filter Software Images.

    The lack of filtering makes the selection process more difficult when there are many Software Images to select from.

  • Fixed Issue 118764: On the Edge Listings page, the Assign Operator Profile dropdown does not allow users to filter items.

    The lack of filter options makes the Operator Profile selection process more difficult than it should be.

  • Fixed Issue 118767: On the Edge Overview page, users cannot filter items and make selections when updating the Edge License selection.

    The lack of filtering makes the selection process more difficult when there are many Edge Licenses to choose from.

  • Fixed Issue 121336: When an Edge or Profile has "Partner Gateway Assignment" configured on the Configure > Device page and changes are made via an API call, the "Profile Update" Event always shows the Gateways as deleted.

    This is purely cosmetic and has no functional effect. The issue occurs because the Gateways field is deprecated in the configuration.

  • Fixed Issue 121995: A customer receives HA Failover alerts even though they turned those alerts off on the Configure > Alerts page of the Orchestrator.

    When enterprise alerts are enabled but HA Failover alerts are not enabled, the HA Failover alert is being sent to enterprise users.

  • Fixed Issue 122940: When a user is on the Gateway Management page, they may be redirected to the wrong screen in some cases.

    When a customer wants to assign a Secure VPN Gateway, the Orchestrator should redirect to the Assign Datacenter Gateway screen, but it incorrectly redirects to the Assign Super Gateway screen.

  • Fixed Issue 123593: For a customer enterprise site deployed with a High Availability topology where Edge Network Intelligence Analytics is activated, in rare instances, the HA Edge may not pull the Analytics configuration from the Edge Network Intelligence backend.

    It is possible that both Active and Standby Edges will obtain the same token from the ENI backend. If the Standby Edge gets the token after the Active Edge, then Active Edge's token becomes stale, resulting in this scenario.

  • Fixed Issue 123619: If a VMware SASE Orchestrator does not have access to the internet (for example, one that is on-premises), the Monitor > Edge > Overview page is empty, displaying no information.

    When the Orchestrator does not have access to the Internet, the Edge Overview page is not accessible because of a lack of access to Google services.

  • Fixed Issue 123867: The link metrics and series APIs return an error.

    When the link metrics or series API is called without a list of metrics, the API mistakenly returns an error instead of adding all applicable metrics to the response.

    On an Orchestrator without a fix for this issue, a user must submit an API request with a list of metric fields to return.

  • Fixed Issue 124092: On the Overlay Flow Control screen, the user cannot see the Segment name.

    When trying to filter for the Segment name, the Segment name does not be populated as part of the results.

  • Fixed Issue 124743: On the Monitor > Edge > Overview screen of the VMware SASE Orchestrator, the Links Status grid never completes loading.

    Link Status cannot be loaded due when one or more WAN links is configured as a Backup or Hot Standby. In this instance, there is an absent Backup/Hot Standby status in one of the links which makes the UI fail.

    A user can work around this issue by toggling the Link Mode for the link's "Backup Transport" from "Hot Standby" to "Active" and save, and then back to "Hot Standby".

  • Fixed Issue 126257: The top value on the Monitor > Edge > Links page is not visible to users when there are a lot of extreme high values.

    This is because the chart height was previously summed using lower values, which made the chart inaccurate and unable to be aligned with the scale.

  • Fixed Issue 127281: Under the Configure > Overlay Flow Control page of the Orchestrator UI, static configurations on routed subinterfaces do not show up on the OFC page as connected routes.

    When a static configuration is done on a routed non-wan interface or subinterface, they are expected to show up on the OFC page as connected routes. But this is not working for subinterfaces using IPv6 configurations where IPv6 is not also activated on the parent interface of the subinterface.

  • Fixed Issue 127636: On the Monitor > Edge > Sources page of the VMware SASE Orchestrator UI, a user searching a Source by FQDN does not work.

    Search by FQDN functionality is not working as expected when using the New UI which prevents a user from locating a Source using a standard method. This includes not having the option of searching by a partial string.

    You can still search by IP address, but you must use a full string.

  • Fixed Issue 127849: The View Certificate button is grayed out and not clickable on the Edge > Configuration > Overview screen.

    This issue prevents users from viewing Edge certificates. Without a UI fix, a user can view an Edge certificate by navigating to the Edge list on the Configuration tab and search for the desired Edge.

Known Issues

Open Issues in Release 5.4.0.

Edge/Gateway Known Issues

  • Issue 14655:

    Plugging or unplugging an SFP adapter may cause the device to stop responding on the Edge 540, Edge 840, and Edge 1000 and require a physical reboot.

    Workaround: The Edge must be physically rebooted. This may be done either on the Orchestrator using Remote Actions > Reboot Edge, or by power-cycling the Edge.

  • Issue 25504:

    Static route costs greater than 255 may result in unpredictable route ordering.

    Workaround: Use a route cost between 0 and 255.

  • Issue 25595:

    A restart may be required for changes to static SLA on a WAN overlay to work properly.

    Workaround: Restart Edge after adding and removing Static SLA from WAN overlay.

  • Issue 25742:

    Underlay accounted traffic is capped at a maximum of the capacity towards the VMware SD-WAN Gateway, even if that is less than the capacity of a private WAN link which is not connected to the Gateway.

  • Issue 25758:

    USB WAN links may not update properly when switched from one USB port to another until the VMware SD-WAN Edge is rebooted.

    Workaround: Reboot the Edge after moving USB WAN links from one port to another.

  • Issue 25885:

    A large configuration update on the Partner Gateway (e.g. 200 BGP-configured VRFs) may cause latency to increase for approximately 2-3 seconds for some traffic via the VMware SD-WAN Gateway.

    Workaround: No workaround available.

  • Issue 25921:

    VMware SD-WAN Hub High Availability failover takes longer than expected (up to 15 seconds) when there are three thousand branch Edges connected to the Hub.

    Workaround: No workaround available.

  • Issue 25997:

    The VMware SD-WAN Edge may require a reboot to properly pass traffic on a routed interface that has been converted to a switched port.

    Workaround: Reboot the Edge after making the configuration change.

  • Issue 26421:

    The primary Partner Gateway for any branch site must also be assigned to a VMware SD-WAN Hub cluster for tunnels to the cluster to be established.

    Workaround: No workaround available.

  • Issue 28175:

    Business Policy NAT fails when the NAT IP overlaps with the VMware SD-WAN Gateway interface IP.

    Workaround: No workaround available.

  • Issue 31210:

    VRRP: ARP is not resolved in the LAN client for the VRRP virtual IP address when the VMware SD-WAN Edge is primary with a non-global CDE segment running on the LAN interface.

    Workaround: No workaround available.

  • Issue 32731:

    Conditional default routes advertised via OSPF may not be withdrawn properly when the route is deactivated.

    Workaround: Reactivating the route, followed by deactivating it again will retract it successfully.

  • Issue 32960:

    Interface “Autonegotiation” and “Speed” status might be displayed incorrectly on the Local Web UI for activated VMware SD-WAN Edges.

    Workaround: Refer to the Orchestrator UI under Remote Diagnostics > Interface Status.

  • Issue 32981:

    Hard-coding speed and duplex on a DPDK-configured port may require a VMware SD-WAN Edge reboot for the configurations to take effect as it requires turning DPDK off.

    Workaround: There is no workaround for this issue.

  • Issue 34254:

    When a Zscaler CSS is created and the Global Segment has FQDN/PSK settings configured, these settings are copied to Non-Global Segments to form IPsec tunnels to a Zscaler CSS.

  • Issue 35778:

    When there are multiple user-defined WAN links on a single interface, only one of those WAN links can have a GRE tunnel to Zscaler.

    Workaround: Use a different interface for each WAN link that needs to build GRE tunnels to Zscaler.

  • Issue 36923:

    Cluster name may not be updated properly in the NetFlow interface description for a VMware SD-WAN Edge which is connected to that Cluster as its Hub.

  • Issue 38682:

    VMware SD-WAN Edge acting as a DHCP server on a DPDK-configured interface may not properly generate “New Client Device" events for all connected clients.

  • Issue 38767:

    When a WAN overlay that has GRE tunnels to Zscaler configured is changed from auto-detect to user-defined, stale tunnels may remain until the next restart.

    Workaround: Restart the Edge to clear the stale tunnel.

  • Issue 39374:

    Changing the order of VMware SD-WAN Partner Gateways assigned to a VMware SD-WAN Edge may not properly set Gateway 1 as the local Gateway to be used for bandwidth testing.

  • Issue 39608:

    The output of the Remote Diagnostic “Ping Test” may display invalid content briefly before showing the correct results.

    Workaround: There is not workaround for this issue.

  • Issue 39624:

    Ping through a subinterface may fail when the parent interface is configured with PPPoE.

    Workaround: There is no workaround for this issue.

  • Issue 39753:

    Toggling Dynamic Branch-to-Branch VPN to off may cause existing flows currently being sent using Dynamic Branch-to-Branch to stall.

    Workaround: Only deactivate Dynamic Branch-to-Branch VPN in a maintenance window.

  • Issue 40421:

    Traceroute is not showing the path when passing through a VMware SD-WAN Edge with an interface configured as a switched port.

  • Issue 40096:

    If an activated VMware SD-WAN Edge 840 is rebooted, there is a chance an SFP module plugged into the Edge will stop passing traffic even though the link lights and the VMware SD-WAN Orchestrator will show the port as 'UP'.

    Workaround: Unplug the SFP module and then replug it back into the port.

  • Issue 42278:

    For a specific type of peer misconfiguration, the VMware SD-WAN Gateway may continuously send IKE init messages to a Non-SD-WAN peer. This issue does not disrupt user traffic to the Gateway; however, the Gateway logs will be filled with IKE errors and this may obscure useful log entries.

  • Issue 42872:

    Activating Profile Isolation on a Hub profile where a Hub Cluster is associated does not revoke the Hub routes from the routing information base (RIB).

  • Issue 43373:

    When the same BGP route is learnt from multiple VMware SD-WAN Edges, if this route is moved from preferred to eligible exit in the Overlay Flow Control, the Edge is not removed from the advertising list and continues to be advertised.

    Workaround: Activate Distributed Cost Calculation (DCC) on the VMware SD-WAN Orchestrator.

  • Issue 44995:

    OSPF routes are not revoked from VMware SD-WAN Gateways and VMware SD-WAN Spoke Edges when the routes are withdrawn from the Hub Cluster.

  • Issue 45189:

    With source LAN side NAT is configured, the traffic from a VMware SD-WAN Spoke Edge to a Hub Edge is allowed even without the static route configuration for the NAT subnet.

  • Issue 45302:

    In a VMware SD-WAN Hub Cluster, if one Hub loses connectivity for more than 5 minutes to all of the VMware SD-WAN Gateways common between itself and its assigned Spoke Edges, the Spokes may in rare conditions be unable to retain the hub routes after 5 minutes. The issue resolves itself when the Hub regains contact with the Gateways.

  • Issue 46053:

    BGP preference does not get auto-corrected for overlay routes when its neighbor is changed to an uplink neighbor.

    Workaround: An Edge Service Restart will correct this issue.

  • Issue 46137:

    A VMware SD-WAN Edge running 3.4.x software does not initiate a tunnel with AES-GCM encryption even if the Edge is configured for GCM.

    Workaround: If a customer is using AES-256, they must explicitly deactivate GCM from the Orchestrator prior to upgrading their Edges to a 4.x Release. Once all their Edges are running a 4.x release, the customer may choose between AES-256-GCM and AES-256-CBC.

  • Issue 46216:

    On a Non SD-WAN Destinations via Gateway or Edge where the peer is an AWS instance, when the peer initiates Phase-2 re-key, the Phase-1 IKE is also deleted and forces a rekey. This means the tunnel is torn down and rebuilt, causing packet loss during the tunnel rebuild.

    Workaround: To avoid tunnel destruction, configure the Non SD-WAN Destinations via Gateway/Edge or CSS IPsec rekey timer to less than 60 minutes. This prevents AWS from initiating the re-key.

  • Issue 46391:

    For a VMware SD-WAN Edge 3800, the SFP1 and SFP2 interfaces each have issues with Multi-Rate SFPs (i.e. 1/10G) and should not be used in those ports.

    Workaround: Please use single rate SFP's per the KB article VMware SD-WAN SupportedSFP Module List (79270). Multi-Rate SFPs may be used with SFP3 and SFP4.

  • Issue 47664:

    In a Hub and Spoke configuration where Branch-to-Branch via Hub VPN is not configured, trying to U-turn Branch-to-Branch traffic using a summary route on an L3 switch/router will cause routing loops.

    Workaround: Configure Cloud VPN to activate Branch-to-Branch VPN and select “Use Hubs for VPN”.

  • Issue 47681:

    When a host on the LAN side of a VMware SD-WAN Edge uses the same IP as that Edge’s WAN interface, the connection from the LAN host to the WAN does not work.

  • Issue 48530:

    VMware SD-WAN Edge 6x0 models do not perform autonegotiation for triple speed (10/100/1000 Mbps) copper SFP's.

    Workaround: Edge 520/540 supports triple speed copper SFPs but this model has been marked for End-of-Sale by Q1 2021.

  • Issue 50518:

    On a VMware SD-WAN Gateway where PKI is configured, if >6000 PKI tunnels attempt to connect to the Gateway, the tunnels may not all come up because inbound SAs do not get deleted.

    Note:

    Tunnels using pre-shared key (PSK) authentication do not have this issue.

  • Issue 52955: DHCP decline is not sent from Edge and DHCP rebinding is not restarted after DAD failure in Stateful DHCP.

    If DHCPv6 server allocates an address which is detected as duplicate by the kernel during a DAD check then the DHCPv6 client does not send a decline. This will lead to traffic dropping as the interface address will be marked as DAD check failed and will not be used. This will not lead to any traffic looping in the network but traffic blackholing will be seen.

    Workaround: There is no workaround for this Issue.

  • Issue 53219: After a VMware SD-WAN Hub Cluster rebalances, a few Spoke Edges may not have their RPF interface/IIF set properly.

    On the affected Spoke Edges, multicast traffic will be impacted. What happens is that after a cluster rebalance, some of the Spoke Edge fail to send a PIM join.

    Workaround: This issue will persist until the affected Spoke Edge has an Edge Service restart.

  • Issue 53934: In an enterprise where a VMware SD-WAN Hub Cluster is configured, if the primary Hub has Multihop BGP neighborships on the LAN side, the customer may experience traffic drops on a Spoke Edge when there is a LAN side failure or when BGP is not configured on all segments.

    In a Hub cluster, the primary Hub has Multihop BGP neighborship with a peer device to learn routes. If the physical interface on the Hub by which BGP neighborship is established, goes down, then BGP LAN routes may not become zero despite BGP view being empty. This may cause Hub Cluster rebalancing to not happen. The issue may also be observed when BGP is not configured for all segments and when there are one or more Multihop BGP neighborships.

    Workaround: Restart the Hub which had the LAN-side failure (or BGP not activated).

  • Issue 57210: Even when a VMware SD-WAN Edge is working normally and is able to reach the internet, the LED in the Local UI's Overview page shows as "Red".

    The Edge's Local UI determines the Edge's connectivity by whether it can resolve a well known name via Google's DNS resolver (8.8.8.8). If it cannot do so for any reason, then it thinks it is offline and shows the LED as red.

    Workaround: There is no workaround for this issue, except to ensure that DNS traffic to 8.8.8.8 can reach the destination and be resolved successfully.

  • Issue 61543: If more than one 1:1 NAT rule is configured on different interfaces with the same Inside IP, the inbound traffic can be received on one interface and the outbound packets of the same flow can be routed via different interface.

    For the NAT flows from Outside to Inside, the 1:1 NAT rules will be matched against the Outside IP and the interface where the packets are received. For the outbound packets of the same flow, the VMWare SD-WAN Edge will try to match the NAT rules again comparing the Inside IP and the outbound traffic can be routed via the interface configured in the first matching rule with "Outbound Traffic" configured.

    Workaround: There is no workaround for this issue outside of ensuring no more than one 1:1 NAT rule is configured with a particular Inside IP address.

  • Issue 65560: Traffic from a customer to PE (Provider Edge) device fails.

    BGP neighborship between a Partner Gateway and Provider Edge does not get established when tag-type is selected as "none" on the handoff configuration. This is because ctag, stag values get picked from /etc/config/gatewayd instead of the handoff configuration on the Orchestrator when tag-type is "none".

    Workaround: Update the ctag, stag values to 0 each under vrf_vlan->tag_info in /etc/config/ gatewayd. Do a vc_procmon restart.

  • Issue 67879: A Cloud Security Service (CSS) tunnel is deleted after a user changes a WAN Overlay setting from auto-detect to user-defined on a WAN interface setting.

    After saving the changes, the CSS tunnels do not come back up until the customer takes down and then puts back up the tunnel. Changing the WAN configuration will bring down the CSS tunnel and parse the CSS setup again. However, in some corner cases, the nvs_config>num_gre_links is 0 and the CSS tunnel fails to come up.

    Workaround: Deactivate the CSS setup, and then reactivate it and this will bring the CSS tunnel up.

  • Issue 68057: DHCPv6 release packet is not sent from the VMware SD-WAN Edge on the changing of a WAN interface address mode from DHCP stateful to static IPv6 address and the lease remains active till reaching its valid time.

    The DHCPv6 client possesses a lease which it does not release when the configuration change is done. The lease remains valid till its lifetime expires in the DHCPv6 server and is deleted.

    Workaround: There is no way of remediating this issue as the lease would remain active till valid lifetime.

  • Issue 68851: If a VMware SD-WAN Edge and VMware SD-WAN Gateway each have the same TCP syslog server configured, the TCP connection is not established from the Edge to the syslog server.

    If the Edge and Gateway each have the same TCP server and if the syslog packets from the Edge are routed via the Gateway, the syslog server sends a TCP reset to the Edge.

    Workaround: Send the syslog packets direct from the Edge instead of routing via a Gateway or configure a different syslog server for the Edge and Gateway.

  • Issue 72762: For a customer enterprise site deployed with an Enhanced High Availability topology, the Remote Diagnostic > Traceroute does not work when run for the a WAN link connected to the Standby Edge.

    The Orchestrator UI shows the Standby Edge proxy WAN link when running the Remote Diagnostic > Interface Status, so the interface is up and link is recognized. In addition, a traceroute run for a Standby Edge proxy WAN link does not work when run from the Active Edge.

    Workaround: There is no workaround for this issue.

  • Issue 75171: An Edge client user does not receive an IP address if the DHCP server is configured on a Non SD-WAN Destination via Gateway site where the Edge acts as a relay agent.

    The issue is the result of the DHCP Offer messages being dropped by the Edge because the Edge code does not account for this DHCP configuration.

    Workaround: Locate the DHCP server any place other than a Non SD-WAN Destination.

  • Issue 81852: For a VMware SD-WAN Edge that is using a Zscaler type Cloud Security Service (CSS) which uses GRE tunnels that has turned on L7 Health Check, when that Edge is upgraded to Release 5.0.0, in some instances the customer may observe L7 Health Check errors.

    This is typically seen during software upgrade or during startup time. When L7 Health check for a CSS using GRE tunnels is turned on, error messages related to socket getaddress error may be seen. The observed error is intermittently seen, and not consistent. Because of this, L7 Health Check probe messages are not sent out.

    Workaround: Without the fix, to remediate the issue, a user needs to turn off and then turn back on the L7 Health Check configuration, and this feature would then work as expected.

  • Issue 82184: On a VMware SD-WAN Edge which is running Edge Release 5.0.0, when a traceroute or traceroute6 is run to the Edge's br-network IPv4/IPv6 address, the traceroute will not properly terminate when a UDP probe used.

    Traceroute or traceroute6 to the Edge's br-network IPv4/IPv6 address will not properly when Default Mode (in other words, UDP probe) is used.

    Workaround: Use -I option in traceroute and traceroute6 to use ICMP probe and then traceroute to br-network IPv4/IPv6 address will work as expected.

  • Issue 83166: When a VMware SD-WAN Gateway is freshly deployed with a AWS c5.4xlarge instance type from the AWS Portal with IPv6 option selected, neither IPv6 or the default routes are configured.

    As a result of IPv6 and default routes not being configured, the AWS Gateway IPv6 management tunnels are not forming and the Gateway will not work.

    Workaround: There is no workaround for this issue, avoid deploying a Gateway with the properties mentioned above.

  • Issue 85402: For a customer enterprise using BGP with Partner Gateways configured, a user may obseve that some BGP neighborships are down and this causes customer traffic issues.

    If a customer has maximum-prefix configured on a router which has BGP peering with the Edge and Gateway, the BGP session may be dropped by the router.

    For example, if the router has BGP configured to only receive max 'n' number of prefixes, but the Edge and Gateway have more than 'n' number of prefixes to be advertised in the absence of any filters. Now if the BGP filter configuration is changed on the Orchestrator, even if the total number of prefixes allowed in the outbound direction is less than 'n', the issue will be encountered where more than 'n' prefixes are sent to the peer before any filters are applied. This causes the router to tear down the session.

    Workaround: If BGP goes down due to this issue (Maximum Number of Prefixes Reached), flap BGP on the peer using CLI (For FRR/Cisco, "neighbor x shut" followed by "no neighbor x shut"), and the BGP will produce only the desired number of prefixes advertised to the peer.

  • Issue 95399: When an Ethernet link is physically unplugged from, or plugged into a VMware SD-WAN Edge interface, the VMware SASE Orchestrator does not receive a notification of this event from the Edge and does not show this in customer Events.

    Customers rely on the Orchestrator to report these events on the Events page and when they are not being recorded this makes monitoring their various sites more unreliable.

    Workaround: There is no workaround for this issue.

  • Issue 98136: For customer enterprises using a Hub/Spoke topology where Dynamic Branch To Branch VPN is configured, client users behind a SD-WAN Spoke Edge may observe that some traffic has unexpected latency resulting from the traffic using a sub-optimal path.

    Spoke Edge traffic that experiences this issue uses a route that was initially a non-uplink route for a Hub Edge not included in the Profile the Spoke Edge was using. A Dynamic Branch-to-Branch VPN tunnel can be formed from the Spoke Edge to the Hub Edge because of traffic being sent towards some other unrelated prefix and in this instance the non-uplink route is installed in the Spoke Edge.

    As a result of this non-uplink route, all traffic towards this prefix starts going through the

    Hub Edge and the non-uplink route becomes uplink (community change to uplink community) but the non-uplink route installed previously is not revoked and the traffic takes the Hub Edge path as long as the Dynamic Branch-to-Branch VPN tunnel remains up.

    Workaround: Wait for the Dynamic Branch-to-Branch VPN tunnel to tear down, after which the uplink route will not be installed in the Spoke Edge when a new Dynamic Branch-to-Branch VPN tunnel is formed towards the Hub Edge.

  • Issue 102583: For a customer enterprise site deployed with a High Availability topology with VNFs installed, where the HA Edge pair or either Edge models 520/540 or 610, reachability to the Standby Edge's VNF from a LAN-connected client may fail.

    The Ethernet switch board on these Edge models drops ARP reply packets sent to a LAN client from a Standby Edge's VNF resulting in a loss of reachability.

    Workaround: There is no workaround for this issue.

  • Issue 106289: A VMware SD-WAN Hub Edge may drop packets on flows to connected Spoke Edges or backhaul flows.

    The Backhaul flow flag is set during the QoS synchronization process, there is a place in the code where it sets it during flow creation. The Edge should set this flag only as a result of QoS synchronization message processing.

    Workaround: Should this issue be encountered on a Hub Edge without a fix, flush the flows on the Hub Edge to temporarily remediate the issue.

  • Issue 110561: Dynamic tunnels may not come up between the same set of VMware SD-WAN Edges with bidirectional traffic when traffic stops and then restarts.

    Issue is observed in a test environment where there are 6000 dynamic tunnels with high bidirectional traffic being sent between the Edges. Even in lower scale testing at 1000 dynamic tunnels, not all the tunnels come up.

    Workaround: There is no workaround for this issue.

  • Issue 111085: When a VMware SD-WAN Edge's WAN link is configured with an IP address in the same network as the Edge's loopback IP, the Edge uses the MAC address of the WAN interface while responding to an ARP request for the Edge's loopback IP address.

    This can cause ARP spoof and results in the Management IP being deprecated and network disruptions as a result.

    Workaround: There is no workaround for this issue.

  • Issue 111592: For a customer enterprise using a Hub/Spoke topology where Business Policies are configured to use internet backhaul, internet traffic using the backhaul rule may be either slow or not work at all.

    In some instances during the creation of the flow, the Business Policy matching is changed due to updated Deep Packet Inspection (DPI) information. This could lead to the loss of the logical ID of the Hub Edge or Non SD-WAN Destination, which is supposed to backhaul the packets.

    Workaround: Flush flows to force the flows to be reinspected by the DPI engine.

  • Issue 117876: In a customer site using a High Availability topology, if a user activates or deactivates the Enhanced Firewall Services, a VMware SD-WAN HA Edge may experience multiple restarts.

    When Enhanced Firewall Services is activated or deactivated, only the Active Edge's Device Settings configuration is synchronized immediately with the Standby Edge, with the remainder of the configuration synchronization is only in response to a Standby Edge heartbeat. When the Active Edge is restarted to apply the latest configuration prior to receiving a heartbeat from the Standby Edge it will result in a configuration mismatch between the two HA Edges and they will undergo multiple restarts to complete the configuration synchronization.

    Workaround: The only workaround is to turn on or off Enhanced Firewall Services during a maintenance window for HA Edges.

  • Issue 111840: On a customer enterprise deployed with a Hub/Spoke topology which uses 9 or more Hub Edges, client users may experience poor traffic quality due to suboptimal routing.

    When a Spoke Edge is configured with multiple Hub Edges, a route via the Hub Edge is preferred over the direct route, leading to suboptimal routing.

    Workaround: Configure Hub Edges first, followed by VPN Hubs in the Branch to Hub site list.

  • Issue 118704: A user may observe abnormally high latency values for paths measured between SD-WAN Edges and SD-WAN Gateways even though actual Edge-to-Gateway packet latency is much lower.

    A race condition has been identified with clock synchronization resulting in latency values measured incorrectly. This issue is cosmetic and there is no performance impact to customer traffic but it does negatively impact a customer's ability to properly monitor Edge links and paths.

    Workaround:Clock synchronization can be reset by restarting the Edge Service. This can be done using the Orchestrator UI by navigating to the Diagnostics > Remote Actions page, and then checking the affected Edge and selecting the Restart Service option.

  • Issue 120309: Combinations of certain PPTP VPN Clients and Servers may not be able to immediately re-establish a connection after it is interrupted when using 1:1 NAT with the PPTP Internet and client device behind the Edge. This issue has been confirmed to occur with Windows Server 2016 and Windows 10 client but may occur with other versions as well.

    The issue is caused by the server reusing the same PPTP call-id for the new connection while the client uses a new call-id. When the server call-id is reused for the new connection, the firewall does not identify it as a new connection.

    On an Edge without a fix for this issue, the workaround is to flush the stale PPTP connection from the NAT table.

    Note:

    The same issue may be encountered with the network locations of the client and server swapped (in other words, the PPTP server on the LAN behind the Edge and client in the Internet/Cloud). This version of the issue is tracked by ticket #109830. The fix here specifically addresses Windows Server 2016 clients. Windows 10 clients still exhibit this issue and is not addressed by this fix.

    Workaround: Flush the stale PPTP connection from the NAT table.

  • Issue 121281: For a customer enterprise site deployed with a High Availability topology, in rare instances the customer may observe that the Standby Edge has experienced a Dataplane Service failure, generated a core, and restarted to recover.

    There would be an Event about the Standby Edge restarting and if the user configures an HA Failed alert, they would be notified of this event. The issue is experienced in rare scenarios when a route is synchronized between the Active and Standby and the Standby Edge service fails due to memory corruption while processing the route synchronization message.

    Workaround: There is no workaround for this issue.

  • Issue 121371: For a customer enterprise configured to use Hub or Cluster Interconnect where the Stateful Firewall or Enhanced Firewall is also used, client users may observe traffic drops.

    Asymmetric routing can occur between the source and destination Cluster Hub nodes such that outbound traffic takes a direct overlay tunnel while the return traffic takes an underlay route.

    Workaround: The only workaround is to deactivate Stateful Firewall or Enhanced Firewall Services.

  • Issue 121606: For customers connected to a Partner Gateway, they may observe that IPsec VPN tunnels are down on the Partner Gateway.

    The Gateways IPsec process supports a maximum of 64 IP addresses on an interface. For this issue on a Partner Gateway, the hand-off IP addresses are getting added to the Gateway's IPsec process interface unconditionally. If the number of hand-off IP addresses exceeds the 64 limit, the older IP addresses are overwritten in the IPsec process, and this causes the tunnels using those IPs to go down.

    Workaround: When this issue is experienced on a Gateway without a fix for this issue, there is no real workaround assuming all the PG handoff IP addresses are configured as expected. Otherwise, removing unnecessary PG handoff IPs may help if it reduces the IPs on the Gateway's IPsec process to below 64. A Gateway service restart will be needed after this configuration change.

    If all handoff IPs are valid, the only alternative is to move a number of IPsec tunnels to a second Partner Gateway sufficient to drop the total number of handoffs below 64 on the affected PG.

    There is a Gateway hotfix build: R5401-20240805-GA-121606 that resolves the issue and if you require this, contact the Support team.

  • Issue 121805: For a VMware SD-WAN Edge deployed with VNF's, when a local VNF is pinged, the user may observe duplicate replies.

    The issue is experienced when a ping to a VNF's IP is done from the Edge.

    Workaround: Always ping from a LAN client behind the Edge, not the Edge itself.

  • Issue 123379: For a customer enterprise site deployed with a High Availability topology, in rare instances the customer may observe the Standby Edge experience a Dataplane Service failure and restart to recover.

    This issue can occur when a user tries to configure IPv6 address on subinterfaces across 128 segments simultaneously using a script. In this scenario the configurations get accumulated in a queue and this causes the Standby Edge's service to fail.

    Workaround: It is advised to configure IPv6 address configurations on subinterfaces in smaller groups so that the system has time to process and apply the configurations.

  • Issue 125509: A customer enterprise using lower end VMware SD-WAN Edge models may experience flaps for BFD, BGP, or OSPF, depending on the routing protocol being used.

    On entry level Edge platforms (510, 520, 540, 610, and 620) at a high flow scale and coupled with dynamic routing and/or High Availability configuration, OSPF/BGP routing flaps may be observed when aggressive Hello and Dead interval timers are configured. In addition, if the customer also uses Edge Network Intelligence with Analytics turned on, the potential to encounter this issue increases.

    Workaround: If experiencing this issue, the workaround is to revert to default interval timers for OSPF (10, 40) or BGP (60, 180) or disable BFD entirely.

  • Issue 127920: An ICMP probe attached to a static route with a secondary IP as the next hop may go down, even though the secondary IP address is up and reachable.

    When an ICMP probe is attached to a static route with a secondary IP as a next hop, the probe is sourcing the main interface IP address instead of the secondary IP address. When the peer is unable to reach the primary IP address, it results in the ICMP probe going down though the secondary IP is up and reachable.

    Workaround: There is no workaround in this scenario beyond not using a secondary IP address.

  • Issue 128379: A BGP summary route that a Spoke Edge advertises to a Hub Edge through an MPLS backbone may go into a loop of withdrawal and advertisement.

    The sequence that leads to this issue is as follows:

    1. The Spoke Edge adds its Autonomous System Number (ASN) and advertises the BGP summary prefix to the customer edge router (CE). Then CE adds its ASN and advertises it to the Hub Edge.

    2. The Hub Edge redistributes the summary prefix into the overlay (using the SD-WAN management protocol VCRP) with all the ASNs it received. The Hub Edge also redistributes the prefixes it received over the uplink.

    3. The Spoke Edge receives the same summary prefix over VCRP with CE's ASN as well. The Spoke Edge redistributes this overlay prefix into BGP. Since as-set is enabled in the aggregate configuration, BGP collects ASNs from the overlay prefix and adds them to the AS_PATH of the summary prefix.

    4. Now CE receives the summary prefix with the updated AS_PATH. Since CE's ASN is part of the updated AS_PATH, CE rejects the summary prefix and withdraws it from the Hub Edge. The Hub Edge then withdraws it from the Spoke Edge over VCRP.

    5. Now the Spoke Edge changes the AS_PATH of the BGP summary prefix back to normal and advertises it again. This cycle repeats forever.

    Workaround: There is no workaround for this issue.

  • Issue 138773: An attempt to perform a packet capture for a Gateway Interface through the Orchestrator UI may not work if the Gateway is using software version 5.4.x.

    The issue is caused by complications with the 5.4.x Gateway's kernel security module (AppArmor) and file download limitations.

    Workaround: A user has the following options:

    • Option 1: Use Gateway CLI.

      • On Gateway CLI > tcpdump.sh

    • Option 2: Update AppArmor profile and tcpdump location (this restores Orchestrator UI functionality).

      • On the Gateway:

        • a] update file /etc/apparmor.d/mgd

          • Add these lines in the profile mgd:

            • At line 26 of the profile insert: /opt/vc/bin/vctcpdump mrUx,

            • At line 39 of the profile insert: /usr/sbin/tcpdump mrUx,

        • b] service apparmor restart

        • c] verify in /var/log/audit/audit.log, NO "DENIED" for tcpdump

  • Issue 142366: For a customer enterprise site connected to Partner Gateways where one or more static routes are configured, client users working behind an SD-WAN Edge may observe intermittent traffic loss if a static route via the Primary Partner Gateway is unreachable.

    When the same static route is reachable via two or more Partner Gateways, if the route via the Partner Gateway in the Primary role is unreachable, traffic from an Edge can experience intermittent traffic loss. This issue is the result of the Edge API failing to properly check for reachability on a route lookup which causes the Edge to continue to use the Primary Partner Gateway even though reachability is false.

    Workaround: The issue can be temporarily remediated by the Partner shutting down the Primary Partner Gateway until the static route becomes reachable again. Shutting down the Primary Partner Gateway prevents the Edge from including it in route reachability lookups and ensures traffic matching that static route uses a secondary Partner Gateway. However this can be disruptive for all customers using this Partner Gateway as their Primary Gateway and should be done in a maintenance window by the Partner if possible.

  • Issue 143450: On a customer enterprise site configured with an Enhanced High Availability topology where Dynamic Branch to Branch VPN is also enabled, client users may observe extended traffic loss after an HA failover.

    The issue can be encountered if the Enhanced HA site also has a Business Policy rule configured which includes mandatory link steering. Combined with Dynamic Branch to Branch, this combination can result in a prolonged period of traffic disruption after an HA failover.

    Workaround: A customer can either remove the Business Policy rule with mandatory link steering entirely, or modify that rule to remove the mandatory link steering option.

  • Issue 143551: A Virtual Edge deployed on Azure which uses Accelerated Networking may experience multiple Dataplane Service failures and be unable to pass traffic when it is upgraded to Release 5.4.0.1, and this state cannot be recovered.

    Edge Release 5.4.0.1 does not properly implement support for Azure Accelerated Networking, and when an Azure Virtual Edge using Accelerated Networking is upgraded to this release, an exception is triggered and the Edge service fails, and then restarts three times in succession. After the third service failure, the Edge service stops entirely. A reboot does not recover the Edge as the issue will persist and restart the cycle of service failures/restarts all over again.

    A customer with an Azure Virtual Edge that does not use Accelerated Networking is not at risk for this issue.

    Workaround: There is no workaround for this issue, and a customer using Accelerated Networking should not upgrade an Azure Virtual Edge to 5.4.0.1.

  • Issue 143828: A customer may observe that an Edge has an unexpectedly high level of memory usage that may get sufficiently high to reach a critical level and trigger a defensive Edge service restart to recover the memory.

    One of the factors that contributes to this memory leak is extensive use of CLI commands on the Edge by either a Partner or Operator as part of troubleshooting or monitoring the Edge. These commands are accumulated and never cleared from the relevant Edge process. The use of Remote Diagnostics on the Orchestrator UI does not contribute to this issue.

    As with any Edge memory usage issue, entry level Edge models with lower RAM specifications (in other words, the Edge models 510, 610, 710, or 520) would be more likely to experience the issue, but it can happen on any Edge model with sufficiently high memory usage.

    Workaround: Extensive use of CLI commands on the Edge by either an Operator or Partner should be avoided, and if troubleshooting work is done, the customer should check the Edge's memory usage on the Edge > Monitor > System page.

  • Issue 145393: A customer enterprise site deployed with an Edge model 620, 640, or 680 where firewall logging is configured may observe that the Edge no longer stores new firewall or standard debugging logs.

    When this issue is encountered, a 6x0 Edge's eMMC storage experiences an excessive level of wear due to the high volume of writes and rewrites that can be triggered by enabling logging for firewall rules which are matched by a large number of new connections per second in a high traffic customer environment. This issue results in the Edge defensively moving the file partition which hosts logging to a read-only state, and no additional logs are stored.

    Workaround: If a customer has an Edge 620, 640, or 640 Edge model and is also using firewall logging, they should avoid enabling logging for firewall rules which can potentially match a large number of new connections in a high traffic environment. The excessive logging frequency that would result can cause undue wear on the Edge's storage and trigger this issue.

Orchestrator Known Issues

  • Issue 21342:

    When assigning Partner Gateways per-segment, the proper list of Gateway Assignments may not show under the Operator option "View" Gateways on the VMware SD-WAN Edge monitoring list.

  • Issue 25932:

    The VMware SD-WAN Orchestrator allows VMware SD-WAN Gateways to be removed from the Gateway Pool even when they are in use.

  • Issue 32335:

    The ‘End User Service Agreement’ (EUSA) page throws an error when a user is trying to accept the agreement.

    Workaround: Ensure no leading or trailing spaces are found in Enterprise Name.

  • Issue 32435:

    A VMware SD-WAN Edge override for a policy-based NAT configuration is permitted for tuples which are already configured at the profile level and vice versa.

  • Issue 32856:

    Though a business policy is configured to use the Hub cluster to backhaul internet traffic, the user can unselect the Hub cluster from a profile on a VMware SD-WAN Orchestrator that has been upgraded from Release 3.2.1 to Release 3.3.x.

  • Issue 35658:

    When a VMware SD-WAN Edge is moved from one profile to another which has a different CSS setting (e.g. IPsec in profile1 to GRE in profile2), the Edge level CSS settings will continue to use the previous CSS settings (e.g. IPsec versus GRE). 

    Workaround: At the Edge level, deactivate GRE, and then reactivate GRE to resolve the issue.

  • Issue 35667:

    When a VMware SD-WAN Edge is moved from one profile to another profile which has the same CSS setting but a different GRE CSS name (the same endpoints), some GRE tunnels will not show in monitoring.

    Workaround: At the Edge level, deactivate GRE and then reactivate GRE to resolve the issue.

  • Issue 36665:

    If the VMware SD-WAN Orchestrator cannot reach the internet, user interface pages that require accessing the Google Maps API may fail to load entirely.

  • Issue 32913:

    After activating High Availability, multicast details for the VMware SD-WAN Edge are not displayed on the Monitoring Page. A failover resolves the issue.

  • Issue 33026:

    The ‘End User Service Agreement’ (EUSA) page does not reload properly after deleting the agreement.

  • Issue 38056:

    The Edge-Licensing export.csv file not show region data.

  • Issue 38843:

    When pushing an application map, there is no Operator event, and the Edge event is of limited utility.

  • Issue 39633:

    The Super Gateway hyperlink does not work after a user assigns the Alternate Gateway as the Super Gateway.

  • Issue 39790:

    The VMware SD-WAN Orchestrator allows a user to configure a VMware SD-WAN Edge’s routed interface to have greater than the supported 32 subinterfaces, creating the risk that a user can configure 33 or more subinterfaces on an interface which would cause a Dataplane Service Failure for the Edge.

  • Issue 41691:

    User cannot change the 'Number of addresses' field although the DHCP pool is not exhausted on the Configure > Edge > Device page.

  • Issue 43276:

    User cannot change the Segment type when a VMware SD-WAN Edge or Profile has a Partner Gateway configured.

    Workaround: Temporarily remove the Partner Gateway configuration from the Profile or Edge so that the Segment can be changed between private and regular. Alternatively, the user can remove the Segment from the profile and make the change from there.

  • Issue 47713:

     If a Business Policy Rule is configured while Cloud VPN is toggled off, the NAT configuration must be reconfigured upon turning on Cloud VPN.

  • Issue 47820:

    If a VLAN is configured with DHCP toggled off at the Profile level, while also having an Edge Override for this VLAN on that Edge with DHCP activated, and there is an entry for the DNS server field set to none (no IP configured), the user will be unable to make any changed on the Configure > Edge > Device page and will get an error message of ‘invalid IP address []’ that does not explain or point to the actual problem.

  • Issue 48085: The VMware SD-WAN Orchestrator allows a user to delete a VLAN which is associated with an interface.

    When encountering this issue, the user would see an error message similar to "VLAN ID [xx] cannot be removed, in use by edge [b1-edge1 (GEx-disabled]".

  • Issue 51722: On the VMware SASE Orchestrator, the time range selector is no greater than two weeks for any statistic in the Monitor > Edge tabs.

    The time range selector does not show options greater than "Past 2 Weeks" in Monitor > Edge tabs even if the retention period for a set of statistics is much longer than 2 weeks. For example, flow and link statistics are retained for 365 days by default (which is configurable), while path statistics are retained only for 2 weeks by default (also configurable). This issue is making all monitor tabs conform to the lowest retained type of statistic versus allowing a user to select a time period that is consistent with the retention period for that statistic.

    Workaround: A user may use the "Custom" option in the time range selector to see data for more than 2 weeks.

  • Issue 60522: On the VMware SD-WAN Orchestrator UI, the user observes a large number of error messages when they try to remove a segment.

    The issue can be observed when adding a segment to a profile and the associating the segment with multiple VMware SD-WAN Edges. When the user attempts to remove the added segment from the profile, they will see a large number of error messages.

    Workaround: There is no workaround for this issue.

  • Issue 82680: For customer using MT-GRE Tunnel Automation, when a user turns off the Cloud-to-Cloud Interconnect (CCI) flag on a VMware SD-WAN Gateway which is configured to use CCI, the Zscaler MT-GRE entries may not get deleted from the Zscaler portal consistently.

    After a CCI site has been deleted from the Gateway, the entries for this site should also be removed. This issue has only been seen during test automation and has not been reproduced manually, but remains a risk.

    Workaround: Manually delete the resource from Zscaler before retrying.

  • Issue 82681: For customer using MT-GRE Tunnel Automation, when a user turns off the Cloud-to-Cloud Interconnect (CCI) flag on a VMware SD-WAN Gateway which is configured to use CCI, and the user deactivates the CCI flag from a VMware SD-WAN Edge with CCI configured which is using a Zscaler Cloud Security Service, the Zscaler MT-GRE entries may not get deleted from the Edge or from the Zscaler portal.

    After a CCI site has been deleted from the Gateway, the entries for this site should also be removed. This issue has only been seen during test automation and has not been reproduced manually, but remains a risk.

    Workaround: Manually delete the resource from Zscaler before retrying.

  • Issue 103769: An Operator may observe that a VMware SASE Orchestrator in a large scale deployment is experiencing performance issues which include 100% disk utilization and the Orchestrator no longer accumulating logs.

    This issue arises a change in logging behavior for the 5.1.0 Orchestrator that may result in the folders that store logs becoming full and also causing the Orchestrator CPU to reach 100% utilization. This issue arises a change in logging behavior for the 5.1.0 Orchestrator that may result in the folders that store logs becoming full and also causing the Orchestrator CPU to reach 100% utilization.

    Workaround: A Superuser Operator needs to log into the Orchestrator and clean up the pending logs.

  • Issue 114475: When an Operator attempts to upgrade a VMware SASE Orchestrator from Release 4.2.0 to 5.1.0, the Orchestrator may report that the upgrade failed.

    In the logs the Operator would observe this entry: Error while initializing CWS Server service Error: Too many connections.

    This issue is triggered by MySQL restarting before vco-db-schema is installed which is caused by MySQL not setting the maximum number of connections. In addition, while the Orchestrator reports that the installation failed, in reality the installation did finish, and they can restart the Orchestrator and all services will work as expected.

  • Issue 117699: An Operator attempting to upgrade a 4.2.x VMware SD-WAN Orchestrator to become a Release 5.2.0 SASE Orchestrator may observe that the upgrade fails.

    The upgrade does not succeed, effectively stuck at the "Waiting for the CWS service up...". This issue is limited to 4.2.x Orchestrators.

    Workaround: The workaround for this issue is to upgrade the 4.2.x Orchestrator to 4.5.1 first, and then to Release 5.2.0.0.

  • Issue 124568: For a VMware SASE Orchestrator deployed with a Disaster Recovery (DR) topology, in rare instances the Operator User may observe that DR halts, leading to a temporary lack of replication between the active and standby Orchestrators.

    The user experience is not affected as this is isolated to DR only. The DR page will indicate an error state instead of STANDBY_RUNNING.

    Workaround: This is an intermittent error and will auto-recover.

  • Issue 125504: If a static route is configured with next hop as a VLAN with IPv4/IPv6 address at the Profile level and then overridden at the Edge level and add an IPv4/IPv6 address to the VLAN, the static route is not marked as N/A and the VMware SASE Orchestrator asks for the interface in a dropdown menu.

    The expected behavior is where a static route configured with a next hop as a VLAN with IPv4/IPv6 address, the Orchestrator does not ask for the interface and the route is marked as N/A.

    Workaround: There is no workaround for this issue.

  • Issue 125663: A user can configure the same IPv4/IPv6 IP address for multiple Edge interfaces.

    The VMware SASE Orchestrator is allowing a user to configure the same IP on multiple WAN, LAN, or Sub Interfaces.

    Workaround: There is no workaround for this issue beyond ensuring you are not configuring the same IP Address for multiple interfaces.

  • Issue 126421: For Partners using a Partner Gateway, when configuring the Hand Off Details, the "Use for Private Tunnels" option is always checked no matter what a user does.

    This is not a cosmetic issue as the Orchestrator will apply the Use for Private Tunnels configuration to the Partner Gateway handoff and can impact customer traffic using the Partner Gateway.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 126425: When looking at Configure > Device > Routing & NAT page at the Profile level, the OSPF On/Off toggle button is missing.

    The OSPF On/Off toggle button was not migrated to the New UI at the Profile level and only shows at the Edge level.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 126465: The VMware SASE Orchestrator UI is not applying changes a user makes to create an Edge Cluster.

    If a user goes to the Configure > Edge > High Availability section of the UI and turns on HA with a Cluster type and creates a Hub Cluster with name xxxx, and saves changes, the user would observe that post-save the Cluster option is not selected under HA section and the created Hub Cluster with name xxxx is not present.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 127152: Users cannot save modified Interfaces with OSPF configurations on the VMware SASE Orchestrator UI.

    At the Profile level, when configuring OSPFv2 or OSPFv3, the Edit Interface dialog becomes invalid after changing any OSPF data.

    Workaround: On an Orchestrator without a fix for this issue, a user would need to activate MD5 Authentication and change the Key ID to any number from 1 to 255, and then deactivate MD5 Authentication.

  • Issue 130115: For a VMware SASE Orchestrator configured with a Disaster Recover (DR) topology, the Active and Standby Orchestrator's DR pages show different details under the History section.

    The user sees additional rows for a failing DR state on the Active Orchestrator compared to the Standby Rows under the History section and these rows are not sorted by time on the Active Orchestrator.

  • Issue 131846: On the Global Settings > Customer Configuration > Partner Hand off > Hand Off Interface page of the Orchestrator UI, when a user clicks the Add button to add a static route, the UI returns an error message.

    When the user clicks the Add button in the static routes section, a new empty row is added to the table, but an error message reading "Cannot save changes. There is one or more errors in your configuration" is displayed. This is caused by a missing validation for the empty static route configuration on the Partner Handoff section of the Global Settings screen, and this issue only affects rows where no information is configured.

    Workaround: There are two workarounds for this problem:

    • Fill in all the mandatory fields. By doing so, the error message ("Cannot save changes...") will be automatically resolved.

    • Remove any newly added empty rows from the section. After removing these empty rows, the error message will be automatically resolved.

  • Issue 131997: An ICMP probe may be erroneously marked as down when it is configured for one segment but not for another.

    The Orchestrator fails to send the NULL ICMP probe configuration when it is not configured in a segment. As a result, the configuration of a different segment is reused and this causes the probes to fail.

    Workaround: On an Orchestrator without a fix for this issue, configure a dummy ICMP probe for the other segments.

  • Issue 133092: For a customer enterprise that deploys one or more Non SD-WAN Destinations via Gateway with redundant Gateways where L7 Health check is enabled, a user may observe the wrong service status for this NSD when looking at Montior > Network Services.

    There is no functional impact for the NSD as this issue is cosmetic. The issue can occur if each Gateway (Primary and Secondary) has a different L7 Health status and there can be a potentially false status when looking at the UI. For example, if the Primary VPN tunnel is down, the user may see the UI report that the NSD is completely down, when in fact the Secondary VPN tunnel is now passing customer traffic to the peer datacenter.

    Workaround: There is no workaround to show the correct status in the UI, but an Operator or Partner can see the correct service status of each VPN in the Gateway if they log into the Gateway via SSH and run debug.py --ike.

check-circle-line exclamation-circle-line close-line
Scroll to top icon