VMware SASE 6.0.0 | 18 September 2024

  • VMware SASE™ Orchestrator Version R6005-20240815-GA

  • VMware SD-WAN™ Gateway Version R6001-20240709-GA

  • VMware SD-WAN™ Edge Version R6001-20240709-GA

Check for additions and updates to these release notes.

What Is in The Release Notes

The release notes cover the following topics:

This release is recommended for all customers who require the features and functionality first made available in Release 6.0.0.

Important:

Release 6.0.0 contains all fixes found in the 5.4.0 Release Notes as follows:

  • All Edge fixes up to build R5401-20240224-GA.

  • All Gateway fixes up to build R5401-20240314-GA-141087.

  • All Orchestrator fixes up to build R5406-20240323-GA.

6.0.0 is a Short-Term Support (STS) Release

VMware SD-WAN/SASE has introduced a Long-Term Support (LTS) policy to enhance the operational efficiency of our partners and customers during the implementation of new software. Going forward, releases will be identified as either LTS or Short-Term Support (STS).

As part of this new policy, SASE/SD-WAN Release 6.0.0 is offered as an STS Release. As an STS Release, the support life for Release 6.0.0 is one year from the GA date, which means the End of Support Life for 6.0.0 Orchestrator/Gateway/Edge is 05/11/2025.

For additional information about Long-Term Support and Short-Term Support releases, see: VMware SD-WAN/SASE Long-Term Support Release (96246).

Compatibility

Release 6.0.0 Orchestrators, Gateways, and Hub Edges support all previous VMware SD-WAN Edge versions greater than or equal to Release 4.2.0.

The following SD-WAN interoperability combinations were explicitly tested:

Orchestrator

Gateway

Edge

Hub

Branch/Spoke

6.0.0

5.0.1

4.2.2

4.2.2

6.0.0

6.0.0

4.2.2

4.2.2

6.0.0

5.1.0

4.3.2

4.3.2

6.0.0

6.0.0

4.3.2

4.3.2

6.0.0

4.5.2

4.5.2

4.5.2

6.0.0

6.0.0

4.5.2

4.5.2

6.0.0

6.0.0

6.0.0

4.5.2

6.0.0

6.0.0

4.5.2

6.0.0

6.0.0

5.0.1

5.0.1

5.0.1

6.0.0

6.0.0

5.0.1

5.0.1

6.0.0

6.0.0

6.0.0

5.0.1

6.0.0

6.0.0

5.0.1

6.0.0

6.0.0

5.1.0

5.1.0

5.1.0

6.0.0

6.0.0

5.1.0

5.1.0

6.0.0

6.0.0

6.0.0

5.1.0

6.0.0

6.0.0

5.1.0

6.0.0

6.0.0

5.2.2

5.2.2

5.2.2

6.0.0

6.0.0

5.2.2

5.2.2

6.0.0

6.0.0

6.0.0

5.2.2

6.0.0

6.0.0

5.2.2

6.0.0

6.0.0

5.2.3

5.2.3

5.2.3

6.0.0

6.0.0

5.2.3

5.2.3

6.0.0

6.0.0

6.0.0

5.2.3

6.0.0

6.0.0

5.2.3

6.0.0

6.0.0

5.4.0

5.4.0

5.4.0

6.0.0

6.0.0

5.4.0

5.4.0

6.0.0

6.0.0

6.0.0

5.4.0

6.0.0

6.0.0

5.4.0

6.0.0

6.0.0

6.0.0

6.0.0

6.0.0
Important:

VMware SD-WAN Release 4.0.x has reached End of Support; Releases 4.2.x, 4.3.x, and 4.5.x have reached End of Support for Gateways and Orchestrators.

  • Release 4.0.x reached End of General Support (EOGS) on September 30, 2022, and End of Technical Guidance (EOTG) December 31, 2022. 

  • Release 4.2.x Orchestrators and Gateways reached End of General Support (EOGS) on December 30, 2022, and End of Technical Guidance on (EOTG) March 30, 2023.   

  • Release 4.2.x Edges reached End of General Support (EOGS) on June 30, 2023, and will reach End of Technical Guidance (EOTG) September 30, 2025.

  • Release 4.3.x Orchestrators and Gateways reached End of General Support (EOGS) on June 30, 2023, and End of Technical Guidance (EOTG) September 30, 2023.

  • Release 4.3.x Edges reached End of General Support (EOGS) on June 30, 2023, and will reach End of Technical Guidance (EOTG) September 30, 2025.

  • Release 4.5.x Orchestrators and Gateways reached End of General Support (EOGS) on September 30, 2023, and End of Technical Guidance on (EOTG) December 31, 2023.

  • For more information please consult the Knowledge Base article: Announcement: End of Support Life for VMware SD-WAN Release 4.x (88319).

Upgrade Paths for Orchestrator, Gateway, and Edge

The following lists the paths for customers wishing to upgrade their Orchestrator, Gateway, or Edge from an older release to Release 6.0.0.

Orchestrator

Orchestrators using Release 5.4.0 or later can be only directly upgraded to Release 6.0.0. 

Gateway

Upgrading a Gateway using Release 5.0.0 or later to Release 6.0.0 is fully supported for all Gateway types.

Important:

When deploying a new Gateway using 6.0.0 the VMware ESXi instance must be either version 6.7, Update 3; version 7.0, Update 3; or version 8.0, Update 1a. Using an earlier ESXi instance will result in the Gateway's Dataplane Service failing when trying to run Release 6.0.0 or later.

Important:

Prior to upgrading a Gateway to 6.0.0, the ESXi instance must be upgraded to either version 6.7, Update 3; version 7.0, Update 3; or version 8.0, Update 1a. Using an earlier ESXi instance will result in the Gateway's Dataplane Service failing when trying to run Release 6.0.0 or later.

Edge

An Edge can be upgraded directly to Release 6.0.0 from Release 4.5.x or later.

New Features and Enhancements

Enhanced Firewall Service Powered by NSX™ Technology

The Enhanced Firewall Service in software version 6.0.0 features URL Filtering, and Malicious IP Filtering in addition to the already available Intrusion Detection System/Intrusion Prevention System (IDS/IPS).

The URL Filtering and Malicious IP Filtering services are powered by VMware’s award-winning NSX Security components, empowering IT administrators to reduce their system’s attack surface. Through URL Filtering, web traffic is filtered based on Category and Reputation. Integrating the capabilities of NSX Security with VMware SD-WAN Edge platforms enables clients to confidently remove legacy firewalls at branch locations without compromising security and experiencing the benefits of streamlined network and security operations. Additionally, clients leverage VMware's investment in threat intelligence.

Security Services Group

Security Service Groups are organized collections of security service settings that are offered as part of the Enhanced Firewall Services. These settings include URL Category, URL Reputation, Malicious IP, and IDS/IPS. These groups are designed to simplify firewall policy management by enabling the creation and reuse of predefined security service configurations across multiple firewall rules. This approach eliminates the need to create and maintain multiple individual security service settings for each firewall rule, thereby streamlining the process and enhancing efficiency.

Enhanced Security Monitoring Dashboard

The Enhanced Security Monitoring Dashboard provides a tailored, comprehensive overview designed for enterprises, offering deep insights into identified threats via IDS/IPS, URL Filtering, and Malicious IP Filtering, thereby enhancing the overall security posture of the enterprise.

Improved Firewall Logging

The Firewall logging feature in Release 6.0.0 presents a comprehensive pane view for each log record selection, encompassing both Firewall and Enhanced Firewall Service engine-related data. Furthermore, new intelligent filters have been integrated to facilitate the searching of logs based on specific engines, including Firewall, Intrusion Detection System/Intrusion Prevention System (IDS/IPS), URL Category, URL Reputation, and Malicious IP Filtering.

Zscaler Automation

Customers using the Zscaler Cloud Security Service (CSS) automation benefit from three enhancements included with Release 6.0.0:

  1. The Edge name is automatically inserted in the Zscaler Location Name field. After establishing an automatic IPsec or GRE tunnel for an Edge segment, the Location Name is automatically created and appears under the Location table. Previously, the Orchestrator generated a Location Name only with the Edge's logical identifier. By automatically inserting the Edge name at the beginning of the Location Name, customers can improve the management of their large enterprises by easily identifying their Edges, especially on the Zscaler portal where they can search for the Edge name to find its location.

    Note:

    This enhancement is applicable for new CSS configurations only.

  2. Configurable subnets for Zscaler Sub-Locations. In prior Orchestrator versions, the Subnets field under Sub-Locations was limited to the subnets utilized by a directly connected Edge, and users could not modify these subnets using the Orchestrator UI. This limitation presented a challenge for a branch office where the LAN-side subnets were one hop away due to the presence of a layer 3 switch between the Edge and LAN devices. Release 6.0.0 corrects this by redesigning the Subnets field to be a configurable text box.

  3. Domestic Preference option available for IPsec tunnels. Previously, the Domestic Preference option was only available for GRE tunnels. This option prioritizes Zscaler data centers located in the same country as the IP address's origin, even if other Zscaler data centers may be geographically closer.

Equal-Cost Multi-Path (ECMP) on the Gateway

An SD-WAN Gateway using Release 6.0.0 includes the capability of Equal-Cost Multi-Path (ECMP) routing over multiple Non SD-WAN Destination (NSD) tunnels. In addition to facilitating numerous Active/Active tunnels from the Gateway to various Destination IP addresses, the ECMP functionality allows traffic to be dispersed evenly across all available paths to the destination, maximizing bandwidth utilization and enhancing resilience.

Intel E810 NIC Support for Gateway Deployments

SD-WAN and Partner Gateways using Release 6.0.0 support the use of the 100GbE Intel® Ethernet Network Adapter E810 using SR-IOV on KVM 22.04 for high performance Data Plane throughput.

Orchestrator GCP Support

Release 6.0.0 adds support for hosting an Orchestrator on the Google Cloud Platform (GCP).

Orchestrator OS Upgrade and Behavior Change

Release 6.0.0 brings the following changes to the Orchestrator

The base operating system for the 6.0.0 Gateway is upgraded to a VMware compiled Ubuntu 22.04 clone.

Important:

In addition, the upgrade process has changed, and this results in the following caveats for Orchestrators using Release 6.0.0 and later:

  • A reboot is required for every Orchestrator software upgrade. This includes an upgrade to a rollup or hotfix build.

  • Any unsupported or undocumented software installed on the Orchestrator by Operators may stop working and be removed during system upgrades.

  • Any undocumented configurations made on an Orchestrator by Operators may stop working and be removed during system upgrades.

  • 30GB of unallocated free physical disk space is required for an Orchestrator deployment.

Please refer to the documentation for details.

Gateway OS Upgrade and Behavior Change

Release 6.0.0 brings the following changes to the Gateway

The base operating system for the 6.0.0 Gateway is upgraded to a VMware compiled Ubuntu 22.04 clone.

Important:

In addition, the upgrade process has changed as the Gateway moves from a package-based upgrade to an image-based upgrade. This change results in the following caveats for Gateways using Release 6.0.0 and later:

  • A reboot is required for every Gateway software upgrade. This includes an upgrade to a rollup or hotfix build.

  • Any unsupported or undocumented software installed on the Gateway by Operators may stop working and be removed during system upgrades.

  • Any undocumented configurations made on a Gateway by Operators may stop working and be removed during system upgrades.

  • There are separate image packages for platforms: Generic (ESXi and KVM), AWS, Azure, and GCP.

Important Notes

VMware Security Advisory 2024-0008

Limitation When Deactivating Autonegotiation on VMware SD-WAN Edge Models 520, 540, 620, 640, 680, 3400, 3800, and 3810

When a user deactivates autonegotiation to hardcode speed and duplex on ports GE1 - GE4 on a VMware SD-WAN Edge model 620, 640 or 680; on ports GE3 or GE4 on an Edge 3400, 3800, or 3810; or on an Edge 520/540 when an SFP with a copper interface is used on ports SFP1 or SFP2, the user may find that even after a reboot the link does not come up.

This is caused by each of the listed Edge models using the Intel Ethernet Controller i350, which has a limitation that when autonegotiation is not used on both sides of the link, it is not able to dynamically detect the appropriate wires to transmit and receive on (auto-MDIX). If both sides of the connection are transmitting and receiving on the same wires, the link will not be detected. If the peer side also does not support auto-MDIX without autonegotiation, and the link does not come up with a straight cable, then a crossover Ethernet cable will be needed to bring the link up.

For more information please see the KB article Limitation When Deactivating Autonegotiation on VMware SD-WAN Edge Models 520, 540, 620, 640, 680, 3400, 3800, and 3810 (87208).

Orchestrator API Changes

Orchestrator API Changes since 5.4.0

Resolved Issues Impacting APIs

API Type

Issue Ticket #

Affected APIs

Description

APIv1

#135190

 
/configuration/updateConfiguration
/configuration/insertConfigurationModule
/configuration/updateConfigurationModule (for type "firewall")

In 5.4.0, Enhanced Firewall actions were configured directly on Firewall Rules. As of 6.0, Firewall Rules do not have Enhanced Firewall actions. Each Security Service Object must configure their own action. Up to one of each type of Security Service Object is referred to by Firewall Rules as a Security Service Group.

APIv2

#136851

 
GET /api/sdwan/v2/enterprises/{enterpriseLogicalId}/firewallIdpsStats
GET /api/sdwan/v2/enterprises/{enterpriseLogicalId}/edges/{edgeLogicalId}/firewallIdpsStats

Removed unnecessary startTime field from response of Enterprise and Edge Firewall IDPS stats APIs. The startTime field is still included for the edgeFirewallIdpsStatsSeries API to populate time series.

The groupBy parameter is now required as this value indicates which metric to return and in its absence the APIs do not work.

Changes to the VMware SASE Orchestrator Portal API ("API v1")

For reference, the complete API Changelog is available for download at developer.vmware.com (see "VMware SD-WAN Orchestrator API v1").

Beginning May

New APIv1 endpoints added in 6.0.0.0

APIv1

Endpoint

Description

 
/enterprise/deleteSecurityObject
/enterprise/getSecurityObjects
/enterprise/insertSecurityObject
/enterprise/updateSecurityObject

These APIs are used to insert/delete/get/update IDS/IPS, URL Category Filtering, URL Reputation, and Malicious IP Filtering objects. Additionally they can be used to insert/delete/update/get security Service Groups, which act as a placeholder/container of any of the previous mentioned objects.

 
/firewall/getMaliciousIpCategories
/firewall/getUrlFilteringCategories

getUrlFilteringCategories is used to retrieve the map of URLs and their corresponding categories according to the Webroot BrightCloud database.

 
/metrics/getEdgeFirewallMaliciousIpMetrics
/metrics/getEnterpriseFirewallMaliciousIpMetrics

/metrics/getEdgeFirewallUrlCategoryMetrics
/metrics/getEnterpriseFirewallUrlCategoryMetrics

/metrics/getEdgeFirewallUrlReputationMetrics
/metrics/getEnterpriseFirewallUrlReputationMetrics

/metrics/getEnterpriseFirewallEdgeCountMetrics
/metrics/getEnterpriseFirewallEdgeSummaryMetrics

These APIs retrieve Enhanced Firewall metrics based on the given time range and which the metrics/dimensions are requested. The "metrics" argument can be an array of strings in case multiple metrics are desired in a single response. The "viewBy" field determines how the metrics are aggregated. Aggregations by Edge are only available on Enterprise-level APIs, while aggregations by client are only available on Edge-level APIs.

Examples:

  • metrics = ["monitoredThreatsCount"], viewBy = "sourceIp" returns the count of MONITOR actions, aggregated by the client IP address.
  • /metrics/getEnterpriseFirewallEdgeCountMetrics returns both the number of current Edges and the number of distinct Edges generating metrics for the given time range.
  • /metrics/getEnterpriseFirewallEdgeSummaryMetrics returns the count of edges generating metrics for the given time range by ATP feature (IDPS, URL Category, URL Reputation, and Malicious IP)

Changes to the VMware SASE Orchestrator API v2

  • Required groupBy Query Parameter in firewallIdpsStats API

    • groupBy query parameter used to be optional in 5.4.0, but is now required in 6.0.0.

  • New Enum Values in Events APIs

    • These new values have no backward compatibility issues with 5.4.0.

  • Gateway Schema Changes

    • handOffDetail and ipsecGatewayDetail in the Gateway object schema changed from a string to an object, with no properties. These changes do not impact backward compatibility with 5.4.0.

    • Comparison Table

      Release 6.0.0

      Release 5.4.0

      		 
      "handOffDetail": {
  "type": "object",
  "properties": {},
  "additionalProperties": false,
  "nullable": true
},
"ipsecGatewayDetail": {
  "type": "object",
  "properties": {},
  "additionalProperties": false,
  "nullable": true
},

      		 
      "handOffDetail": {
  "type": "string",
  "nullable": true 
},
"ipsecGatewayDetail": {
  "type": "string",
  "nullable": true 
},

Developer Documentation

All VMware SASE/SD-WAN API documentation resides on the Developer Documentation Portal at https://developer.vmware.com/apis.

Available Languages

The VMware SASE Orchestrator using version 6.0.0 is localized into the following languages: Czech, English, European Portuguese, French, German, Greek, Italian, Spanish, Japanese, Korean, Simplified Chinese, and Traditional Chinese.

Document Revision History

September 18th, 2024. Twenty-Third Editon.

  • Revised the description for Orchestrator rollup build R6005-20240815-GA in the Orchestrator Resolved Issues section to include the statement that this build also adds support for provisioning and configuring Edge models 710-5G, 720, and 740 on the UI.

Note:

Edge models 710-5G, 720, and 740 must be deployed with a 5.2.4.x Edge build as they do not have 6.0.0.x Edge software support.

September 5th, 2024. Twenty-Second Edition.

August 15, 2024. Eleventh Edition.

  • Adds a new Orchestrator rollup build R6005-20240815-GA to the Orchestrator Resolved Issues section. This is the fourth Orchestrator rollup build and is the new default Orchestrator build for Release 6.0.0.

  • The Orchestrator build R6005-20240815-GA includes the fixes for issues #146441, #147774, #147846, and #148827, which are documented in this section.

July 26, 2024. Tenth Edition.

  • Adds a new Edge/Gateway rollup build R6001-20240709-GA to the Edge/Gateway Resolved Issues section. This is the first Edge/Gateway rollup build and is the new default Edge/Gateway build for Release 6.0.0.

  • This Edge/Gateway build remediates CVE-2024-6387, a critical vulnerability in OpenSSH for both the Edge and the Gateway. For more information on this OpenSSH vulnerability, please consult the VMware Security Advisory the KB article Broadcom Software Defined Edge Division response to CVE-2024-6387 - OpenSSH signal handler race condition vulnerability.

  • The Edge/Gateway build R6001-20240709-GA includes the fix for issue #143575, which is documented in this section.

  • The 6.0.0.1 Gateway build, when combined with a 6.0.0.4 Orchestrator, adds the Non SD-WAN Destination (NSD) IP Portability option. Portable NSD IPs allows Operators to move NSD configurations to different Gateways in the PoP without requiring the customer to reconfigure their tunnels. For more information, see the SD-WAN Operator Guide > Configure Gateways or the SD-WAN Partner Guide > Configure Gateways.

  • Important:

    The NSD IP Portability option must have both an Orchestrator using 6.0.0.4 or later, and a Gateway using 6.0.0.1 or later to work as expected.

July 17, 2024. Ninth Edition.

June 20th, 2024. Eighth Edition.

June 13, 2024. Seventh Edition.

  • Revised and expanded the Zscaler Automation entry located in the New Features and Enhancements section. Previously this entry only documented the automatic insertion of the Edge Name in the Zscaler Location Name field. However, there are two additional enhancements included with the 6.0.0 Orchestrator that were omitted: Configurable subnets for Zscaler Sub-Locations, and the availability of the Domestic Preference option for IPsec tunnels. These two enhancements are now documented under the Zscaler Automation entry.

May 28, 2024. Sixth Edition.

  • Adds a new Orchestrator rollup build R6003-20240523-GA to the Orchestrator Resolved Issues section. This is the second Orchestrator rollup build and is the new default Orchestrator GA build for Release 6.0.0.

  • Orchestrator build R6003-20240523-GA includes the fixes for issues #134814, #141617, #143577, #144613, #144629, #144992, and #145188, each of which is documented in this section.

Note:

Fixed Issue #144613 was erroneously listed in R6002-20240510-GA as fixed. The fix for #144613 is included beginning with the 6.0.0.3 Orchestrator version.

  • Removed the Important Note about Localization Delay, as both Orchestrator and documentation translation are complete.

May 13th, 2024. Fifth Edition.

  • Adds a new Orchestrator rollup build R6002-20240510-GA to the Orchestrator Resolved Issues section. This is the second Orchestrator rollup build and is the new default Orchestrator GA build for Release 6.0.0.

  • Orchestrator build R6002-20240510-GA includes the fixes for issues #99891, #123962, #141974, #142787, #142878, #144393, #144140, #144175, #144280, #144405, #144469, #144613, #144789, and #144829, each of which is documented in this section.

  • Added Open Issue #143828, the Edge/Gateway Known Issues section.

May 10th, 2024. Fourth Edition.

  • This edition marks Phase 2 of Release 6.0.0 and includes GA builds for the Edge and the Gateway.

  • Adds the Edge/Gateway Resolved Issues section for the initial GA version R6000-20240509-GA. This section includes all the tickets fixed for this Edge/Gateway build.

  • The Compatibility table is now fully populated with all Release 6.0.0 Orchestrator, Gateway, and Edge tested interoperability combinations.

  • Adds the section 6.0.0 is a Short-Term Support (STS) Release.

April 23rd, 2024. Third Edition.

  • Adds a new Orchestrator rollup build R6001-20240421-GA to the Orchestrator Resolved Issues section. This is the first Orchestrator rollup build and is the new default Orchestrator GA build for Release 6.0.0.

  • Orchestrator build R6001-20240421-GA includes the fixes for issues #137150, #142575, #142990, #142991, and #143621, each of which is documented in this section.

April 17th, 2024. Second Edition.

April 11th, 2024. First Edition.

Edge/Gateway Resolved Issues

Resolved in Edge/Gateway Version R6001-20240709-GA

Edge/Gateway version R6001-20240709-GA was released on 07/26/24 and is the 1st Edge/Gateway rollup build for Release 6.0.0.

Important:

The Edge and Gateway builds each include a remediation for CVE-2024-6387, a critical vulnerability in OpenSSH. For more information on this OpenSSH vulnerability, please consult the article Broadcom Software Defined Edge Division response to CVE-2024-6387.

Note:

The 6.0.0.1 Gateway build, when combined with a 6.0.0.4 Orchestrator, adds the Non SD-WAN Destination (NSD) IP Portability option. Portable NSD IPs allows Operators to move NSD configurations to different Gateways in the PoP without requiring the customer to reconfigure their tunnels.

There are two important caveats with NSD IP Portability:

  • NSD peers must operate as an IKE responder. In the past, some configurations would permit an initiator-only setup, but that will no longer function.

  • Peers must allow NAT-T (NAT Traversal). The current implementation of NSD Portable IP uses NAT which is internal to the PoP.

For more information, see:

In addition, Edge/Gateway build R6001-20240709-GA addresses a critical issue since Edge/Gateway version R6000-20240509-GA.

  • Fixed Issue 143575: On the Edge 610 and 610-LTE, in some configurations, large, fragmented packets result in the full-sized fragments being dropped on the Gigabit Ethernet ports.

    If any of the interfaces is set to a non-default MTU (i.e. not 1500), then the MTU of the interface from the hardware switch that backs the 6 GE ports to the CPU gets an incorrect MTU applied, resulting in full 1500-byte packets or fragments being dropped.

    The workaround for this issue is to set at least one of the GE ports to an MTU of 1504.

Resolved in Edge/Gateway Version R6000-20240509-GA

Edge/Gateway version R6000-20240509-GA was released on 05-10-2024 and resolves the following issues since Edge version R5401-20240224-GA and Gateway version R5401-20240314-GA-141087.

Note:

This means that a fix for an Edge or Gateway issue listed in the 5.4.0 Release Notes up to these listed builds is included in all Release 6.0.0 builds.

  • Fixed Issue 83526: The L7 Health Check for specific WAN links will show as "up" even if the link itself is "down" if a common health check destination exists for multiple links.

    The L7 Health Check is per-link, but an issue in the code would cause any valid link to be selected. For destinations that are the same across multiple links (such as Zscaler), this would result in the health check falsely reporting that the link was still up.

  • Fixed Issue 86786: When there is an outbound BGP filter with a 0.0.0.0/0 EXACT MATCH rule associated to a default originate (implicitly attached to default originate when associated with an outbound filter for a neighbor), the default route may not be advertised to the neighbor.

    The issue is the result of a design limitation in the Edge's routing process.

    A customer could work around this issue by adding a rule that permits a more specific route to be advertised. Doing this causes the default route and the added permitted route to be advertised.

  • Fixed Issue 90588: On a customer enterprise site using either High Availability topology, if the interface which has BGP enabled becomes down on the Active Edge and triggers an HA failover, traffic convergence takes a longer time even with BGP Graceful Restart configured.

    When the link connected to the BGP neighbor is brought down on the Active Edge, by design there is a 700 ms delay prior to failing over. Within that time the Active Edge's synchronization with the Standby Edge deletes these BGP routes for that BGP peer. Once failover is complete and the Standby Edge becomes the Active Edge, client users would observe traffic loss until the newly Active Edge relearns the routes from the peer.

  • Fixed Issue 110791: For a customer enterprise using BGP for routing, consistent peer flaps over two or more days may cause the BGP process to fail entirely and need to restart.

    During a BGP peer flap, many internal data structures of the Edge's BGP process are removed and added back. After a high number of these remove and re-adds, the Edge may experience memory corruption for BGP internal datastructure instances (in particular: bgp_advertisement_attribute, community_hash_table, attribute_hash table).

    This corruption is rare and has only been observed after the BGP session is flapped every 60 seconds, continuously over a period of three days.

  • Fixed Issue 111323: For a customer enterprise using OSPF for routing, if the Edge does not have an advertise option configured for any interface, the client users may observe traffic issues for some routes.

    If none of the Edge interfaces are configured with an advertise option, the OSPF router-id is set to a hardcoded value 12.1.1.1. This creates a problem when Edges are connected back-to-back and their interfaces do not have the advertise option turned on, and results in neighbors with the same router-id and customer traffic routing problems following from this state.

  • Fixed Issue 111758: A 4-core Gateway may experience a Dataplane Service failure, and restart when running at a large scale.

    Large scale defined as 2500 tunnels from 2200 Spoke Edges. The logs would point to a mutexmon triggering event. The command: ip netns X is run via the vc_spawnf syncronous thread, which runs as a non-RealTime thread when invoked from the parent thread. During a Gateway service start when there is a high resource contention for the CPU among several threads, this thread can be starved since it running as non-RT and does not inherit RT priorities from the parent thread.

  • Fixed Issue 112003: A 4-core Gateway may experience a Dataplane Service failure during a restart of the service when running at a large scale.

    "Large scale" in this instance is as follows: 3500 total tunnels with 800 RSA tunnels, and 2700 PSK Tunnels. There is a 2000 Peer Count, 100K routes, 450-500K flows, 100-150Mbps of throughput, and 50K NATs.

    Similar to #111758, in this instance the vrf_worker thread is run as a non-RealTime thread. During a Gateway service start, when there is high resource contention for CPU among threads while running at scale, the vrf_worker thread can be starved since it running as non-RT thread. The priority of the thread now is elevated to run with RealTime priorities.

  • Fixed Issue 118444: For a customer enterprise site deployed with a High Availability topology, if the customer is using a mismatched pair of Edges where one is Wi-Fi capable and one is not, the HA Edge may not send an event to the Orchestrator indicating a Wi-Fi mismatch has been detected.

    When a Wi-Fi capability mismatch is detected, the Edge should send the HA_WIFI_CAPABILITY_MISMATCH event to the Orchestrator. However, if a particular state variable is not set in the Edge's management process, we discard the management heartbeat from Standby Edge to Active Edge, and if the Wi-Fi capability mismatch event is sent as part of this heartbeat, the event also gets discarded.

  • Fixed Issue 118710: IP address and Netmask may not be assigned to a VLAN interface created on top of an Edge's Wi-Fi VLAN interface.

    The Edge's kernel is not assigning the VLAN interfaces on the top of the Wi-Fi (WLAN) interfaces, and the configuration is not applied to these VLAN interfaces. In addition, the assignment of network interfaces corresponding to the Edge's interface is also missed with the result being that the Wi-Fi interfaces becomes unusable because they have no IP address assigned to them.

  • Fixed Issue 118881: If a customer user activates Profile Isolation for a profile used by Spoke Edges, some routes from other profiles are not deleted from the Spoke Edges.

    The Gateway is supposed to flush all the routes from other profiles as part of Profile Isolation, but in this issue it treats routes that have a reachability of false as not needing to be flushed and those routes are not being flushed with the rest of the routes.

  • Fixed Issue 119428: A Gateway deployed with ESXi version 7.0 GA may experience multiple Dataplane Service failures and stop passing traffic.

    ESXi 7.0 GA has DPDK compatibility issues with the Gateway build that can trigger exceptions and a triple service failure that can only be recovered with a reboot. Release 6.0.0 supports a deployment using ESXi 7.0 Update 3. See: Minimum Hypervisor Hardware Requirements.

  • Fixed Issue 120940: If there are multiple routes for the same prefix in BGP from different neighbors in the same internal VRF (like a VRF created for Non SD-WAN Destinations via Gateway BGP sessions), the same neighbor IP is updated for all the routes.

    An Operator or Partner can observe the issue on a Gateway by using the commands debug.py --routes and debug.py --bgp_view outputs, and is caused by a Gateway's routing process not updating the neighbor IP (source) properly.

  • Fixed Issue 121328: The L7 Health Check returns consistently high times (RTT), even though a ping indicates the actual RTT is much lower.

    The L7 Health Check calculates the time taken for the health check to complete by wrapping a time stamp around the command in a script. Based upon the load on the system, the script itself may experience compute resource starvation. This shows up as L7-HC delta T being higher than the normal packet RTTs. In addition, the L7 Health Check includes two RTTs for the L7 exchange which cannot be compared directly with the on-wire packet RTTs (in other words, a simple ping from the Edge).

    The fix for this issue fetches the "time taken" for the invocation to complete from the command itself with the "-w" option, ensuring an accurate RTT for an L7 Health Check.

  • Fixed Issue 122267: For a customer enterprise site configured with a High Availability topology where the Loss of Signal (LOS) is configured, the newly demoted Standby Edge does not detect the restoration of a link to up after an HA failover.

    As part of the Loss of Signal feature, if a LAN link goes down, a failover is triggered, and the Standby is promoted to Active. The issue is that if the previously downed LAN link is now restored to an up state on the new Standby Edge (previously the Active Edge), the Standby Edge does not detect this state. Should this LAN link go down again, this would not trigger another failover as would be expected with LOS. Only triggering a manual HA failover fully corrects the LAN link's status for both HA Edges.

    The issue is caused by an aspect of HA design where all LAN ports are blocked on a Standby Edge which prevents any detection of a restored LAN link and thus the LAN link count is never updated until a new failover promotes the Standby to Active.

  • Fixed Issue 122314: When a management flows switches from one peer to another, client users behind the Edge may observe application performance degradation.

    When a management flow switches from one peer to another peer, the resequence queue length can become negative in rare conditions. When this occurs, all the buffered packets in the management resequence ring will be immediately flushed out (known as an emergency flush), and this can lead to application performance degradation.

  • Fixed Issue 122988: For a customer site configured with a High Availability topology, a customer may observe that the Standby Edge restarts multiple times.

    This issue can be observed in Events. In addition, if an Enhanced HA topology is used where the Standby Edge also passes traffic, client user traffic using the WAN link(s) on the Standby Edge would also be impacted.

    The issue is triggered by the HA Edge packet forwarding thread being starved due to a file operation and the Standby Edge misses the HA heartbeat, causing the Standby Edge to become active and that triggers an active/active state where the Standby Edge is restarted to recover the state.

    For a site without a fix for this issue, a user can increase the HA failover time from 700ms to 7000ms on a 5.2.0 or later Orchestrator.

  • Fixed Issue 127920: An ICMP probe attached to a static route with a secondary IP as the next hop may go down, even though the secondary IP address is up and reachable.

    When an ICMP probe is attached to a static route with a secondary IP as a next hop, the probe is sourcing the main interface IP address instead of the secondary IP address. When the peer is unable to reach the primary IP address, it results in the ICMP probe going down though the secondary IP is up and reachable.

    There is no workaround in this scenario beyond not using a secondary IP address.

  • Fixed Issue 128150: A user may observe the Edge event MGD_DEVICE_CONFIG_{ERROR,WARNING} for an Edge that is not Wi-Fi radio capable.

    When an Edge's country location is changed, the wireless file is populated for Edges that do have Wi-Fi capability. This causes the non Wi-Fi Edge to send out events to the Orchestrator that are not applicable for that Edge type.

  • Fixed Issue 129000: For customers with large enterprises where their total tunnels exceeds 6000, where Partner Gateways are used, and deploy Hub Clusters as part of that enterprise, the Remote Diagnostics "Rebalance Hub Clusters" and "Redistribute Spokes" do not work properly.

    For large SD-WAN deployments, Hub Clusters rebalancing using "Remote Diagnostics" does not work correctly when the Partner Gateway configured order is different on the Hub Clusters and the Spoke Edges. The cluster members send the rebalance request only to the Primary Gateway and the Spoke Edges ignore the reassignment received if they have a different Primary Gateway.

    There is no workaround for this issue. A customer support engineer can manually rebalance the Hub Clusters if required.

  • Fixed Issue 129317: On the Diagnostics > Remote Diagnostics page of the Orchestrator UI, when running the List Paths diagnostic outputs inaccurate information if the customer enterprise uses Hub Clusters/Spoke Edges and the request is for "Interconnect Paths".

    For any Hub Cluster Edge, when a Spoke Edge (which is not a part of the interconnect cluster) is selected as the peer device, and the Interconnect Paths option is checked, all paths are listed as part of the List Paths output. The expected output should be: "There are no interconnect paths to this peer", as only interconnect tunnels should be displayed and in this scenario there are none to display.

  • Fixed Issue 129559: For a customer enterprise using a Hub Cluster/Spoke Edges topology, a user may observe the Event message "Cluster Interconnect Tunnel down" on a Spoke Edge.

    The Cluster Interconnect Tunnel down event is reserved for Edge Hub Cluster's interconnect with another Edge Hub Cluster, and should not be seen on Spoke Edges connected to the Edge Hub Cluster. Spoke Edges should have their own distinct Event for when a Spoke Edge's tunnel with a Edge Hub Cluster goes down.

  • Fixed Issue 129697: For a customer enterprise using a Hub Cluster/Spoke Edges topology, after a user activates the Hub or Cluster Interconnect feature on Hub Cluster profiles, the Event "Cluster Interconnect Tunnel Up" may be missing for Spoke Edges.

    After Hub or Cluster Interconnect is activated, all affected Edges should send an Even message "Cluster Interconnect Tunnel Up". In this issue, only the Edge Hub Clusters are sending the Event message and the Spoke Edges are not.

  • Fixed Issue 131674: ICMP traffic from a Spoke Edge using internet backhaul via a Hub Edge fails if the ICMP flow has to be steered from one link to a different one.

    ICMP traffic passing on one of two or more links is expected to be steered to a different link if the existing link goes down or is unusable per QoE.

  • Fixed Issue 132112: When OSPF is enabled on an interface, local connected routes are advertised as OSPF external routes and get added to the Edge's FIB.

    When an Edge interface is down or disabled, the connected routes are not revoked from the Gateway but instead continue to be advertised as reachable. This results in user traffic getting dropped because this traffic is still using routes that are not reachable.

  • Fixed Issue 132975: When a Partner or Operator generates a diagnostic bundle for a Gateway, the resulting bundle is missing the logs in the /var/log folder, including gwd.

    The logs in /var/log are missing with the following error "Unable to add var/log to diag dump: Filesize would require ZIP64 extensions" in the /var/log/mgdtools.log. The reason for the exclusion was the total size of /var/log exceeded 2GB and the process used for compression was throwing an exception. The excessive file size of /var/log is the result of a corrupted /var/log/lastlog file which had inconsistent size reporting, and this caused the entire /var/log directory to be excluded from the Gateway's diagnostic dump.

    On a Gateway without a fix for this issue, a user can add the var/log/lastlog to the /opt/vc/etc/vcdiag_excludes.json.

  • Fixed Issue 133081: For a customer enterprise site deployed with a High Availability topology, a user may observe an event for an Edge Service failure for the Standby Edge.

    If an invalid route is synchronized to the Standby Edge, this Edge experience a Dataplane Service failure and restarts to recover. The issue is caused by corrupted data in the synchronization message which triggers the Edge service failure when processed. The fix adds a checksum for the synchronization data and if the checksum does not match on the Standby Edge, then the packet is ignored from processing. While this issue is not disruptive in a Standard HA topology, an Enhanced HA topology could see some customer traffic disruption for packets using a WAN link on the Standby Edge.

  • Fixed Issue 134920: For an Orchestrator deployed in an On-Premises environment where internet access is not used and thus the Orchestrator cannot connect to the Global Services Manager (GSM), if the Orchestrator enables the Enhanced Firewall Service and IDS/IPS is configured, all traffic will be blocked.

    The Global Services Manager (GSM) is a key component of the Enhanced Firewall Service and, if this service cannot connect to the GSM, the issue will occur even if the Enhanced Firewall Service has a rule to allow all traffic. The only workarounds are to either have internet service for the On Prem Orchestrator or deactivate the Enhanced Firewall Service.

  • Fixed Issue 135583: For a Gateway running at a high scale, the Gateway's NAT process may abruptly stop and all traffic being NAT'd immediately drops.

    When the Gateway's NAT process receives a request from the Gateway service to create a new NAT entry for a new connection but it has run out of available port entry (PAT) objects, the Gateway's NAT process will clear its entire NAT table and restart, effectively denying the NAT entry request and preventing the connection from being established.

    With this fix, the Gateway's NAT process gracefully denies the request by returning an error code to the Gateway. New connections can be established when existing connections are closed or timed out.

  • Fixed Issue 136336: For a customer who configures a Cloud Security Service (CSS) with a Zscaler type, return traffic from Zscaler may get dropped if the Edge has the Common Criteria Firewall enabled.

    The Edge would have a Business Policy rule to backhaul internet traffic via that Zscaler tunnel and in this case the return traffic is dropped due to a Reverse Path Forwarding (RPF) failure that occurs during the route lookup for the return traffic.

  • Fixed Issue 136949: After an Edge restarts, it may be missing Branch to Branch routes from its RIB and FIB, resulting in some customer traffic disruption.

    This issue is caused by a delay in processing the 'tunnel up' event towards the Gateways immediately after an Edge service restart. The result is routes being received from the Gateways and the stale PI timer starts right after, which removes the routes from the Edge's Routing Information Base (RIB) and Forwarding Information Base (FIB) post 5 on expiry of a stale PI timer of 5 minutes.

    If experiencing this issue on an Edge without a fix for this issue, the routes need to be re-initiated.

  • Fixed Issue 137912: For a customer enterprise using OSPF for routing, when the inbound filter properties of an OSPF interface are changed from learn, to ignore, there may be an Edge memory leak.

    The Edge service uses temporary lists for synchronizing OSPF route additions and deletions to the Orchestrator. With this issue those temporary lists are not being freed if the routes learned earlier are denied by changes in the OSPF inbound filters. The temporary lists can accumulate and take up Edge memory.

  • Fixed Issue 137932: For a customer using Partner Gateways, when the Handoff IP Address is changed on the Orchestrator UI, this change is not applied to the Gateway even though the UI shows that it has been applied, which leads to customer traffic failure.

    This can be confirmed on the Gateway using the debug.py --ifaces command. The issue is caused by a defect in the Gateway process for handling only a handoff IP address update, though a first time handoff configuration does work.

    The workaround is to disable handoff for both the enterprise and the segment and save the configurations. Then re-enable handoff and configure the new handoff IP address. This should only be done in a maintenance window.

  • Fixed Issue 138131: When a Non SD-WAN Destination (NSD) BGP neighbor is configured as passive, the SD-WAN Gateway does not form the BGP session.

    This issue is due to a defect in the Gateway that prevents NSD BGP sessions from initiating TCP connections.

  • Fixed Issue 138452: For a customer who uses a Cloud Security Service (CSS) or a Non SD-WAN Destination via Edge, if a user changes an Edge interface's WAN overlay from auto detect to user defined, the customer may observe duplicate tunnel entries and traffic drops.

    When an interface overlay is modified, the expected behavior is for the Edge to delete all the CSS/NDS paths linked with the interface. With this issue the Edge is deleting only one CSS/NSD path linked to the interface, which results in duplicate tunnels.

    On an Edge without a fix, a user should only change the Edge interface WAN overlay in a maintenance window. After the change is made, disable all CSS or NSD's via Edge and then reenable them as this will delete all the tunnels.

  • Fixed Issue 138755: A customer enterprise may experience packet drops for self-destined flows when the Enhanced Firewall Service is activated, and the Stateful Firewall is deactivated.

    When the Stateful Firewall is deactivated, self-destined traffic is subject to the firewall. So if the Enhanced Firewall Service is activated on the matching policy and the Enhanced Firewall Service engine is not yet ready, client users would observe packet drops with counter "suricate_not_ready".

    On an Edge without a fix for this issue, the customer can activate Stateful Firewall to ensure that self-destined packets are not subjected to firewall rules (if that is the intention).

  • Fixed Issue 138900: For a customer site deployed with a High Availability topology, the customer may observe Standby edge restart resulting from Standby Edge having high memory usage.

    If memory usage is sufficiently high, the Edge defensively restarts the Dataplane Service to clear the issue, and this impacts traffic, if that Standby has any enhanced HA interface configured. In case of standard HA, there is no functionality impact. The high memory usage is the result of a memory leak which is caused by Edge refreshing all IPsec tunnel certificates which triggers a tear down and rebuilding of all IPsec tunnels when there is change in the certificate. Buildup is relatively small.

    On an HA site without a fix for this issue, the only workaround is to monitor memory usage and, should usage get too high, defensively restart Standby edge during a maintenance window to clear memory.

  • Fixed Issue 140033: For a customer enterprise deployed with a Hub/Spoke topology, a Spoke Edge may experience a Dataplane Service failure and restart when sending a high number of flows to the Hub Edge when the Hub Edge restarts.

    When there are large scale number of backhaul flows to the Hub Edge and the Hub Edge goes down (as during a restart), the Spoke Edge's route lookup process is performed for every flow and ends up iterating all the peer routes by holding the peer routes lock in a tight loop since the peer route reachability would be marked as false, this triggers a mutexmon induced SIGXCPU as the other threads wait for the peer routes lock.

  • Fixed Issue 141041: For a customer enterprise site deployed with a High Availability topology with VNFs installed, where the HA Edge pair or either Edge models 520/540 or 610, reachability to the Standby Edge's VNF from a LAN-connected client may fail.

    The Ethernet switch board on these Edge models drops ARP reply packets sent to a LAN client from a Standby Edge's VNF resulting in a loss of reachability.

    This issue was first observed for the 5.4.x Edge build and documented in 102583 of the 5.4.0 Release Notes. It is tracked and resolved with this ticket for 6.0.0.

  • Fixed Issue 141043: Traffic for a Business Policy Rule configured to rate limit both inbound and outbound traffic at 0% may cause a packet queue build and impact other customer traffic.

    When a business policy with a 0% rate limit is used to blackhole traffic matching that rule, the traffic is blackholed as expected, but a packet queue buildup also occurs. This can result in netscheduler queue drops even after the rule is deleted or modified.

    On an Edge without a fix for this issue, avoid using Business Policy Rules to drop traffic and use Firewall Rules instead.

  • Fixed Issue 141087: An Edge deployed as a Hub in a Cluster, or a Gateway may experience a Dataplane Service failure and restart if Dynamic Branch-to-Branch is enabled. If one or more customer enterprises connected to a VMware SD-WAN Gateway use Dynamic Branch-to-Branch, the Gateway may experience a Dataplane Service failure, generate a core, and restart to restore functionality.

    When an Edge asks the Gateway to provide the endpoint information for another Edge to form a Dynamic Branch-to-Branch tunnel, the Gateway adds this information to a hash table. Once the information is delivered to the requesting Edge, the entry is removed. The Gateway service runs a timer where, at 60 second intervals, the Gateway removes the entries which have not been delivered to the requesting Edge.

    In this context, the Gateway service can experience a mutex induced service failure caused by the hash table iterating for over 90 seconds (which triggers an exception). An Operator running a backtrace would observe that the failure points to vc_de2e_entry_timeout_iter_cb.

  • Fixed Issue 141327: If a user performs a VNF insert using a Palo Alto Networks (PAN) VNF for an Edge model 520v, the VNF Insertion remains down after the required Edge Service restart.

    The issue occurs when using the VM-50 lite Authcode. If encountering this issue on an Edge without a fix for it, redeploying the VNF is the only remediation.

  • Fixed Issue 141336: For a customer enterprise site deployed with a High Availability topology, if the customer configures the HA interface to a non-default interface (in other words, some interface other than GE1), when they enable HA, they may observe the Orchestrator UI reporting that the peer state is unknown.

    When HA is initially enabled with the default GE1 interface, a virtual MAC address is programmed for all interfaces except the GE1 HA Interface. However, if HA is disabled and then re-enabled with a non-default, non-GE1 HA Interface, the new HA interface continues to be programmed with a virtual MAC address. As a result, the Active Edge drops the peer heartbeat since the source MAC address is matching to the received interface MAC and sets the Standby Edge peer state as "Unknown".

    On an HA Edge without a fix, prior to re-enabling HA with a non-default HA Interface (GE1), a Technical Support Engineer needs to delete the /velocloud/ha/virtualmacs file from the Active Edge. Contact Support if this step is needed.

  • Fixed Issue 141834: The Edge model 710-W local user interface may not function properly.

    There are two states where the 710-W local UI can not function properly:

    • When the Edge is an unactivated state.

    • When the Edge 710-W is activated with 2.4 Ghz Wi-Fi radio disabled on the Orchestrator.

    Without a fix, a user would need to turn on the 2.4 Ghz Wi-Fi on the Orchestrator in the profile and not override the radio settings on the Edge.

  • Fixed Issue 142366: For a customer enterprise site connected to Partner Gateways where one or more static routes are configured, client users working behind an SD-WAN Edge may observe intermittent traffic loss if a static route via the Primary Partner Gateway is unreachable.

    When the same static route is reachable via two or more Partner Gateways, if the route via the Partner Gateway in the Primary role is unreachable, traffic from an Edge can experience intermittent traffic loss. This issue is the result of the Edge API failing to properly check for reachability on a route lookup which causes the Edge to continue to use the Primary Partner Gateway even though reachability is false.

    The issue can be temporarily remediated by the Partner shutting down the Primary Partner Gateway until the static route becomes reachable again. Shutting down the Primary Partner Gateway prevents the Edge from including it in route reachability lookups and ensures traffic matching that static route uses a secondary Partner Gateway. However this can be disruptive for all customers using this Partner Gateway as their Primary Gateway and should be done in a maintenance window by the Partner if possible.

  • Fixed Issue 142529: When a customer switches a Cloud Security Service (CSS) GRE tunnel configuration from manual to automatic for an Edge, the Edge may experience three successive Dataplane Service failures and stop passing traffic entirely until restarted.

    The Edge will still be reachable on the management side through the Orchestrator UI and the service can be restarted through Remote Actions.When a CSS tunnel with GRE tunneling protocol is switched from manual to automatic, this causes a change in the tunneling protocol from GRE to IPsec. However the IPsec-related parameters are not configured, resulting in the IKE identification of NULL, and this triggers an exception in the Edge service and the resulting triple failure.

    On an Edge without a fix for this issue, if the customer wishes to switch from manual to automatic, the workaround is to delete the CSS configuration completely and create a new one in a maintenance window.

  • Fixed Issue 143485: Customers using SNMP monitoring may observe that they intermittently lose SNMP monitoring for hours at a time.

    The any of the below commands are run, traceback can be encountered:

    • /opt/vc/bin/snmpdebug.py --ha verp

    • /opt/vc/bin/snmpdebug.py --path

    • /opt/vc/bin/snmpdebug.py -v --arp

    The traceback triggers an error that can be seen in the affected Edge's snmpagent logs and temporarily stops SNMP monitoring.

Orchestrator Resolved Issues

Resolved in Orchestrator Version R6005-20240815-GA

Orchestrator version R6005-20240815-GA was released on 08/15/24 and is the 5th Orchestrator rollup for Release 6.0.0.

Build 6.0.0.5 adds support for provisioning and configuring Edge models 710-5G, 720, and 740 on the UI.

Note:

Edge models 710-5G, 720, and 740 must be deployed with a 5.2.4.x Edge build as they do not have 6.0.0.x Edge software support.

This Orchestrator rollup build also addresses the below critical issues since the 4th Orchestrator rollup build, R6004-20240716-GA.

  • Fixed Issue 146441: An Operator or Partner may observe assigning an Operator Profile to a customer fails.

    With this issue, when a user attempts to save the configuration change to the new profile, the Orchestrator would display the following error: CANNOT_EXCLUDE_EDGE_OVERRIDDEN_OPERATOR_PROFILE: Operator profile(s) #####,#####,#####,##### are in use by edges and cannot be excluded.

    The issue is traced to the Orchestrator backend not handling Edge software images marked as "Deprecated" properly.

    On an Orchestrator without a fix for this issue, the user can use the Classic Orchestrator to assign the affected Operator Profile.

  • Fixed Issue 147774: An Enterprise Administrator User may observe that an attempt to generate a new Report fails.

    When this issue is encountered, the Orchestrator posts "Unknown error occured". The issue is caused by the Orchestrator's reporting service caching stale, invalidated API tokens. When the reporting service attempted to fetch report-related data from the Orchestrator portal with these invalid API tokens, the calls failed.

  • Fixed Issue 147846: For Partners and Customers signed up for Zero Touch Provisioning (ZTP) on a 5.x Orchestrator, when the Orchestrator is upgraded to a 6.0.x version, the user may observe the ZTP sign-up page shows even though they are already signed up.

    This is a cosmetic issue as the customer's ZTP functionality is still in effect on the Orchestrator, but users would could not see inventory lists or assign Edges.

  • Fixed Issue 148827: After an Orchestrator is upgraded to software version 6.0.0.3 or 6.0.0.4, a customer may observe that one or more Edges show as offline on the Orchestrator UI.

    The Orchestrator is marking the Edge as offline because the Edge certificate is not being downloaded by the Orchestrator.

Resolved in Orchestrator Version R6004-20240716-GA

Orchestrator version R6004-20240716-GA was released on 07/17/24 and is the 4th Orchestrator rollup for Release 6.0.0.

Important:

This Orchestrator build remediates CVE-2024-6387, a critical vulnerability in OpenSSH. For more information on this OpenSSH vulnerability, please consult the VMware Security Advisory the KB article Broadcom Software Defined Edge Division response to CVE-2024-6387 - OpenSSH signal handler race condition vulnerability.

Note:

The 6.0.0.4 Orchestrator build, when combined with a 6.0.0.1 Gateway build, adds the Non SD-WAN Destination (NSD) IP Portability option. Portable NSD IPs allows Operators to move NSD configurations to different Gateways in the PoP without requiring the customer to reconfigure their tunnels.

There are two important caveats with NSD IP Portability:

  • NSD peers must operate as an IKE responder. In the past, some configurations would permit an initiator-only setup, but that will no longer function.

  • Peers must allow NAT-T (NAT Traversal). The current implementation of NSD Portable IP uses NAT which is internal to the PoP.

For more information, see:

In addition, this Orchestrator build addresses several critical issues since Orchestrator version R6003-20240523-GA.

  • Fixed Issue 118684: For customers using the monitoring/getEnterpriseEdgeStatus API where ClickHouse is enabled to store statistics data, the customer may observe data discrepancies for Edge statistics.

    When this issue is encountered, the API only returns the edgeLogicalId instead of returning both the edgeLogicalId and the edgeId. This behavior results in data discrepancies and user confusion as they rely on the retrieval of edge IDs for various use cases within their environments. The resolution to the issue ensures that the getEnterpriseEdgeStatus API returns the edgeLogicalId and the edgeId, bringing alignment with our documentation and providing a more predictable user experience.

  • Fixed Issue 133452: For an Orchestrator where a large number of Gateways are deployed, if a user tries to load the Gateways page, it may not load.

    The issue is caused by the API network/getNetworkGatewayPools reading more than 1000 database records when it should be reading far fewer to achieve the same result. The fix optimizes this API to efficiently return the list of Gateways for the Orchestrator with no page time outs.

  • Fixed Issue 139409: For a customer enterprise deployed on an Orchestrator where the customer deploys one or more Cloud Security Services (CSS), if the customer makes a change to a CSS on the peer side while the Orchestrator is being upgraded, an Edge in their enterprise may experience multiple Dataplane Service failures in succession and, after the third such failure, stop passing traffic until the Edge is manually restarted by the user.

    When this issue is encountered, the Orchestrator creates more than one CSS creation automation action for a single Edge WAN link, and this erroneously creates multiple CSS tunnels. The Edge receives this wrong data, which triggers an exception that causes the Edge service to fail. The nature of the issue means that even after the Edge recovers from the initial failure, the Orchestrator will continue to provide the erroneous tunnel information, causing additional failures and, on the third such failure in succession, the Edge defensively stops trying to recover which results in all customer traffic dropping.

    On an Orchestrator without a fix for this issue, the user must clear the erroneous CSS tunnel information the Orchestrator is generating to remediate the issue, and this is done by disabling all CSS instances for that Edge, wait for the tunnels to be deleted, and then re-enable all CSS instances.

  • Fixed Issue 144891: When a user navigates to Monitor > Edge > Sources tab, they cannot change the hostname for a Client.

    A user should have the option to change the hostname for a client by clicking the Edit icon and opening the Change Hostname box. While they can enter in the text under the Change Hostname field, when they click Save Changes, the new hostname is not applied.

  • Fixed Issue 145134: For a customer enterprise using Zero Touch Provisioning (ZTP) and who deploy one or more sites with a High Availability topology, users may observe deployed HA Edge serial numbers in the ZTP Available inventory list.

    When an Orchestrator requests the Edge inventory from the Maestro service, it is expected to exclude Edges that are being used in HA pairs. The issue is caused by the Orchestrator not checking if the incoming serial numbers are specifically used as HA pair serial numbers and results in the Orchestrator hiding some of HA serial numbers, but not all of them.

  • Fixed Issue 145783: On some Orchestrators that use customized branding, users may observe that control buttons at the top of the page are difficult to locate.

    Some branded Orchestrators have a header color scheme that matches the colors of the top-right controls (Help, User Information, and Menu) on the Global Settings page, with the result that the buttons blend with the Orchestrator coloring and become difficult to locate.

    This issue does not affect VMware cloud hosted Orchestrators, which use a default color scheme whose header colors contrast with interface buttons.

  • Fixed Issue 145810: For a customer enterprise that deploys two or more Non SD-WAN Destination (NSD) via Gateway where redundant Gateways are configured, if the user has BGP for one NSD, and then looks to enable BGP for an additional NSD, they may observe that the Orchestrator automatically changes the Gateway assignment to another Gateway already in use by a different NSD.

    In order to enable BGP on a second NSD, the redundant Gateways have to be ones unused by the first NSD already configured with BGP. To meet this condition, the user manually changes the second NSD's redundant Gateway assignments to ones not used by the first NSD. The issue is that, upon enabling BGP, that the Gateways are reverted back to the original Gateways already in use by the first NSD.

Resolved in Orchestrator Version R6003-20240523-GA

Orchestrator build R6003-20240523-GA was released on 05-28-2024 and is the 3rd Orchestrator rollup for Release 6.0.0.

This Orchestrator rollup build addresses the below critical issues since the 2nd Orchestrator rollup build, R6002-20240510-GA.

  • Fixed Issue 134814: A customer using Zero Touch Provisioning (ZTP) may observe that their inventory of Edges is not being represented correctly on the Orchestrator.

    When the Orchestrator pulls the pending inventory from an internal process, if some inventory Edges already have logical Edges with the same serial numbers, the Orchestrator moves these inventory Edges to "Assigned inventory" automatically. During this operation the Orchestrator does not take into account if the logical Edge is already activated. This can lead confusion for Partners or Customers using ZTP.

  • Fixed Issue 141617: When a customer enterprise Profile is updated, on the Monitor > Events page the "Profile updated" Event Detail has the Principle field showing as "unknown".

    For the "Profile updated" Event, a user should see when clicking on the Event to open the Event Detail box, the user should observe a value for the "principle" field. With this issue that value is always "unknown" and prevents the user from knowing what aspect of the Profile was updated.

  • Fixed Issue 143577: On the Edge > Configure > Device page of the Orchestrator UI, changes to an Edge's configuration do not save if the Edge does not have a license.

    The Orchestrator UI will not allow a user to create an Edge without binding it to a license, so this only impacts API users that did not associate a license to an Edge. Or an Edge without a license, the Orchestrator sometimes does not perform license validation when saving Edge device configuration changes even though that field is meant to be mandatory.

    On an Orchestrator without a fix for this issue, the customer or partner needs to associate a license to the API-created Edge before attempting to modify its configuration settings.

    In addition, the UI does not highlight the error except for adding an asterisk (*) to required fields. This part of the issue will be fixed in a future release.

  • Fixed Issue 144613: On the Edge/Profile > Configure > Firewall page of the Orchestrator UI, a user may observe they cannot save a Firewall configuration after modifying it.

    This issue is found on Orchestrators upgrade to Release 6.0.0.1. After the upgrade to this version, legacy firewall rules are not being correctly updated, and this results in additional firewall module updates attempts to fail.

  • Fixed Issue 144629: If an Operator adds a new Certificate Authority (CA) authority to the Orchestrator, if a user renews the certificate on an Edge, paths to the Gateway may be torn down with traffic disruption resulting from this.

    With this issue, the CA chain is not being sent to the Edge, which results in the paths to the Gateway being torn down.

  • Fixed Issue 144992: On the Configure > Cloud Hub page of the Orchestrator UI, a user may observe that they cannot configure either a Cloud Hub or a Credential.

    The Cloud Hub page would show as blank on the UI. The is the result of a defect in the Orchestrator microservice related to Cloud Hubs.

  • Fixed Issue 145188: An Operator may observe that an Orchstrator upgrade to 6.0.0.2 fails.

    The failure can occur for Orchestrators with high file I/O. During the upgrade some files may be deleted during the execution of file_store metadata file generation. In this situation, the Operator would see an error similar to below in the /var/log/vco_software_update.log.

    xxxx-xx-xx 09:26:28,547 - UPGRADE - WARNING - Command '/store2/velocloud/software_update/_staging/scripts/P10-1-copy-vco-conffiles.sh' failed with error 1
xxxx-xx-xx 09:26:28,547 - UPGRADE - WARNING - =========== Upgrade failed ===========
xxxx-xx-xx 09:26:28,965 - UPGRADE - ERROR - installer failed with return code 1. See /var/log/vco_software_update.log for details

     If upgrading to the Orchestrator to 6.0.0.2 or earlier on a large scale Orchestrator, the Operator should stop Orchestrator services prior to starting the upgrade with this command: systemctl stop backend upload portal reporting.

    Services will start back up after the Orchestrator upgrade automatically. 

Resolved in Orchestrator Version R6002-20240510-GA

Orchestrator build R6002-20240510-GA was released on 05-13-2024 and is the 2nd Orchestrator rollup for Release 6.0.0.

This Orchestrator rollup build addresses the below critical issues since the 1st Orchestrator rollup build, R6001-20240421-GA.

  • Fixed Issue 99891: When a Partner Administrator logs onto the VMware SASE Orchestrator and uses the New UI, when they are on the Manger Partner Customers page and select Customer > Global Settings > Customer Configuration, they will observe an error.

    The error message reads "not allowed to method" when the Partner Administrator attempts to access the customer configuration even though they have full privileges to that page.

  • Fixed Issue 123962: A user may observe that after downloading a custom role package from the Orchestrator, when the user attempts to upload this package using the 'Upload Permission' option to another Orchestrator or under a different name, the upload is not successful.

    When this issue is encountered, the user would also observe the error "The scope specified in the package is not accessible to the user", and the new permissions never appear in the Service Permissions list at the Customer level. The issue is the result of the Orchestrator failing to open the uploaded custom role package.

  • Fixed Issue 141974: On the Authentication > User Management > Service Permissions page for the Orchestrator UI, if a user creates a new Enterprise Security admin user where that role is denied the Update privilege for Customer General Information under Enterprise Settings, that user can continue to update that section of the UI.

    The Orchestrator is not applying the customized user privilege to this role package and the result is the Enterprise Security user can upate the Customer General Information section.

  • Fixed Issue 142787: If a user switches to a different Segment on the Orchestrator UI, they may observe that Business Policy Rules are missing on the new segment page.

    When the switched Segment does not have rules setup specifically for it, the Orchestrator UI does not prepopulate the data with the default Business Policy rules.

  • Fixed Issue 142878: On the Authentication > User Management > Service Permissions page for the Orchestrator UI, if a user removes the privilege "Edge Device Configuration Visibility Mode" from the Standard Operator user role, the Operator with that role can still override the Edge Visibility mode.

    In effect, the Standard Operator can check the Override box for the Visibility Mode feature for any Edge and save that change.

  • Fixed Issue 144140: A user with a customized role that denies them the privilege of Network Services - Create can still update Cloud Security Services (CSS) details on the Orchestrator UI.

    Users with this customized role can access the Cloud Security Service list details in the Configure > Network Services section, and change CSS details.

  • Fixed Issue 144175: When a user makes a configuration change at the Profile level on the Orchestrator, they may observe that it takes a long time (up to 4 hours) for the changes to be applied to the Edges using the Profile.

    The issue is related to the control plane spread factor variable, which is set in Orchestrator's System Properties. When a spread factor is set, the configuration update to the Edges is usually spread over a few heartbeats. For example, if the control plane update is set to 20, it should take up to 20 heartbeats for all Edges to receive the configuration update. When this issue is encountered, the spread factor does not work correctly and changes can take hours to be applies to all Edges in the Profile.

    On an Orchestrator without a fix for this issue, the workaround is to trigger an Edge specific configuration change to force that Edge's control plane to update immediately.

  • Fixed Issue 144280: When looking at the Edge > Monitor > Flows page of the Orchestrator UI, the Business Policy column may show as empty.

    Each flow has a Business Policy rule associated with it and should have that rule displayed, for this issue none of the flows shows a rule on the UI.

  • Fixed Issue 144393: When an Orchestrator is upgraded to version 6.0.0.1, the Operator user may observe that the migration of some customer statistics fails.

    The issue specifically impacts the /store2/velocloud/file_store folder, causing operational disruptions by misaligning user and group information to its subdirectories and preventing the Orchestrator from processing the backlog of files mostly related to PATHSTATS.

  • Fixed Issue 144405: On the Edge > Configure page of the Orchestrator UI, the Assign Software Image box is missing a description of the Edge software image.

    For each Edge software image the UI should include the description found in the Operator Profile but for this issue there is only the name showing.

  • Fixed Issue 144469: After an Orchestrator is upgraded to version 6.0.0.1, users who log in with Single Sign On (SSO) may have difficulty logging in.

    With this issue, users are only able to authenticate after multiple attempts because some SSO logins are timing out too quickly on the Orchestrator.

  • Fixed Issue 144789: When an Orchestrator is upgraded to version 6.0.0.1, users may observe that they cannot download reports.

    This issue can occur if the report storage location is not the default one, after the container is restarted the Orchestrator is looking for the reports in the default location and not the custom one.

  • Fixed Issue 144825: When an Orchestrator is upgraded to version 6.0.0.1, customers may observe that Edge activations and Edge certificate renewals are failing.

    As part of the Orchestrator upgrade, the master back-end node is also being restarted, which resets the Certificate Authority (CA) passphrase. This affects customers by preventing them from activating Edges, and causing Edge certificate renewals to fail. On an affected Orchestrator, an Operator needs to force reset the CA passphrase to temporarily remediate the issue.

Resolved in Orchestrator Version R6001-20240421-GA

Orchestrator build R6001-20240421-GA was released on 04-23-2024 and is the 1st Orchestrator rollup for Release 6.0.0.

This Orchestrator rollup build addresses the below critical issues since the original GA build, R6000-20240409-GA.

  • Fixed Issue 137150: On the Authentication > User Management > Service Permissions page for the Orchestrator UI, if a user has their Read or Create privilege removed for the Service Permissions page, that user may observe that entire User Management tab has been removed.

    The Orchestrator is removing the entire User Management page including the roles and authentications when it should only limit access to a subset of that screen.

  • Fixed Issue 142575: A user may observe that they cannot save a configuration profile which is duplicated form an existing profile if that profile has IDS/IPS activated for it.

    The upgrade to 6.0.0.0 includes a patch to extract all firewall modules across all enterprises using their enterprise logical ID. For some enterprises this is null and those modules were not updated with Security Service Group configurations. The result was those modules retaining pre-6.0.0 Enhanced Firewall Service type configurations. When the profile is cloned and and a save is attempted, the configuration validation checks fail and an error was thrown to the user.

  • Fixed Issue 142990: For customer enterprises utilizing the API on the Orchestrator, the API definition for generateVpnGatewayConfiguration is missing in the request and response Swagger specification.

    As a result of a change in the JSON schema to allow the API to use either the old or new format for request and response, the API definition is missing in the the Swagger API specification for generateVpnGatewayConfiguration.

    This issue does not affect the functionality of generating the VPN Gateway configuration, as these APIs will work as expected.

  • Fixed Issue 142991: For an Orchestrator deployed with a Disaster Recover (DR) topology, DR may fail after upgrading both the Active and Standby Orchestrator to Release 6.0.0.

    The issue is caused by missing tables in the dockerized service related tables. As a result, the Standby Orchestrator fails to replicate the database copy from the Active Orchestrator.

    Should an Operator encounter this issue the steps to remediate it are as follows:

    1. Break DR.

    2. Set vco.disasterRecovery.skip.chStagingDeletion to True on the Standby Orchestrator.

    3. Reconfigure DR.

  • Fixed Issue 143621: When an Orchestrator is upgraded to the 6.0.0.0 software build, a user may observe that the UI is inaccessible.

    The issue is the result of the Orchestrator's database management system (ClickHouse) failing to start due to a permission error.

Resolved in Orchestrator Version R6000-20240409-GA

Orchestrator version R6000-20240409-GA was released on 04-11-2024 and resolves the following issues since Orchestrator version R5406-20240323-GA. This means that a fix for an Orchestrator issue listed in the 5.4.0 Release Notes up to the 5.4.0.6 build is included in all Release 6.0.0 builds.

  • Fixed Issue 97078: When a user creates a Zscaler Cloud Security Service (CSS) tunnel, the user cannot change the name of the tunnel.

    When a user creates a Zscaler CSS tunnel, the location name will be "edge_<logicalId>" which is not ideal when managing a large network. Orchestrator Release 6.0.0 adds the option to edit this field and add a name to the logical ID for that Edge.

  • Fixed Issue 104395: The Monitor > Network > Overview page's "Links Down" section shows only the first 10 sites with links down. When a user clicks on the "View All" button, the Orchestrator UI takes the user to the Edges section which lists all Edges regardless of link status even though the user only wants to see sites with Edges that have one or more links down.

    When a user clicks on the “Links down” icon, the list should show all down links, not just 10 and there is no possibility to filter Edges by link or Edge status after the 'Show All' button is clicked.

  • Fixed Issue 105414: On the Configure > Profile/Edge > Device > Cloud VPN page of the Orchestrator UI, if a user clicks on Edit Hub, the option 'Auto Select VPN Hub' is shown even though this option has no functionality.

    The check box for Auto Select VPN Hub does nothing and creates confusion for customers trying to use it. This option is removed on the 6.0.0 Orchestrator.

  • Fixed Issue 105568: When attempting to create a Custom Report on the Orchestrator UI, a user may not be able to click the NEXT button at the Select Edges step of the wizard.

    The issue is the result of a race condition between the time the filter is applied and when the Orchestrator UI validates the presence of selected Edges. 

  • Fixed Issue 105584: When attempting to create a Custom Report on the Orchestrator UI, a user may not be able to click the NEXT button at the Select Edges step of the wizard.

    The issue is the result of a race condition between the time the filter is applied and when the Orchestrator UI validates the presence of selected Edges.

  • Fixed Issue 108266: On the User Management page of the Orchestrator UI, a User page shows the Activation State instead of the Status, and Locked Status is not shown at all.

    This issue is seen at all user levels (Enterprise, Partner, and Operator). Both the Users list and individual user's details page show Activation State field. This should read as Activation Status. Information if a user is locked is not shown on the user's detail page.

  • Fixed Issue 113021: Portal APIs that have pagination support do not work properly if "sortBy" parameter is not specified in the request body. Duplicate entries are found in the response when walking through pagination.

    If a user makes an API call to any paginated API such as "/enterprise/getEnterpriseEdges" with "limit" parameter specified in the request body but not "sortBy" parameter, when walking through the pagination by following "nextPageLink" in the response, duplicate entries are seen in the response.

    Without a fix for this in the Orchestrator, a user should always include "sortBy" parameter in the request body of paginated APIs.

  • Fixed Issue 122365: Users does not have an explicit field in the Orchestrator to add in a Twilio messagingServiceSid field value to leverage Twilio’s Messaging Service functionality.

    With new SMS regulations enforced in several countries, it is mandated to use country specific local number to send local SMS. This change has increased complexity and, to reduce this complexity, Twilio has introduced Messaging Services.

    Messaging Service has a `Sender Pool` that allows a user to add one or more Twilio sender numbers (country specific numbers) and based on the 'To' number, Twilio’s Messaging Service selects the best sender from the Sender pool and sends out the SMS.

    In order to leverage this functionality, the Orchestrator has been enhanced to accept a new System Property field: service.twilio.messagingServiceSid (MessagingServiceSid value is generated in Twilio when Messaging Service is created).

    If a user wants to leverage Twilio’s Messaging Service:

    • The user should create and configure the Messaging Service in Twilio.

    • Create a new string field System Property in the Orchestrator by the name service.twilio.messagingServiceSid and add the MessagingServiceSid value to it.

    • Once service.twilio.messagingServiceSid is added, service.twilio.phoneNumber is no longer used. The user should remove the service.twilio.phoneNumber System Property.

    As a workaround for an Orchestrator without this enhancement, a user can use the existing field service.twilio.phoneNumber to add in the messaging Service Sid value and still continue to leverage Twilio's Messaging Service functionalities. But this is not officially documented on Twilio's website.

  • Fixed Issue 123091: On the Configure > Edges page of the Orchestrator UI, when a user is attempting to Assign Software Image to the selected Edges, they observe all the Operator Profiles available on that Orchestrator versus just the ones assigned to that enterprise by the Partner.

    This display can cause confusion for the customer and the expectation is that they would only see the Operator Profiles they can actually use.

  • Fixed Issue 125298: On the Orchestrator UI, if the resolution height is less than 600 px, the user may observe that some buttons merge together.

    The "My Account" And "Logout" can merge with each other at lower resolutions and the scroll bar is also not there to move the page vertically. 

  • Fixed Issue 126409: On the Gateways page, a user can select a federated Gateway and observe that they have the option to delete it through the Orchestrator UI.

    There should be no option to delete a Federated Gateway from the Orchestrator UI as they are shared across other Orchestrators which may still be using them. The Orchestrator backend already prevents the actual deletion of a Federated Gateway, but this option should not be presented.

  • Fixed Issue 126518: On the Configure > Profile > Device > Routing & NAT > BGP section of the Orchestrator UI, a user may observe that the symbol to clone a BGP Filter Rule is the same as the symbol to delete the rule.

    A minor cosmetic issue that could cause the user to click on the wrong action and since the other action is delete, it would have a meaningful impact if the change was saved.

  • Fixed Issue 128355: On the Configure > Edge > VPN Services > Cloud Security Service page of the Orchestrator UI, a user can configure a Zscaler CSS to use IKEv1 at the Edge level on the Orchestrator UI.

    If a user changes the Override status to True, they can change the Key Exchange Protocol to IKEv1 when the Profile specifies IKEv2.

  • Fixed Issue 128478: On the Global Settings > Customer Configuration > Configure Hand Off > Configure BFD & BGP > Static Routes page of the Orchestrator UI, a user can configure duplicate subnets for enterprise level static routes.

    The Orchestrator lacks a validation to confirm subnet uniqueness on this page which can result in traffic issues for the customer enterprise.

  • Fixed Issue 128631: On the Global Settings > Customer Configuration > Gateway Pool > Partner Hand Off > Customer BGP Priority page of the Orchestrator UI, the 'Community' and 'Community 2' fields accept a value with any range.

    The Orchestrator lacks a value range validation for these fields. The fix for this issue enforces a range of 0 to 65535 and will throw an error for anything outside of that.

  • Fixed Issue 128979: The Manage Customers page may take up to 10 seconds to load.

    This issue can be encountered on an Orchestrator with >2000 enterprises. In such a case, the API can take 5 - 10 seconds to return. The extended load time is caused by the getNetworkEnterprises API call taking a long time to return. The fix for this issue restores page load time back to less than a second.

  • Fixed Issue 129156: On the Configure > Edges screen of the Orchestrator UI, when a user tries to assign an Operator Profile to several Edges, each of which is using a different Operator Profile, the available Operator Profiles are not shown in the table.

    The Assign Operator Profile screen simply displays a "No items found to select" message and the user cannot assign any Operator Profile to these selected Edges. Instead the user needs to check all the Edges using the same Operator Profile only and assign the new Operator Profile in that manner.

  • Fixed Issue 130577: An enterprise user cannot enable or disable the Self Service Password Reset option for their account.

    When an enterprise user makes a change to the Self Service Password Reset option, only the Two Factor Authentication parameter is changed.

  • Fixed Issue 130870: If a user generates and downloads a User List report from the Orchestrator, the CSV file does not include the Two Factor Authentication column.

    The report column is labeled Two Factor and would confirm whether that user is using Two Factor Authentication by listing either a 0 or 1.

  • Fixed Issue 130943: A customer may observe that on the Monitor > Links page for an Edge that statistics are missing.

    LINK_DEAD QoE events are not inserted when the link's internalId is passed by the Edge instead of its logicalId. When this occurs, the pushLinkQualityEvents API schema fails validation. This would be observed in the Orchestrator logs.

  • Fixed Issue 131809: On customer enterprise's Monitor > Events page of the Orchestrator UI, if a user attempts to use the search filter to filter for Event Name, the list is not complete.

    The search filter dropdown menu is missing several event names, and this prevents users from searching for just Events that include that event name.

  • Fixed Issue 133079: A user cannot configure an IPv6 address if the configuration does not include a Gateway IP address for the WAN-enabled routed interface.

    Under IPv6 Settings, when a user enables IPv6 on an Edge's routed interface, selects Addressing Type as Static, and populated only the IP Address and CIDR Prefix values while leaving the Gateway field empty, the user will not be able to save the configuration as the Gateway field is mandatory. The issue is that the Orchestrator UI does not mark this field as mandatory and also does not throw an error with a reason for the error when attempting to save.

  • Fixed Issue 135479: An Operator Standard Administrator can change the Certificate Mode for an enterprise even though they do not have the privilege to do so.

    API clients cannot update Enterprise PKI (Private Key Infrastructure) mode via updateEnterprise API.

    The Orchestrator allows updates to an endpoint PKI mode via the updateEnterprise API call. However, the privilege mapping for that update is incorrect. As a result, the endpoint PKI mode should only be updated via the enterprise/updateEnterpriseEndpointPkiMode which expects UPDATE, ENTERPRISE_PKI.

  • Fixed Issue 135854: Under Configure > Network Services of the Orchestrator UI, a user can create, delete, and update the IaaS and Cloud Subscriptions even though their privilege to do so was removed.

    The privilege can be removed in the Service Permissions screen for Cloud Subscription Service and the expected behavior is for a user to not be able to take any action that is turned off for that set of Privileges.

  • Fixed Issue 136085: On the Authentication > User Management > Service Permissions page for the Orchestrator UI, if a user creates a Privilege Bundles for an Enterprise Superuser or Standard Admin and includes the Profile Device Authentication Settings privilege set, if the user turns off the Create option, either user type can still create services.

    Orchestrator is not enforcing customer packages that include the Profile Device Authentication Settings privilege set.

  • Fixed Issue 136672: On the Global Settings > Customer Configuration > Security Policy page of the Orchestrator UI, the Encryption options always show 'AES 128 CBC' when they should show 'AES 128'.

    AES 128 CBC should only show as an option if the Turn off GCM check box is selected.

  • Fixed Issue 136829: On the Configure > Edge/Profile > Business Policy page of the Orchestrator UI, when creating a new rule and using Source = Object Groups, the Address Group selector does not open up to a new popup dialogue.

    The desired behavior is for this popup to reveal descriptions and offer filtering/searching. When selected, the Address Group's name and description should also display in the Edit dialog.

  • Fixed Issue 137233: On the Global Settings > Customer Configuration > Gateway Pool > Partner Hand Off > Customer BGP Priority page of the Orchestrator UI, integer values for a Community greater than 65535 will not be properly applied on the Edges.

    BGP filters can also be configured at the Configure > Edge/Profile > Device page or the Configure > Network Services > Non SD-WAN Destination > BGP. In all these BGP filters, it is possible to select an action in the filter with type as 'community'. Here, any community value greater than 65535 in integer format will not be properly applied on the Edges.

  • Fixed Issue 138417: Users cannot access the Orchestrator's microservices micro frontend.

    When an Operator Superuser updates the microservices redirection URL System Property: xxxRedirectionUrl twice quickly in succession, the change is not reflected in the Orchestrator's nginx configurations.

    Without a fix, the Operator Superuser needs to manually update the nginx configuration file with the correct redirection URL.

  • Fixed Issue 138712: On a customer enterprise which uses a Hub/Spoke topology and where Internet Backhaul is configured for the Hub Edges, a user may not observe the option to select Backhaul Hubs in the drop down list of options when configuring a Business Policy rule.

    To show the Backhaul Hub Edges in the drop-down, the Orchestrator uses the name of the object, which is unique to each Backhaul Hub in mapping with the current rule. The issue is triggered if the Hub Edge name is updated in the configuration module.

  • Fixed Issue 138896: The Orchestrator may send the wrong Wi-Fi Radio settings to an Edge.

    In a situation where the Edge is in a country that does not permit 5 GHz (For example, China or Taiwan), OR if the country is not set. In either case, the Orchestrator should force the Edge Wi-Fi radio to use 2.4 GHz if the Edge is not otherwise overridden by the user. This is not occurring.

  • Fixed Issue 138964: On the Configure > Edge > Device page of the Orchestrator UI, a user may find when they click "Add VLAN", nothing happens.

    The issue arises out of a missing IPv6 detail in an existing VLAN configuration. As a result, when a new VLAN creation is attempted, the Orchestrator UI first retrieves the list of existing VLANs, and this triggers an error and prevents any new VLANs being created until this detail is fixed in the existing VLAN.

  • Fixed Issue 139261: On the Configure > Edge > Device page of the Orchestrator UI, when configuring a VLAN, an IPv4 address ending in 255 (for example, xxx.xx.xx.255) throws an error and the configuration cannot be saved.

    The error reads "Invalid IP address" even though this is a valid address when configuring a IPv4 DHCP server for a VLAN.

  • Fixed Issue 139744: When configuring NTP private server configuration leads to an error message that states "Invalid NTP source interface".

    In an NTP private server configuration from a Profile, validation fails when the Source Interface value is null in the NTP module. The fix includes a validation to check for a null value to avoid this validation error.

  • Fixed Issue 139799: On the Global Settings > Customer Configuration > SD-WAN page of the Orchestrator UI, a user may observe that they cannot edit the SD-WAN License section.

    A user with suitable privileges should have the option to edit their list of licenses, but that is not the case with this issue.

  • Fixed Issue 139900: On the Monitor > Network Services page for a Zscaler CSS (Cloud Security Service), a user may observe multiple action entries in a NOTIFIED state that are stuck in that state for a long time.

    These states can last for over a month or more. The issue is the result of a defect in the Orchestrator's backend API that checks for stale job status and fails to collect the right set of actions and instead just returns an empty array. So the stuck jobs just stay stuck because they are never retriggered.

  • Fixed Issue 140263: Cluster information does not show for the Gateway logs.

    This issue is the result of the Orchestrator sending a null value for the field autorebalance, where the Auto Re-Balance option is set to off for the Edge Cluster. When this occurs, the Edge cannot parse the JSON value and does not create logs for the Cluster.

  • Fixed Issue 140322: When attempting to create a new profile on a customer enterprise, the user may find the effort times out on the Orchestrator UI.

    Even though the UI time's out the request for a new enterprise profile, the Orchestrator's backend does complete the API request which would be revealed if refreshing the browser ~120 seconds after the timeout. This issue is most likely encountered for an enterprise which uses a large number of Cloud Security Service site objects or generally where there is a feature that creates network_service enterprise objects.

  • Fixed Issue 140640: On the Edge > Monitor > Flows page of the Orchestrator UI, the user cannot view a flow matching a business policy for an Edge may not be seen.

    A customer should be able to view the Business Policy rule associated with their enterprise or only an Edge under the Monitor > Flows screen.

Known Issues

Open Issues in Release 6.0.0.

Edge/Gateway Known Issues

  • Issue 14655:

    Plugging or unplugging an SFP adapter may cause the device to stop responding on the Edge 540, Edge 840, and Edge 1000 and require a physical reboot.

    Workaround: The Edge must be physically rebooted. This may be done either on the Orchestrator using Remote Actions > Reboot Edge, or by power-cycling the Edge.

  • Issue 25504:

    Static route costs greater than 255 may result in unpredictable route ordering.

    Workaround: Use a route cost between 0 and 255.

  • Issue 25595:

    A restart may be required for changes to static SLA on a WAN overlay to work properly.

    Workaround: Restart Edge after adding and removing Static SLA from WAN overlay.

  • Issue 25742:

    Underlay accounted traffic is capped at a maximum of the capacity towards the VMware SD-WAN Gateway, even if that is less than the capacity of a private WAN link which is not connected to the Gateway.

  • Issue 25758:

    USB WAN links may not update properly when switched from one USB port to another until the VMware SD-WAN Edge is rebooted.

    Workaround: Reboot the Edge after moving USB WAN links from one port to another.

  • Issue 25885:

    A large configuration update on the Partner Gateway (e.g. 200 BGP-configured VRFs) may cause latency to increase for approximately 2-3 seconds for some traffic via the VMware SD-WAN Gateway.

    Workaround: No workaround available.

  • Issue 25921:

    VMware SD-WAN Hub High Availability failover takes longer than expected (up to 15 seconds) when there are three thousand branch Edges connected to the Hub.

    Workaround: No workaround available.

  • Issue 25997:

    The VMware SD-WAN Edge may require a reboot to properly pass traffic on a routed interface that has been converted to a switched port.

    Workaround: Reboot the Edge after making the configuration change.

  • Issue 26421:

    The primary Partner Gateway for any branch site must also be assigned to a VMware SD-WAN Hub cluster for tunnels to the cluster to be established.

    Workaround: No workaround available.

  • Issue 28175:

    Business Policy NAT fails when the NAT IP overlaps with the VMware SD-WAN Gateway interface IP.

    Workaround: No workaround available.

  • Issue 31210:

    VRRP: ARP is not resolved in the LAN client for the VRRP virtual IP address when the VMware SD-WAN Edge is primary with a non-global CDE segment running on the LAN interface.

    Workaround: No workaround available.

  • Issue 32731:

    Conditional default routes advertised via OSPF may not be withdrawn properly when the route is deactivated.

    Workaround: Reactivating the route, followed by deactivating it again will retract it successfully.

  • Issue 32960:

    Interface “Autonegotiation” and “Speed” status might be displayed incorrectly on the Local Web UI for activated VMware SD-WAN Edges.

    Workaround: Refer to the Orchestrator UI under Remote Diagnostics > Interface Status.

  • Issue 32981:

    Hard-coding speed and duplex on a DPDK-configured port may require a VMware SD-WAN Edge reboot for the configurations to take effect as it requires turning DPDK off.

    Workaround: There is no workaround for this issue.

  • Issue 34254:

    When a Zscaler CSS is created and the Global Segment has FQDN/PSK settings configured, these settings are copied to Non-Global Segments to form IPsec tunnels to a Zscaler CSS.

  • Issue 35778:

    When there are multiple user-defined WAN links on a single interface, only one of those WAN links can have a GRE tunnel to Zscaler.

    Workaround: Use a different interface for each WAN link that needs to build GRE tunnels to Zscaler.

  • Issue 36923:

    Cluster name may not be updated properly in the NetFlow interface description for a VMware SD-WAN Edge which is connected to that Cluster as its Hub.

  • Issue 38682:

    VMware SD-WAN Edge acting as a DHCP server on a DPDK-configured interface may not properly generate “New Client Device" events for all connected clients.

  • Issue 38767:

    When a WAN overlay that has GRE tunnels to Zscaler configured is changed from auto-detect to user-defined, stale tunnels may remain until the next restart.

    Workaround: Restart the Edge to clear the stale tunnel.

  • Issue 39374:

    Changing the order of VMware SD-WAN Partner Gateways assigned to a VMware SD-WAN Edge may not properly set Gateway 1 as the local Gateway to be used for bandwidth testing.

  • Issue 39608:

    The output of the Remote Diagnostic “Ping Test” may display invalid content briefly before showing the correct results.

    Workaround: There is not workaround for this issue.

  • Issue 39624:

    Ping through a subinterface may fail when the parent interface is configured with PPPoE.

    Workaround: There is no workaround for this issue.

  • Issue 39753:

    Toggling Dynamic Branch-to-Branch VPN to off may cause existing flows currently being sent using Dynamic Branch-to-Branch to stall.

    Workaround: Only deactivate Dynamic Branch-to-Branch VPN in a maintenance window.

  • Issue 40421:

    Traceroute is not showing the path when passing through a VMware SD-WAN Edge with an interface configured as a switched port.

  • Issue 40096:

    If an activated VMware SD-WAN Edge 840 is rebooted, there is a chance an SFP module plugged into the Edge will stop passing traffic even though the link lights and the VMware SD-WAN Orchestrator will show the port as 'UP'.

    Workaround: Unplug the SFP module and then replug it back into the port.

  • Issue 42278:

    For a specific type of peer misconfiguration, the VMware SD-WAN Gateway may continuously send IKE init messages to a Non-SD-WAN peer. This issue does not disrupt user traffic to the Gateway; however, the Gateway logs will be filled with IKE errors and this may obscure useful log entries.

  • Issue 42872:

    Activating Profile Isolation on a Hub profile where a Hub Cluster is associated does not revoke the Hub routes from the routing information base (RIB).

  • Issue 43373:

    When the same BGP route is learnt from multiple VMware SD-WAN Edges, if this route is moved from preferred to eligible exit in the Overlay Flow Control, the Edge is not removed from the advertising list and continues to be advertised.

    Workaround: Activate Distributed Cost Calculation (DCC) on the VMware SD-WAN Orchestrator.

  • Issue 44995:

    OSPF routes are not revoked from VMware SD-WAN Gateways and VMware SD-WAN Spoke Edges when the routes are withdrawn from the Hub Cluster.

  • Issue 45189:

    With source LAN side NAT is configured, the traffic from a VMware SD-WAN Spoke Edge to a Hub Edge is allowed even without the static route configuration for the NAT subnet.

  • Issue 45302:

    In a VMware SD-WAN Hub Cluster, if one Hub loses connectivity for more than 5 minutes to all of the VMware SD-WAN Gateways common between itself and its assigned Spoke Edges, the Spokes may in rare conditions be unable to retain the hub routes after 5 minutes. The issue resolves itself when the Hub regains contact with the Gateways.

  • Issue 46053:

    BGP preference does not get auto-corrected for overlay routes when its neighbor is changed to an uplink neighbor.

    Workaround: An Edge Service Restart will correct this issue.

  • Issue 46137:

    A VMware SD-WAN Edge running 3.4.x software does not initiate a tunnel with AES-GCM encryption even if the Edge is configured for GCM.

    Workaround: If a customer is using AES-256, they must explicitly deactivate GCM from the Orchestrator prior to upgrading their Edges to a 4.x Release. Once all their Edges are running a 4.x release, the customer may choose between AES-256-GCM and AES-256-CBC.

  • Issue 46216:

    On a Non SD-WAN Destinations via Gateway or Edge where the peer is an AWS instance, when the peer initiates Phase-2 re-key, the Phase-1 IKE is also deleted and forces a rekey. This means the tunnel is torn down and rebuilt, causing packet loss during the tunnel rebuild.

    Workaround: To avoid tunnel destruction, configure the Non SD-WAN Destinations via Gateway/Edge or CSS IPsec rekey timer to less than 60 minutes. This prevents AWS from initiating the re-key.

  • Issue 46391:

    For a VMware SD-WAN Edge 3800, the SFP1 and SFP2 interfaces each have issues with Multi-Rate SFPs (i.e. 1/10G) and should not be used in those ports.

    Workaround: Please use single rate SFP's per the KB article VMware SD-WAN SupportedSFP Module List (79270). Multi-Rate SFPs may be used with SFP3 and SFP4.

  • Issue 47664:

    In a Hub and Spoke configuration where Branch-to-Branch via Hub VPN is not configured, trying to U-turn Branch-to-Branch traffic using a summary route on an L3 switch/router will cause routing loops.

    Workaround: Configure Cloud VPN to activate Branch-to-Branch VPN and select “Use Hubs for VPN”.

  • Issue 47681:

    When a host on the LAN side of a VMware SD-WAN Edge uses the same IP as that Edge’s WAN interface, the connection from the LAN host to the WAN does not work.

  • Issue 48530:

    VMware SD-WAN Edge 6x0 models do not perform autonegotiation for triple speed (10/100/1000 Mbps) copper SFP's.

    Workaround: Edge 520/540 supports triple speed copper SFPs but this model has been marked for End-of-Sale by Q1 2021.

  • Issue 50518:

    On a VMware SD-WAN Gateway where PKI is configured, if >6000 PKI tunnels attempt to connect to the Gateway, the tunnels may not all come up because inbound SAs do not get deleted.

    Note:

    Tunnels using pre-shared key (PSK) authentication do not have this issue.

  • Issue 52955: DHCP decline is not sent from Edge and DHCP rebinding is not restarted after DAD failure in Stateful DHCP.

    If DHCPv6 server allocates an address which is detected as duplicate by the kernel during a DAD check then the DHCPv6 client does not send a decline. This will lead to traffic dropping as the interface address will be marked as DAD check failed and will not be used. This will not lead to any traffic looping in the network but traffic blackholing will be seen.

    Workaround: There is no workaround for this Issue.

  • Issue 53219: After a VMware SD-WAN Hub Cluster rebalances, a few Spoke Edges may not have their RPF interface/IIF set properly.

    On the affected Spoke Edges, multicast traffic will be impacted. What happens is that after a cluster rebalance, some of the Spoke Edge fail to send a PIM join.

    Workaround: This issue will persist until the affected Spoke Edge has an Edge Service restart.

  • Issue 53934: In an enterprise where a VMware SD-WAN Hub Cluster is configured, if the primary Hub has Multihop BGP neighborships on the LAN side, the customer may experience traffic drops on a Spoke Edge when there is a LAN side failure or when BGP is not configured on all segments.

    In a Hub cluster, the primary Hub has Multihop BGP neighborship with a peer device to learn routes. If the physical interface on the Hub by which BGP neighborship is established, goes down, then BGP LAN routes may not become zero despite BGP view being empty. This may cause Hub Cluster rebalancing to not happen. The issue may also be observed when BGP is not configured for all segments and when there are one or more Multihop BGP neighborships.

    Workaround: Restart the Hub which had the LAN-side failure (or BGP not activated).

  • Issue 57210: Even when a VMware SD-WAN Edge is working normally and is able to reach the internet, the LED in the Local UI's Overview page shows as "Red".

    The Edge's Local UI determines the Edge's connectivity by whether it can resolve a well known name via Google's DNS resolver (8.8.8.8). If it cannot do so for any reason, then it thinks it is offline and shows the LED as red.

    Workaround: There is no workaround for this issue, except to ensure that DNS traffic to 8.8.8.8 can reach the destination and be resolved successfully.

  • Issue 61543: If more than one 1:1 NAT rule is configured on different interfaces with the same Inside IP, the inbound traffic can be received on one interface and the outbound packets of the same flow can be routed via different interface.

    For the NAT flows from Outside to Inside, the 1:1 NAT rules will be matched against the Outside IP and the interface where the packets are received. For the outbound packets of the same flow, the VMWare SD-WAN Edge will try to match the NAT rules again comparing the Inside IP and the outbound traffic can be routed via the interface configured in the first matching rule with "Outbound Traffic" configured.

    Workaround: There is no workaround for this issue outside of ensuring no more than one 1:1 NAT rule is configured with a particular Inside IP address.

  • Issue 65560: Traffic from a customer to PE (Provider Edge) device fails.

    BGP neighborship between a Partner Gateway and Provider Edge does not get established when tag-type is selected as "none" on the handoff configuration. This is because ctag, stag values get picked from /etc/config/gatewayd instead of the handoff configuration on the Orchestrator when tag-type is "none".

    Workaround: Update the ctag, stag values to 0 each under vrf_vlan->tag_info in /etc/config/ gatewayd. Do a vc_procmon restart.

  • Issue 67879: A Cloud Security Service (CSS) tunnel is deleted after a user changes a WAN Overlay setting from auto-detect to user-defined on a WAN interface setting.

    After saving the changes, the CSS tunnels do not come back up until the customer takes down and then puts back up the tunnel. Changing the WAN configuration will bring down the CSS tunnel and parse the CSS setup again. However, in some corner cases, the nvs_config>num_gre_links is 0 and the CSS tunnel fails to come up.

    Workaround: Deactivate the CSS setup, and then reactivate it and this will bring the CSS tunnel up.

  • Issue 68057: DHCPv6 release packet is not sent from the VMware SD-WAN Edge on the changing of a WAN interface address mode from DHCP stateful to static IPv6 address and the lease remains active till reaching its valid time.

    The DHCPv6 client possesses a lease which it does not release when the configuration change is done. The lease remains valid till its lifetime expires in the DHCPv6 server and is deleted.

    Workaround: There is no way of remediating this issue as the lease would remain active till valid lifetime.

  • Issue 68851: If a VMware SD-WAN Edge and VMware SD-WAN Gateway each have the same TCP syslog server configured, the TCP connection is not established from the Edge to the syslog server.

    If the Edge and Gateway each have the same TCP server and if the syslog packets from the Edge are routed via the Gateway, the syslog server sends a TCP reset to the Edge.

    Workaround: Send the syslog packets direct from the Edge instead of routing via a Gateway or configure a different syslog server for the Edge and Gateway.

  • Issue 81852: For a VMware SD-WAN Edge that is using a Zscaler type Cloud Security Service (CSS) which uses GRE tunnels that has turned on L7 Health Check, when that Edge is upgraded to Release 5.0.0, in some instances the customer may observe L7 Health Check errors.

    This is typically seen during software upgrade or during startup time. When L7 Health check for a CSS using GRE tunnels is turned on, error messages related to socket getaddress error may be seen. The observed error is intermittently seen, and not consistent. Because of this, L7 Health Check probe messages are not sent out.

    Workaround: Without the fix, to remediate the issue, a user needs to turn off and then turn back on the L7 Health Check configuration, and this feature would then work as expected.

  • Issue 82184: On a VMware SD-WAN Edge which is running Edge Release 5.0.0, when a traceroute or traceroute6 is run to the Edge's br-network IPv4/IPv6 address, the traceroute will not properly terminate when a UDP probe used.

    Traceroute or traceroute6 to the Edge's br-network IPv4/IPv6 address will not properly when Default Mode (in other words, UDP probe) is used.

    Workaround: Use -I option in traceroute and traceroute6 to use ICMP probe and then traceroute to br-network IPv4/IPv6 address will work as expected.

  • Issue 85402: For a customer enterprise using BGP with Partner Gateways configured, a user may obseve that some BGP neighborships are down and this causes customer traffic issues.

    If a customer has maximum-prefix configured on a router which has BGP peering with the Edge and Gateway, the BGP session may be dropped by the router.

    For example, if the router has BGP configured to only receive max 'n' number of prefixes, but the Edge and Gateway have more than 'n' number of prefixes to be advertised in the absence of any filters. Now if the BGP filter configuration is changed on the Orchestrator, even if the total number of prefixes allowed in the outbound direction is less than 'n', the issue will be encountered where more than 'n' prefixes are sent to the peer before any filters are applied. This causes the router to tear down the session.

    Workaround: If BGP goes down due to this issue (Maximum Number of Prefixes Reached), flap BGP on the peer using CLI (For FRR/Cisco, "neighbor x shut" followed by "no neighbor x shut"), and the BGP will produce only the desired number of prefixes advertised to the peer.

  • Issue 92421: When a public and private overlay are configured on the same Edge interface with different custom VLAN tags, there is a chance that the underlay routed traffic may get dropped.

    When a public and private overlay are configured on the same interface with different custom VLAN tags, the Edge may learn the ARP entries with the wrong VLAN tags, resulting in the traffic being dropped.

    Workaround: Avoid using this configuration. This issue is fixed on Release 5.4.0 and later.

  • Issue 98136: For customer enterprises using a Hub/Spoke topology where Dynamic Branch To Branch VPN is configured, client users behind a SD-WAN Spoke Edge may observe that some traffic has unexpected latency resulting from the traffic using a sub-optimal path.

    Spoke Edge traffic that experiences this issue uses a route that was initially a non-uplink route for a Hub Edge not included in the Profile the Spoke Edge was using. A Dynamic Branch-to-Branch VPN tunnel can be formed from the Spoke Edge to the Hub Edge because of traffic being sent towards some other unrelated prefix and in this instance the non-uplink route is installed in the Spoke Edge.

    As a result of this non-uplink route, all traffic towards this prefix starts going through the

    Hub Edge and the non-uplink route becomes uplink (community change to uplink community) but the non-uplink route installed previously is not revoked and the traffic takes the Hub Edge path as long as the Dynamic Branch-to-Branch VPN tunnel remains up.

    Workaround: Wait for the Dynamic Branch-to-Branch VPN tunnel to tear down, after which the uplink route will not be installed in the Spoke Edge when a new Dynamic Branch-to-Branch VPN tunnel is formed towards the Hub Edge.

  • Issue 110561: Dynamic tunnels may not come up between the same set of VMware SD-WAN Edges with bidirectional traffic when traffic stops and then restarts.

    Issue is observed in a test environment where there are 6000 dynamic tunnels with high bidirectional traffic being sent between the Edges. Even in lower scale testing at 1000 dynamic tunnels, not all the tunnels come up.

    Workaround: There is no workaround for this issue.

  • Issue 111085: When a VMware SD-WAN Edge's WAN link is configured with an IP address in the same network as the Edge's loopback IP, the Edge uses the MAC address of the WAN interface while responding to an ARP request for the Edge's loopback IP address.

    This can cause ARP spoof and results in the Management IP being deprecated and network disruptions as a result.

    Workaround: There is no workaround for this issue.

  • Issue 113877: For customers who configure BGP/GRE LAN, those using a TGW GRE will experience BGP flaps and traffic interruption on the TGW secondary tunnel in all segments when the BGP configuration for TGW GRE is modified on the Global segment.

    When customer changes a BGP configuration of TGW GRE on the global segment then the secondary tunnel in the global and other segments flap, leading to a BGP connection reset and reconvergence and traffic interruption. The BGP connection will form again, and traffic will restore.

    Workaround: There is no workaround for this issue.

  • Issue 117876: In a customer site using a High Availability topology, if a user activates or deactivates the Enhanced Firewall Services, a VMware SD-WAN HA Edge may experience multiple restarts.

    When Enhanced Firewall Services is activated or deactivated, only the Active Edge's Device Settings configuration is synchronized immediately with the Standby Edge, with the remainder of the configuration synchronization is only in response to a Standby Edge heartbeat. When the Active Edge is restarted to apply the latest configuration prior to receiving a heartbeat from the Standby Edge it will result in a configuration mismatch between the two HA Edges and they will undergo multiple restarts to complete the configuration synchronization.

    Workaround: The only workaround is to turn on or off Enhanced Firewall Services during a maintenance window for HA Edges.

  • Issue 118704: A user may observe abnormally high latency values for paths measured between SD-WAN Edges and SD-WAN Gateways even though actual Edge-to-Gateway packet latency is much lower.

    A race condition has been identified with clock synchronization resulting in latency values measured incorrectly. This issue is cosmetic and there is no performance impact to customer traffic but it does negatively impact a customer's ability to properly monitor Edge links and paths.

    Workaround:Clock synchronization can be reset by restarting the Edge Service. This can be done using the Orchestrator UI by navigating to the Diagnostics > Remote Actions page, and then checking the affected Edge and selecting the Restart Service option.

  • Issue 118710: The IP address and mask are not allocated to the VLAN interfaces that are created on the WLAN interfaces.

    The Edge kernel fails to assign the VLAN interfaces onto the WLAN interfaces, which prevents the Edge from applying the configuration to these VLAN interfaces. Additionally, the assignment of network interface (netif) corresponding to vc-ifaces is also omitted, rendering these interfaces unusable since there are no IP addresses assigned to them.

    Workaround: Assign a physical interface (in other words, an Edge port like GE1, GE2 or the like) to the VLANs.

  • Issue 125274: When a customer runs an SNMP walk, the loopback interface of the VMware SD-WAN Edge is not discovered.

    The Edge loopback interface is a unique interface category that the Edge does not classify as either WAN or LAN. As a result, the loopback interface is not in the allow list of interfaces to process for the snmp-request.

    Workaround: There is no workaround for this issue. The loopback interface status would have to be individually monitored through the Orchestrator UI.

  • Issue 130674: IPv6 remote routes may be missing if the only physical interface configured with IPv6 is disconnected during Edge boot up and only later reconnected.

    An Edge will not install any IPv6 remote routes when the only physical interface configured with IPv6 is not connected to the Edge while it is rebooting and then only later connected.

    Workaround: Enable IPv6 loopback, or use a switched interface assigned to a IPv6 VLAN.

  • Issue 130885: An OSPFv3 route tag may not be updated for an IPv6 external route.

     In some corner cases, OSPFv3 does not consider the tag updated by the neighbor for an external route if the update is received within very short interval.

    Workaround: Withdraw and re-advertise the external route from the OSPFv3 neighbor.

  • Issue 132492: For a customer who has one or more Non SD-WAN Destinations via Edge configured and uses BGP, when no traffic is passing through the IPsec tunnels, the customer may observe that the tunnels are torn down and BGP routes flap.

    This issue is only seen when there is no traffic on the path from the NSD via Edge to the peer. The issue stems from a premature IKE Phase 1 rekey on the Edge and the peer sends multiple Dead Peer Detection (DPD) packets with an old cookie that the Edge does not acknowledge. This results in the peer side deleting both Phase 1 and Phase 2 IKE and tearing down the tunnels which also causes BGP flaps.

    Workaround: A user could set up a LAN side client to send a continuous ping to the NSD via Edge peer to prevent the scenario from arising.

  • Issue 133678: If a VMware SD-WAN Edge is configured for IPv4/IPv6 dual stack, the Edge may lose connectivity to the Orchestrator if the IPv4 link is down.

    This issue can occur if the Edge was activated with only an IPv4 link, and an IPv6 link is added only later to the device. At the time the Edge is activated, only the IPv4 Orchestrator address is written to the Edge that manages Orchestrator connectivity. Adding an IPv6 link later does not add the IPv6 address to the file and so if the IPv4 link is removed, the Edge loses connectivity to the Orchestrator.

    Workaround: An Edge activated with only an IPv4 link would need to keep at least one IPv4 WAN link connected even though the Edge is dual stack.

  • Issue 135827: For a customer site deployed with a High Availability topology, the customer may observe multiple HA failovers due to the site experiencing an active-active (split brain) condition.

    User would observe a HA_SPLIT_BRAIN_DETECTED on the Events page. The HA Standby Edge may miss the HA heartbeat from the Active Edge and promotes itself to an Active state. When the HA heartbeat is resumed it will report the HA_SPLIT_BRAIN_DETECTED event to the Orchestrator and the Standby Edge will restart to tie-break the HA split brain. This issue is observed where the enterprise uses Edge Network Intelligence with Analytics turned on and runs aggressive route timers.

    Workaround: To mitigate the risk of an active-active panic, configure the HA failover time to a higher value.

  • Issue 135938: For an Edge configured with a routed LAN interface and a secondary IP address configured on the routed interface, traffic sent to the secondary IP address connected interface is NAT'd with the parent interface's IP address.

    Whether the user checks the NAT Direct Traffic option or not has not impact, as the traffic is sent out based on the NAT direct configuration of the parent interface.

    Workaround: There is no workaround beyond ensuring that the secondary IP address is configured with the expectation that the NAT Direct Traffic option is only applied at the parent level.

  • Issue 138023: For a customer using a Partner Gateway (PG), a PG-BGP session does not come up when the BGP local IP address and PG Handoff local IP address are from the same subnet.

    SD-WAN treats this scenario as two interfaces on a router from the same subnet, which is not supported and can lead to ARP related issues.

    Workaround: Change the configuration to avoid the above scenario.

  • Issue 138464: On a customer enterprise site using a High Availability topology, a user may observe high memory utilization of the Standby Edge or even see an Event where the Standby Edge restarted due to high memory usage.

    With this issue, Standby Edge memory utilization increases rapidly when the HA Edges handle a higher number of concurrent connections per second. Once the memory utilization reaches 60% and is sustained there for more than 90 seconds then the Standby Edge's service defensively restarts to recover memory. In an Enhanced High Availaiblity topology this can cause customer traffic disruption for the part that the Standby Edge is handling through its WAN link(s).

    Workaround: On the Monitor > Events page for that HA Edge, monitor the memory utilization threshold limit warning Event or monitor memory utilization on the Monitor > HA Standby page and reduce the number of concurrent connections per second to maintain the memory utilization to less then 60%.

  • Issue 139855: For a customer enterprise where a High Availability topology is used and the Edges are virtual (not hardware Edges), if a user changes any Edge device setting, the Edge may delete the default route.

    This issue is limited to sites where the virtual HA Edges use a unique MAC Address on the LAN interface and have routing configured on the LAN interface. In that scenario the default route via a route interface (WAN overlay) and LAN interface may be removed after any changes on the Configure > Edge > Device page, resulting in customer traffic disruption.

    Workaround: Perform a network service restart to repopulate the default routes.

  • Issue 140194: For a customer enterprise site deployed with an Enhanced High Availability topology where a PPPoE link is used on the Standby Edge interface, an SNMPWalk does not work properly for this site.

    SNMPWalk output is incomplete for interface related MIBs when there is a PPPoE interface on the Standby Edge in Enhanced HA.

    Workaround: None.

  • Issue 140785: An SD-WAN Edge configured with IPv4 and IPv6 loop back interfaces and their advertise flags enabled may experience a Dataplane Service Failure and restart to recover.

    Packet fragmentation from packets 1350 bytes and greater is triggering an exception with the Edge service if configured as above and causing a service failure.

    Workaround: There is no workaround for this issue.

  • Issue 141008: On the Diagnostics > Remote Diagnostics page of the Orchestrator UI, Traceroute using an IP/Hostname destination does not for IPv6 addresses.

    The result from an IPv6 Traceroute shows the destination alone, and intermediate hops do not display. IPv4 addresses work as expected.

    Workaround: There is no workaround for this issue.

  • Issue 141113: An SNMP walk may get timed out and fail to complete when the Edge has an interface configured for a PPPoE link which is stuck in a down state.

    This issue only occurs if the PPPoE link on the interface is never up, if the interface is up and goes down for some reason the SNMP walk will successfully complete.

    Workaround: Ensure that any configured PPPoE link is capable of coming up, in other words ensure the peer PPPoE server is enabled.

  • Issue 141621: On a customer enterprise site deployed with a High Availability topology, in rare instances the Standby Edge may restart multiple times in response to an Active/Active (split-brain) state.

    When a LAN or WAN interface goes down on the Active Edge, the Standby Edge immediately takes the active role based on a higher LAN/WAN interface count. Due to a timing issue, this action could be taken before the Active Edge can restart to demote itself to a standby role. As a result the newly active Standby Edge reports an Active/Active Panic which triggers multiple Edge service restarts to clear the issue. A site deployed with an Enhanced HA topology would experience customer traffic disruption for those flows using the WAN links connected to the Standby Edge.

    Workaround: There is no workaround beyond ensuring the Active Edge has a high number of interfaces than the Standby Edge so that the issue could be avoided if an interface on the Active Edge went down.

  • Issue 142531: For a customer enterprise using a Hub/Spoke topology deployed in a multicast environment, the multicast receiver stops receiving traffic after 210 seconds.

    After 210 seconds, the Edge sends a prune message on the multicast that results in traffic loss. The multicast routes remain present and after 20-60 seconds, the multicast stream resumes working. The issue occurs on the Edge during the RPT to SPT switchover, if it is unable to send periodic (S,G) joins to the PIM upstream neighbor. This results in the upstream router sending a PIM prune towards the RP after the 3 min 30 seconds timer expiration, resulting in traffic loss.

    Workaround: There is no workaround for this issue.

  • Issue 143450: On a customer enterprise site configured with an Enhanced High Availability topology where Dynamic Branch to Branch VPN is also enabled, client users may observe extended traffic loss after an HA failover.

    The issue can be encountered if the Enhanced HA site also has a Business Policy rule configured which includes mandatory link steering. Combined with Dynamic Branch to Branch, this combination can result in a prolonged period of traffic disruption after an HA failover.

    Workaround: The customer can either remove the Business Policy rule with mandatory link steering entirely, or modify that rule to remove the mandatory link steering option.

  • Issue 143828: A customer may observe that an Edge has an unexpectedly high level of memory usage that may get sufficiently high to reach a critical level and trigger a defensive Edge service restart to recover the memory.

    One of the factors that contributes to this memory leak is extensive use of CLI commands on the Edge by either a Partner or Operator as part of troubleshooting or monitoring the Edge. These commands are accumulated and never cleared from the relevant Edge process. The use of Remote Diagnostics on the Orchestrator UI does not contribute to this issue.

    As with any Edge memory usage issue, entry level Edge models with lower RAM specifications (in other words, the Edge models 510, 610, 710, or 520) would be more likely to experience the issue, but it can happen on any Edge model with sufficiently high memory usage.

    Workaround: Extensive use of CLI commands on the Edge by either an Operator or Partner should be avoided, and if troubleshooting work is done, the customer should check the Edge's memory usage on the Edge > Monitor > System page.

  • Issue 145393: A customer enterprise site deployed with an Edge model 620, 640, or 680 where firewall logging is configured may observe that the Edge no longer stores new firewall or standard debugging logs.

    When this issue is encountered, a 6x0 Edge's eMMC storage experiences an excessive level of wear due to the high volume of writes and rewrites that can be triggered by enabling logging for firewall rules which are matched by a large number of new connections per second in a high traffic customer environment. This issue results in the Edge defensively moving the file partition which hosts logging to a read-only state, and no additional logs are stored.

    Workaround: If a customer has an Edge 620, 640, or 640 Edge model and is also using firewall logging, they should avoid enabling logging for firewall rules which can potentially match a large number of new connections in a high traffic environment. The excessive logging frequency that would result can cause undue wear on the Edge's storage and trigger this issue.

Orchestrator Known Issues

  • Issue 21342:

    When assigning Partner Gateways per-segment, the proper list of Gateway Assignments may not show under the Operator option "View" Gateways on the VMware SD-WAN Edge monitoring list.

  • Issue 25932:

    The VMware SD-WAN Orchestrator allows VMware SD-WAN Gateways to be removed from the Gateway Pool even when they are in use.

  • Issue 32335:

    The ‘End User Service Agreement’ (EUSA) page throws an error when a user is trying to accept the agreement.

    Workaround: Ensure no leading or trailing spaces are found in Enterprise Name.

  • Issue 32435:

    A VMware SD-WAN Edge override for a policy-based NAT configuration is permitted for tuples which are already configured at the profile level and vice versa.

  • Issue 32856:

    Though a business policy is configured to use the Hub cluster to backhaul internet traffic, the user can unselect the Hub cluster from a profile on a VMware SD-WAN Orchestrator that has been upgraded from Release 3.2.1 to Release 3.3.x.

  • Issue 35658:

    When a VMware SD-WAN Edge is moved from one profile to another which has a different CSS setting (e.g. IPsec in profile1 to GRE in profile2), the Edge level CSS settings will continue to use the previous CSS settings (e.g. IPsec versus GRE). 

    Workaround: At the Edge level, deactivate GRE, and then reactivate GRE to resolve the issue.

  • Issue 35667:

    When a VMware SD-WAN Edge is moved from one profile to another profile which has the same CSS setting but a different GRE CSS name (the same endpoints), some GRE tunnels will not show in monitoring.

    Workaround: At the Edge level, deactivate GRE and then reactivate GRE to resolve the issue.

  • Issue 36665:

    If the VMware SD-WAN Orchestrator cannot reach the internet, user interface pages that require accessing the Google Maps API may fail to load entirely.

  • Issue 32913:

    After activating High Availability, multicast details for the VMware SD-WAN Edge are not displayed on the Monitoring Page. A failover resolves the issue.

  • Issue 33026:

    The ‘End User Service Agreement’ (EUSA) page does not reload properly after deleting the agreement.

  • Issue 38056:

    The Edge-Licensing export.csv file not show region data.

  • Issue 38843:

    When pushing an application map, there is no Operator event, and the Edge event is of limited utility.

  • Issue 39633:

    The Super Gateway hyperlink does not work after a user assigns the Alternate Gateway as the Super Gateway.

  • Issue 39790:

    The VMware SD-WAN Orchestrator allows a user to configure a VMware SD-WAN Edge’s routed interface to have greater than the supported 32 subinterfaces, creating the risk that a user can configure 33 or more subinterfaces on an interface which would cause a Dataplane Service Failure for the Edge.

  • Issue 41691:

    User cannot change the 'Number of addresses' field although the DHCP pool is not exhausted on the Configure > Edge > Device page.

  • Issue 43276:

    User cannot change the Segment type when a VMware SD-WAN Edge or Profile has a Partner Gateway configured.

    Workaround: Temporarily remove the Partner Gateway configuration from the Profile or Edge so that the Segment can be changed between private and regular. Alternatively, the user can remove the Segment from the profile and make the change from there.

  • Issue 47713:

     If a Business Policy Rule is configured while Cloud VPN is toggled off, the NAT configuration must be reconfigured upon turning on Cloud VPN.

  • Issue 47820:

    If a VLAN is configured with DHCP toggled off at the Profile level, while also having an Edge Override for this VLAN on that Edge with DHCP activated, and there is an entry for the DNS server field set to none (no IP configured), the user will be unable to make any changed on the Configure > Edge > Device page and will get an error message of ‘invalid IP address []’ that does not explain or point to the actual problem.

  • Issue 48085: The VMware SD-WAN Orchestrator allows a user to delete a VLAN which is associated with an interface.

    When encountering this issue, the user would see an error message similar to "VLAN ID [xx] cannot be removed, in use by edge [b1-edge1 (GEx-disabled]".

  • Issue 51722: On the VMware SASE Orchestrator, the time range selector is no greater than two weeks for any statistic in the Monitor > Edge tabs.

    The time range selector does not show options greater than "Past 2 Weeks" in Monitor > Edge tabs even if the retention period for a set of statistics is much longer than 2 weeks. For example, flow and link statistics are retained for 365 days by default (which is configurable), while path statistics are retained only for 2 weeks by default (also configurable). This issue is making all monitor tabs conform to the lowest retained type of statistic versus allowing a user to select a time period that is consistent with the retention period for that statistic.

    Workaround: A user may use the "Custom" option in the time range selector to see data for more than 2 weeks.

  • Issue 60522: On the VMware SD-WAN Orchestrator UI, the user observes a large number of error messages when they try to remove a segment.

    The issue can be observed when adding a segment to a profile and the associating the segment with multiple VMware SD-WAN Edges. When the user attempts to remove the added segment from the profile, they will see a large number of error messages.

    Workaround: There is no workaround for this issue.

  • Issue 82095: User can configure invalid device settings for Edge VLANs that will result in significant connectivity issues for the Edge.

    The Orchestrator is not attempting to validate device configurations. In particular, a VLAN configuration for a switched port with an empty table. Some configurations can be so full of errors that the Edge's management process will fail.

    Workaround: Review all VLAN Device settings and ensure they are valid as the Orchestrator is not checking.

  • Issue 82680: For customer using MT-GRE Tunnel Automation, when a user turns off the Cloud-to-Cloud Interconnect (CCI) flag on a VMware SD-WAN Gateway which is configured to use CCI, the Zscaler MT-GRE entries may not get deleted from the Zscaler portal consistently.

    After a CCI site has been deleted from the Gateway, the entries for this site should also be removed. This issue has only been seen during test automation and has not been reproduced manually, but remains a risk.

    Workaround: Manually delete the resource from Zscaler before retrying.

  • Issue 82681: For customer using MT-GRE Tunnel Automation, when a user turns off the Cloud-to-Cloud Interconnect (CCI) flag on a VMware SD-WAN Gateway which is configured to use CCI, and the user deactivates the CCI flag from a VMware SD-WAN Edge with CCI configured which is using a Zscaler Cloud Security Service, the Zscaler MT-GRE entries may not get deleted from the Edge or from the Zscaler portal.

    After a CCI site has been deleted from the Gateway, the entries for this site should also be removed. This issue has only been seen during test automation and has not been reproduced manually, but remains a risk.

    Workaround: Manually delete the resource from Zscaler before retrying.

  • Issue 103769: An Operator may observe that a VMware SASE Orchestrator in a large scale deployment is experiencing performance issues which include 100% disk utilization and the Orchestrator no longer accumulating logs.

    This issue arises a change in logging behavior for the 5.1.0 Orchestrator that may result in the folders that store logs becoming full and also causing the Orchestrator CPU to reach 100% utilization. This issue arises a change in logging behavior for the 5.1.0 Orchestrator that may result in the folders that store logs becoming full and also causing the Orchestrator CPU to reach 100% utilization.

    Workaround: A Superuser Operator needs to log into the Orchestrator and clean up the pending logs.

  • Issue 117699: An Operator attempting to upgrade a 4.2.x VMware SD-WAN Orchestrator to become a Release 5.2.0 SASE Orchestrator may observe that the upgrade fails.

    The upgrade does not succeed, effectively stuck at the "Waiting for the CWS service up...". This issue is limited to 4.2.x Orchestrators.

    Workaround: The workaround for this issue is to upgrade the 4.2.x Orchestrator to 4.5.1 first, and then to Release 5.2.0.0.

  • Issue 125082: If a user configures a VMware SD-WAN Edge with an overridden DNS Server IP address on a VLAN, and then changes an interface setting for the Profile that Edge is using, the DNS Server IP address is no longer present for the Edge VLAN.

    The New UI does not send the override flag inside of the DHCP section and this causes any Profile changes to trigger an override of the DHCP section.

    Workaround: There is no workaround for this issue.

  • Issue 125504: If a static route is configured with next hop as a VLAN with IPv4/IPv6 address at the Profile level and then overridden at the Edge level and add an IPv4/IPv6 address to the VLAN, the static route is not marked as N/A and the VMware SASE Orchestrator asks for the interface in a dropdown menu.

    The expected behavior is when a static route is configured with a next hop as a VLAN with IPv4/IPv6 address, the Orchestrator does not ask for the interface and the route is marked as N/A.

    Workaround: There is no workaround for this issue.

  • Issue 125663: A user can configure the same IPv4/IPv6 IP address for multiple Edge interfaces.

    The VMware SASE Orchestrator is allowing a user to configure the same IP on multiple WAN, LAN, or Sub Interfaces.

    Workaround: There is no workaround for this issue beyond ensuring you are not configuring the same IP Address for multiple interfaces.

  • Issue 126421: For Partners using a Partner Gateway, when configuring the Hand Off Details, the "Use for Private Tunnels" option is always checked no matter what a user does.

    This is not a cosmetic issue as the Orchestrator will apply the Use for Private Tunnels configuration to the Partner Gateway handoff and can impact customer traffic using the Partner Gateway.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 126425: When looking at Configure > Device > Routing & NAT page at the Profile level, the OSPF On/Off toggle button is missing.

    The OSPF On/Off toggle button was not migrated to the New UI at the Profile level and only shows at the Edge level.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 126465: The VMware SASE Orchestrator UI is not applying changes a user makes to create an Edge Cluster.

    If a user goes to the Configure > Edge > High Availability section of the UI and turns on HA with a Cluster type and creates a Hub Cluster with name xxxx, and saves changes, the user would observe that post-save the Cluster option is not selected under HA section and the created Hub Cluster with name xxxx is not present.

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 126695: If a user is configuring webhooks for Alerts, when they click on the "Configure Payload Template" button the menu is not displayed.

    This issue occurs when configuring webhooks on the SD-WAN > Settings > Alerts > Webhooks page of the UI. When looking at the browser console, a user would also observe the message: ERROR TypeError: Cannot read properties of undefined (reading 'invalid').

    Workaround: There is no workaround for this issue on an Orchestrator with only a New User Interface.

  • Issue 127152: Users cannot save modified Interfaces with OSPF configurations on the VMware SASE Orchestrator UI.

    At the Profile level, when configuring either OSPFv2/OSPFv3, the Edit Interface dialog becomes invalid after changing any OSPF data.

    Workaround: On an Orchestrator without a fix for this issue, a user would need to activate MD5 Authentication and change the Key ID to any number from 1 to 255, and then deactivate MD5 Authentication.

  • Issue 130115: For a VMware SASE Orchestrator configured with a Disaster Recover (DR) topology, the Active and Standby Orchestrator's DR pages show different details under the History section.

    The user sees additional rows for a failing DR state on the Active Orchestrator compared to the Standby Rows under the History section and these rows are not sorted by time on the Active Orchestrator.

    Workaround: No workaround for this issue.

  • Issue 131846: On the Global Settings > Customer Configuration > Partner Hand off > Hand Off Interface page of the Orchestrator UI, when a user clicks the Add button to add a static route, the UI returns an error message.

    When the user clicks the Add button in the static routes section, a new empty row is added to the table, but an error message reading "Cannot save changes. There is one or more errors in your configuration" is displayed. This is caused by a missing validation for the empty static route configuration on the Partner Handoff section of the Global Settings screen, and this issue only affects rows where no information is configured.

    Workaround: There are two workarounds for this problem:

    • Fill in all the mandatory fields. By doing so, the error message ("Cannot save changes...") will be automatically resolved.

    • Remove any newly added empty rows from the section. After removing these empty rows, the error message will be automatically resolved.

  • Issue 131997: An ICMP probe may be erroneously marked as down when it is configured for one segment but not for another.

    The Orchestrator fails to send the NULL ICMP probe configuration when it is not configured in a segment. As a result, the configuration of a different segment is reused and this causes the probes to fail.

    Workaround: On an Orchestrator without a fix for this issue, configure a dummy ICMP probe for the other segments.

  • Issue 133092: For a customer enterprise that deploys one or more Non SD-WAN Destinations via Gateway with redundant Gateways where L7 Health check is enabled, a user may observe the wrong service status for this NSD when looking at Monitor > Network Services.There is no functional impact for the NSD as this issue is cosmetic. The issue can occur if each Gateway (Primary and Secondary) has a different L7 Health status and there can be a potentially false status when looking at the UI. For example, if the Primary VPN tunnel is down, the user may see the UI report that the NSD is completely down, when in fact the Secondary VPN tunnel is now passing customer traffic to the peer datacenter.

    There is no functional impact for the NSD as this issue is cosmetic. The issue can occur if each Gateway (Primary and Secondary) has a different L7 Health status and there can be a potentially false status when looking at the UI. For example, if the Primary VPN tunnel is down, the user may see the UI report that the NSD is completely down, when in fact the Secondary VPN tunnel is now passing customer traffic to the peer datacenter.

    Workaround: There is no workaround to show the correct status in the UI, but an Operator or Partner can see the correct service status of each VPN in the Gateway if they log into the Gateway via SSH and run debug.py --ike.

  • Issue 139854: On the Configure > Edge > Device > Interfaces page of the Orchestrator UI, when a user configures a route interface for IPv6, the UI does not treat the Gateway field as mandatory.

    The user can leave the Gateway field blank and save the configuration, even though this IPv6 interface configuration will not work without a Gateway.

    Workaround: Ensure the Gateway field is configured.

  • Issue 142316: Updating a VLAN or Edge routed interface setting of a profile used by a large number of Edges (>7500) takes a long time to complete and may even time out in some instances.

    If the attempt times out on the Orchestrator UI, if the user reloads the Configure > Profile > Device page after more than 60 seconds, the changes are expected to show as applied and there is no need to do the same changes again.

    Workaround: There is no workaround for this issue.

  • Issue 142456: On the Monitor > Firewall Logs page of the Orchestrator UI, a user may not be able to sort data on this page.

    A user should be able to click on the column header to sort between the various data included in a firewall log, but cannot.

    Workaround: There is no workaround for this issue.

  • Issue 142608: On the Profile/Edge > Configure > Device > Routing & NAT > Multicast page of the Orchestrator UI, when a user enables Multicast with a RP configured they may find that they cannot configure a 'Join Prune Send Interval' value of less than 60 seconds.

    The Join Prune Send Interval setting is found under the Advanced Settings - PIM Timers and the expected behavior is for this value to be configurable in a range from 5-600 seconds.

    Workaround: There is no workaround for this issue.

  • Issue 142666: On the Configure > Edge page, the Edge Details dialogue has a small font size which makes the text difficult to read.

    On the Configure > Edge page, the Edge's details are shown next to the Edge name at the top when the user clicks the drop down symbol and the text automatically adjusts the font size to something that fits that box, something very small and hard to read.

    Workaround: A user can increase the zoom percentage on their browser to better see the smaller text.

  • Issue 142672: On the Edges > Monitor > Sources page of the Orchestrator UI, a user cannot change the Host Name for an entry.

    The user can click the Change Hostname option, but on the dialog box, if they enter a different host name and try to Save Changes, the Orchestrator throws an error and the changes are not saved.

    Workaround: There is no workaround for this issue.

  • Issue 142785: For a customer enterprise site where the Edge has a WAN link configured for dual-stack mode (IPv4 and IPv6) and IP Preference is IPv4, if a Cloud Security Service (CSS) is configured for this Edge but the user does not configure FQDN, the user can change the preference mode to IPv6.

    Changing the IP Preference mode when a CSS or a Non SD-WAN Destination (NSD) via Edge is deployed should be blocked as both types use the IPv4 WAN Overlay and changing to IPv6 will cause traffic to drop. The expected Orchestrator UI behavior is for the IP Preference option to be grayed out and not accessible while a CSS or NSD via Edge is associated with that interface.

    Workaround: There is no workaround beyond ensuring IPv4 is used and not changed as the IP Preference.

  • Issue 146898: For a customer deploying a Non SD-WAN Destination (NSD) via Gateway where BGP is configured on the NSD, if the customer migrates the NSD to a different Gateway using the Self-Service Gateway Migration feature on the Orchestrator, the BGP configurations are not migrated and all BGP sessions are dropped post-migration.

    In this scenario, the existing Gateway assigned to the NSD is in a quiesced state and requires migration to another Gateway. The customer then navigates to Service Settings > Gateway Migration on the Orchestrator and initiates the Gateway Migration process to move their NSD to another Gateway. Post-migration, the BGP Local ASN & Router ID information is not populated on the new Gateway and results in NSD BGP sessions not coming up with all routes being lost and traffic using those routes is disrupted until the user manually recreates all BGP settings.

    This is a Day 1 issue and while the Gateway Migration feature accounts for many critical NSD settings, the NSD's BGP settings are not accounted for, and their loss post-migration is an expected behavior.

    Workaround: The migration of a Gateway should be done in a maintenance window only. Prior to the migration, the user should document all BGP settings and be prepared to manually reconfigure these settings post-migration to minimize impact to customer users.

  • Issue 148274: A user may observe gaps in charts on the Monitor > Edge section of the Orchestrator UI.

    When an Orchestrator's database receives a high volume of simultaneous queries, it does not properly queue requests beyond the maximum concurrent query limit.

    As a consequence, queries are rejected, leading to client failures. Clients without proper retry mechanisms or error handling capabilities fail to correctly handle this failure, resulting in gaps in visualizations, like Monitor > Edge charts (QoE, Links, Flows, and so forth).

    Workaround: There is no workaround for this issue.

check-circle-line exclamation-circle-line close-line
Scroll to top icon