Updated March 31st, 2023 VMware SD-WAN™ Edge Version R440-20210701-GA-61583 Check regularly for additions and updates to these release notes. |
What is in the Release Notes
The release notes cover the following topics:- Recommended Use
- Compatibility
- Important Notes
- New Features
- Orchestrator API Changes
- Revision History
- Resolved Issues
- Known Issues
Recommended Use
This release is recommended for all customers who require the features and functionality first made available in Release 4.4.0.
Compatibility
Using VMware SASE requires Release 4.4.0 or above on all components.
Note: Release 4.4.0 can only be used in new deployments. Customers with existing networks deployments using Release 4.3.x or earlier cannot upgrade to this release.
Important Notes
VMware SASE™
Beginning with Release 4.4.0, VMware offers a comprehensive cloud-delivered Secure Access Service Edge (“SASE”) platform that brings together industry-leading solutions from VMware. As part of this transformation we are changing the release naming from VMware SD-WAN Release to VMware SASE Release to reflect the broader set of SASE services delivered by our solution.
VMware Secure Access Service Edge (SASE) is a cloud-hosted solution that enables any user on any device and at any location a reliable, efficient, and optimal access to any cloud application with security enforcement. As an extensible solution VMware SASE delivers networking and security services using a global network of VMware SASE PoPs (Points of Presence). VMware SASE uses a single management pane of glass to configure, manage and monitor these services. The solution offers actionable insights into end-user and IoT device operational experience.
The solution is available as a managed service or DIY, leveraging a global network of cloud service nodes.
VMware SASE is comprised of four solutions:
-
VMware SD-WAN™ is the industry-leading software-defined WAN, delivering high-performance, reliable branch access for easy deployment, and management across cloud or hybrid environments.
-
VMware Cloud Web Security™ is a cloud hosted service that protects users and infrastructure accessing SaaS and Internet applications from threats, offering visibility, control, and compliance.
-
VMware Secure Access™ is a remote access solution that is based on a Zero Trust Network Access (ZTNA) framework. The cloud-hosted solution offers multiple benefits over the traditional VPN solutions enabling users a consistent, optimal and secure cloud application access.
-
VMware Edge Network Intelligence™ is a vendor agnostic AIOps solution focused on the enterprise edge that ensures end user and IoT client operational assurance as traffic from these devices traverse wireless and wired LAN, SD-WAN, and SASE.
Notes on Rollout
- VMware SASE Points of Presence (PoPs) are being deployed with Release 4.4.0. For any questions regarding your nearest PoP, please contact your VMware sales representative.
- Partners who plan to migrate existing SD-WAN customers to take advantage of VMware SASE services should contact their VMware sales representative.
BGPv4 Filter Configuration Delimiter Change for AS-PATH Prepending
Through Release 3.x, the VMware SD-WAN BGPv4 filter configuration for AS-PATH prepending supported both comma and space based delimiters. However, beginning in Release 4.0.0 and forward, VMware SD-WAN will only support a space based delimiter in an AS-Path prepending configuration.
Customers deploying a network with Release 4.4.0 whose only previous experience was on Release 3.x need to ensure their AS-PATH prepending configurations use only spaces for delimiters to avoid incorrect BGP best route selection.
New Features
VMware Cloud Web Security
VMware Cloud Web Security is a cloud hosted service that protects users and infrastructure accessing SaaS and Internet applications from a changing landscape of internal and external threats, offers visibility and control, and ensures compliance.
Cloud Web Security is delivered through a global network of VMware SASE Points-of-Presence (PoPs) to ensure that users located anywhere and connecting over any device have a secure, consistent, and optimal access to applications. Cloud Web Security simplifies management of security services and helps IT tighten the security posture while balancing user productivity.
Cloud Web Security provides IT teams the visibility and control they need to maintain a strong security posture while adhering to compliance needs with the following advantages:
- Agile Security Posture. As a cloud hosted service any threat detected anywhere by Cloud Web Security is immediately blocked for all customers taking advantage of the cloud-native properties.
- Secure Seamless Access for Anywhere Workforce. Leveraging a global network of VMware SASE PoPs, Cloud Web Security delivers secure and optimal access to users for Internet and SaaS applications
- Simplified Operations. Cloud Web Security uses a centralized management pane using the VMware SD-WAN Orchestrator for network services and security services simplifying deployment and operations of a distributed workplace.
- Reducing Operational Cost. Cloud Web Security offers cost savings from managing the life cycle and refresh cycle of physical or virtual appliances deployed on-premises.
What's Offered in Release 4.4.0
VMware Cloud Web Security empowers IT administrators to reduce the attack surface with URL filtering and content filtering. URL filtering helps control the web sites employees can access. Content filtering helps limit the type of content users can access. Content is inspected for malware attacks from known viruses using up-to-date threat intelligence. The solution protects against zero-day malware with sandbox support where the content is inspected in a contained environment. Cloud Web Security includes support for SSL proxy with decryption to inspect the vast majority of web applications that are SSL encrypted today. The solution also offers rich traffic and threat visualization via security dashboards and web logs.
VMware Cloud Web Security will be available to customers in 2 Editions:
1. Standard Edition
2. Advanced Edition
Release 4.4.0 will be launched with the Cloud Web Security Standard Edition package. In addition, we will make active the Advanced Sandbox feature at launch of the release. While the Advanced Sandbox feature is a part of Cloud Web Security Advanced Edition package, customers would be able to preview the capabilities for a limited time till we launch the Cloud Web Security Advanced Edition.
Here is a summary of the Cloud Web Security features that will be available in the Standard and Advanced Editions. The highlighted cells are the features that will be available at GA of Release 4.4.0.
Edition |
Standard |
Advanced |
URL Filtering |
X |
X |
SSL Inspection |
X |
X |
Content Filtering |
X |
X |
File Hash Check |
X |
X |
Anti-Virus |
X |
X |
Authentication |
X |
X |
Basic Sandbox |
X |
X |
Advanced Sandbox* |
X |
X |
SIEM Logging** |
X |
X |
Inline CASB |
Visibility |
Visibility and Control |
Inline DLP |
X |
|
Microsoft MCAS integration |
X |
X |
* Part of Cloud Web Security Advanced Edition package available for initial preview.
** SIEM Logging was initially listed as an available feature, however the API needed to implement SIEM Logging remains in development and thus this feature is effectively unavailable in Release 4.4.0. The required SIEM Logging API implementation will be available with a future release.
The Basic and Advanced Sandbox differ in the type of contents that can be inspected. The following table lists out the differences between the Basic and Advanced Sandbox features:
|
|
Basic Sandbox |
Advanced Sandbox |
Document Types |
Engineering Applications |
|
X |
Productivity Applications |
|
X |
|
Word Processors |
|
X |
|
Spreadsheets |
|
X |
|
Presentation Tools |
|
X |
|
File Types |
Scripts and Executables |
X |
X |
Archives and Compressed Packages |
X |
X |
|
Multimedia |
|
X |
|
Calendar |
|
X |
VMware Secure Access
The VMware Secure Access solution provides remote and mobile users a consistent, optimal and secure cloud application access through a network of worldwide managed service nodes. The solution is based on the Zero Trust Network Access (ZTNA) architecture that offers multiple benefits over the traditional VPN solution:
- VMware Secure Access is cloud-hosted, enabling enterprises to offload the costly deployment, maintenance, and scale of the remote access solution for improved IT efficiency.
- VMware Secure Access offers user- and application-centric access versus network-based access with VPN. Access is granted based on the user identity and end-device posture, with access to only the applications the user needs, significantly reduces the surface of attack.
- Location independent means having a consistent access policy regardless of where the user access from.
The solution leverages VMware’s global Points of Presence (PoPs) footprint and optimizes traffic handling capabilities for lower latency and better application performance, enabling customers to have a branch like experience to remote workers.
What's Offered in Release 4.4.0
VMware SASE Release 4.4.0 also allows remote users to connect to their applications via VMware Secure Access using BYOD or third-party managed devices, in addition to VMware Workspace ONE managed devices available earlier. In this release, the 700-user minimum and 5GB data limits have been lifted, and customers can now self-provision Secure Access without VMware support.
Orchestrator API Changes Since Release 4.3.x
Consumers of the Portal API (“APIv1”) can find a comprehensive changelog available for download on code.vmware.com.
Release 4.4.0 does not impact SD-WAN APIv2.
This release introduces two new Orchestrator APIs: the Cloud Web Security REST API and the Secure Access REST API. We expect to publish documentation for these APIs on developer.vmware.com as they are declared stable for customer and partner use.
Once both API's are published, the Release Notes will be revised and republished with hyperlinks to that documentation. Please periodically check back for updates to this section.
Document Revision History
June 10th, 2021. First Edition
June 22nd, 2021. Second Edition
- Added a new Edge Release Version, R440-20210617-GA to the Resolved Issues section, and added a fixed issue associated with that Release #59509.
July 2nd, 2021, Third Edition
- Added new Edge GA build R440-20210701-GA-61583 to the Resolved section. This Edge addresses new resolved issue #61583.
August 27th, 2021, Fourth Edition
- Added clear wording that 4.4.0 is for greenfield deployments only, which means the customer cannot upgrade to 4.4.0 from an older release, but must deploy as a new enterprise using 4.4.0.
September 16th, Fifth Edition
- Revised the Cloud Web Security features availability table to indicate that SIEM Logging is not available in Release 4.4.0. SIEM Logging was initially listed as an available feature, however the API needed to implement SIEM Logging remains in development and thus this feature is effectively unavailable in Release 4.4.0.
-
Added to Important Notes the Note: BGPv4 Filter Configuration Delimiter Change for AS-PATH Prepending.
October 25th, Sixth Edition
- Removed Open Issue #52104 as it was resolved in 4.2.x and Open Issue #52483 as it was closed prior to this release.
Resolved Issues
The resolved issues are grouped as follows.
Edge Resolved IssueResolved in Version R440-20210702-GA-61583
The below issue has been resolved since Edge version R440-20210617-GA.
- Fixed Issue 61583: If a customer enables High Availability for a site, the VMware SD-WAN Edge will go offline and the site goes down and all customer traffic is disrupted.
When HA is enabled, the Edge will at a minimum go offline for ~5 minutes with customer traffic disrupted during that period. The Edge may be able to roll back to the previous configuration and resume operation after that ~5 minute period, though that Edge would continue to operate as a standalone with HA not possible. However, if the Edge does not successfully roll back to the last configuration, then it will stay down until a local user performs a factory reset followed by an RMA Reactivation (with HA not enabled) to restore connectivity at that site.
For more information please consult the KB article Enabling High Availability on a VMware SD-WAN Edge using Release 4.3.0 GA or 4.4.0 GA may cause the Edge to go offline. (84396)
Resolved in Edge Release R440-20210617-GA
The below issue is resolved since Edge version R440-20210610-GA
- Fixed Issue 59509: VMware SD-WAN Edge does not correctly interpret the Edge License assigned to that Edge.
When connecting to a SASE PoP, the Edge may be using features inconsistent with the license type assigned to the Edge.
SD-WAN Edge/Gateway Resolved Issues
Resolved in Edge Release R440-20210610-GA, there is no Gateway Release.
There are no new issues resolved since Edge version R430-20210528-GA and Gateway version R430-20210605-GA.
There are no new issues resolved since Orchestrator version R430-20210527-GA.
Known Issues
Open Issues in Release 4.4.0
The known issues are grouped as follows.
- SD-WAN Edge/Gateway Known Issues
- Orchestrator Known Issues
- Cloud Web Security Issues
- Secure Access Issues
- Issue 14655:
Plugging or unplugging an SFP adapter may cause the device to stop responding on the Edge 540, Edge 840, and Edge 1000 and require a physical reboot.
Workaround: The Edge must be physically rebooted. This may be done either on the Orchestrator using Remote Actions > Reboot Edge, or by power-cycling the Edge.
- Issue 25504:
Static route costs greater than 255 may result in unpredictable route ordering.
Workaround: Use a route cost between 0 and 255
- Issue 25595:
A restart may be required for changes to static SLA on a WAN overlay to work properly.
Workaround: Restart Edge after adding and removing Static SLA from WAN overlay
- Issue 25742:
Underlay accounted traffic is capped at a maximum of the capacity towards the VMware SD-WAN Gateway, even if that is less than the capacity of a private WAN link which is not connected to the Gateway.
- Issue 25758:
USB WAN links may not update properly when switched from one USB port to another until the VMware SD-WAN Edge is rebooted.
Workaround: Reboot the Edge after moving USB WAN links from one port to another.
- Issue 25855:
A large configuration update on the Partner Gateway (e.g. 200 BGP-configured VRFs) may cause latency to increase for approximately 2-3 seconds for some traffic via the VMware SD-WAN Gateway.
Workaround: No workaround available.
- Issue 25921:
VMware SD-WAN Hub High Availability failover takes longer than expected (up to 15 seconds) when there are three thousand branch Edges connected to the Hub.
- Issue 25997:
The VMware SD-WAN Edge may require a reboot to properly pass traffic on a routed interface that has been converted to a switched port.
Workaround: Reboot the Edge after making the configuration change.
- Issue 26421:
The primary Partner Gateway for any branch site must also be assigned to a VMware SD-WAN Hub cluster for tunnels to the cluster to be established.
- Issue 28175:
Business Policy NAT fails when the NAT IP overlaps with the VMware SD-WAN Gateway interface IP.
- Issue 31210:
VRRP: ARP is not resolved in the LAN client for the VRRP virtual IP address when the VMware SD-WAN Edge is primary with a non-global CDE segment running on the LAN interface.
- Issue 32731:
Conditional default routes advertised via OSPF may not be withdrawn properly when the route is turned off. Re-enabling and disabling the route will retract it successfully.
- Issue 32960:
Interface “Autonegotiation” and “Speed” status might be displayed incorrectly on the Local Web UI for activated VMware SD-WAN Edges.
- Issue 32981:
Hard-coding speed and duplex on a DPDK-configured port may require a VMware SD-WAN Edge reboot for the configurations to take effect as it requires disabling DPDK.
- Issue 34254:
When a Zscaler CSS is created and the Global Segment has FQDN/PSK settings configured, these settings are copied to Non-Global Segments to form IPsec tunnels to a Zscaler CSS.
- Issue 35778:
When there are multiple user-defined WAN links on a single interface, only one of those WAN links can have a GRE tunnel to Zscaler.
Workaround: Use a different interface for each WAN link that needs to build GRE tunnels to Zscaler.
- Issue 35807:
A DPDK routed interface will be deactivated completely if the interface is first deactivated, and then re-activated from the VMware SD-WAN Orchestrator.
- Issue 36923:
Cluster name may not be updated properly in the NetFlow interface description for a VMware SD-WAN Edge which is connected to that Cluster as its Hub.
- Issue 38682:
A VMware SD-WAN Edge acting as a DHCP server on a DPDK-configured interface may not properly generate “New Client Device" events for all connected clients.
- Issue 38767:
When a WAN overlay that has GRE tunnels to Zscaler configured is changed from auto-detect to user-defined, stale tunnels may remain until the next restart.
Workaround: Restart the Edge to clear the stale tunnel.
- Issue 39134:
The System health statistic “CPU Percentage” may not be reported correctly on Monitor > Edge > System for the VMware SD-WAN Edge, and on Monitor > Gateways for the VMware SD-WAN Gateway.
Workaround: Users should use handoff queue drops for monitoring Edge capacity not CPU percentage.
- Issue 39608:
The output of the Remote Diagnostic “Ping Test” may display invalid content briefly before showing the correct results.
- Issue 39624:
Ping through a subinterface may fail when the parent interface is configured with PPPoE.
- Issue 39659:
On a site configured for Enhanced High Availability, with one WAN link on each VMware SD-WAN Edge, when the standby Edge has only PPPoE connected and the active has only non-PPPoE connected, a split brain state (active/active) may be possible if the HA cable fails.
- Issue 39753:
Disabling Dynamic Branch-to-Branch VPN may cause existing flows currently being sent using Dynamic Branch-to-Branch to stall.
- Issue 40096:
If an activated VMware SD-WAN Edge 840 is rebooted, there is a chance an SFP module plugged into the Edge will stop passing traffic even though the link lights and the VMware SD-WAN Orchestrator will show the port as 'UP'.
Workaround: Unplug the SFP module and then replug it back into the port.
- Issue 40421:
Traceroute is not showing the path when passing through a VMware SD-WAN Edge with an interface configured as a switched port.
- Issue 42278:
For a specific type of peer misconfiguration, the VMware SD-WAN Gateway may continuously send IKE init messages to a Non-SD-WAN peer. This issue does not disrupt user traffic to the Gateway; however, the Gateway logs will be filled with IKE errors and this may obscure useful log entries.
- Issue 42388:
On a VMware SD-WAN Edge 540, an SFP port is not detected after disabling and reenabling the interface from the VMware SD-WAN Orchestrator.
- Issue 42488:
On a VMware SD-WAN Edge that has a switched port with VRRP turned on, if the cable is disconnected and the Edge Service is restarted, the LAN connected routes are advertised.
Workaround: There is no workaround for this issue.
- Issue 42872:
Enabling Profile Isolation on a Hub profile where a Hub cluster is associated does not revoke the Hub routes from the routing information base (RIB).
- Issue 43373:
When the same BGP route is learnt from multiple VMware SD-WAN Edges, if this route is moved from preferred to eligible exit in the Overlay Flow Control, the Edge is not removed from the advertising list and continues to be advertised.
Workaround: Turn on Distributed Cost Calculation on the VMware SD-WAN Orchestrator.
- Issue 44526: For an enterprise where two different sites deploy their VMware SD-WAN Edges as Hubs while also using a high-availability topology, and each site uses the other Hub site as a Hub in its profile. If one of the Hub sites triggers an HA failover, it may take up to 30 minutes for both Hub Edges to reestablish tunnels with each other.
On an HA failover, both Hub Edges try to initiate a tunnel with each other at the same time and neither replies to the peer, the packet exchange between both Hubs occurs, but IKE never succeeds. This leads to a deadlock that has been observed to take up to 30 minutes to resolve on its own. The issue is intermittent and does not occur after every HA failover.
Workaround: To prevent this issue from occurring, the customer should configure only one of the two HA Hub sites to use the other Hub site as a Hub for itself. For example, where there are two HA Hub sites, Hub1 and Hub2, Hub1 could have Hub2 as a Hub for itself in its profile, but Hub2 must not use Hub1 as a Hub in its profile.
- Issue 44995:
OSPF routes are not revoked from VMware SD-WAN Gateways and VMware SD-WAN Spoke Edges when the routes are withdrawn from the Hub Cluster.
- Issue 45189:
With source LAN side NAT is configured, the traffic from a VMware SD-WAN Spoke Edge to a Hub Edge is allowed even without the static route configuration for the NAT subnet.
- Issue 45302:
In a VMware SD-WAN Hub Cluster, if one Hub loses connectivity for more than 5 minutes to all of the VMware SD-WAN Gateways common between itself and its assigned Spoke Edges, the Spokes may in rare conditions be unable to retain the hub routes after 5 minutes. The issue resolves itself when the Hub regains contact with the Gateways.
- Issue 46053:
BGP preference does not get auto-corrected for overlay routes when its neighbor is changed to an uplink neighbor.
Workaround: An Edge Service Restart will correct this issue.
- Issue 46137:
A VMware SD-WAN Edge running 3.4.x software does not initiate a tunnel with AES-GCM encryption even if the Edge is configured for GCM.
- Issue 46216:
On a Non SD-WAN Destinations via Gateway or Edge where the peer is an AWS instance, when the peer initiates Phase-2 re-key, the Phase-1 IKE is also deleted and forces a re-key. This means the tunnel is torn down and rebuilt, causing packet loss during the tunnel rebuild.
Workaround: To avoid tunnel destruction, configure the Non SD-WAN Destinations via Gateway/Edge or CSS IPsec rekey timer to less than 60 minutes. This prevents AWS from initiating the re-key.
- Issue 46391:
For a VMware SD-WAN Edge 3800, the SFP1 and SFP2 interfaces each have issues with Multi-Rate SFPs (i.e. 1/10G) and should not be used in those ports.
Workaround: Please use single rate SFP's per the KB article VMware SD-WAN Supported SFP Module List (79270). Multi-Rate SFPs may be used with SFP3 and SFP4.
- Issue 46918:
A VMware SD-WAN Spoke Edge using the 3.4.2 Release does not update the private network id of a Cluster Hub node properly.
- Issue 47084:
A VMware SD-WAN Hub Edge cannot establish more than 750 PIM (Protocol-Independent Multicast) neighbors when it has 4000 Spoke Edges attached.
- Issue 47355:
When the same route is learned via local underlay BGP, Hub BGP and/or statically configured on the Partner Gateway, the sorting order of the routes is incorrect with the Hub BGP being preferred over the underlay BGP.
- Issue 47681:
When a host on the LAN side of a VMware SD-WAN Edge uses the same IP as that Edge’s WAN interface, the connection from the LAN host to the WAN does not work.
- Issue 48166:
A VMware SD-WAN Virtual Edge on KVM is not supported when using a Ciena virtualization OS and the Edge will experience recurring Dataplane Service Failures.
- Issue 48175:
A VMware SD-WAN Edge running Release 3.4.2 will form an OSPF adjacency on a non-global segment if the non-global segment has an interface configured in the same IP range as an interface configured on the global segment
- Issue 48502:
In some scenarios, a VMware SD-WAN Hub Edge being used to backhaul internet traffic may experience a Dataplane Service Failure due the improper handling of backhaul return packets.
- Issue 48530:
VMware SD-WAN Edge 6x0 models do not perform autonegotiation for triple speed (10/100/1000 Mbps) copper SFP's.
Workaround: Edge 520/540 supports triple speed copper SFPs but this model has been marked for End-of-Sale by Q1 2021.
- Issue 48597: Multihop BGP neighborship does not stay up if one of the two paths to the peer goes down
If there is a Multihop BGP neighborship with a peer to which there are multiple paths and one of them goes down, user will notice that the BGP neighborship goes down and does not come up using the other available path(s). This includes the Local IP-loopback neighborship case too.
Workaround: There is no workaround for this issue.
- Issue 48666:
IPsec-fronted Gateway Path MTU calculation does not account for 61 Byte IPsec overhead, resulting in higher MTU advertisement to LAN client and subsequent IPsec packet fragmentation.
Workaround: There is no workaround for this issue.
- Issue 49172:
A Policy Based NAT rule configured with the same NAT subnet for two different VMware SD-WAN Edges does not work.
- Issue 49738:
In some cases, when a VMware SD-WAN Spoke Edge is configured to use multiple Hub Edges, the Spoke Edge may not form tunnels to one of the Hubs configured in the Hub list.
- Issue 50518:
On a VMware SD-WAN Gateway where PKI is turned on, if >6000 PKI tunnels attempt to connect to the Gateway, the tunnels may not all come up because inbound SAs do not get deleted.
Note: Tunnels using pre-shared key (PSK) authentication do not have this issue.
- Issue 51036: ifSpeed reports a 0 value for operational VMware SD-WAN Edge interfaces when polling the Edge via SNMP.
This is an expected behavior for DPDK configured ports. Currently the only way to get speed values for DPDK configured ports is by using the command "debug.py --dpdk_ports". But the SNMP module running on an Edge does not rely on this command to extract speed values for DPDK-configured ports. SNMP only queries via the kernel interface, which unfortunately does not populate speed values for dpdk_ports.
- Issue 51428: Multicast traffic loss may be observed on a site where the VMware SD-WAN Edge has a sub-interface configured with PIM.
When a sub-interface configured with PIM is moved from a segment to another on the fly, pimd (the process that manages PIM) may restart and the site would experience intermittent multicast traffic loss.
Workaround: Deactivate the sub-interface first, and then move the sub-interface to another segment. Once moved, re-activate the sub-interface.
- Issue 52955: DHCP decline is not sent from Edge and DHCP rebinding is not restarted after DAD failure in Stateful DHCP.
If DHCPv6 server allocates an address which is detected as duplicate by the kernel during a DAD check then the DHCPv6 client does not send a decline. This will lead to traffic dropping as the interface address will be marked as DAD check failed and will not be used. This will not lead to any traffic looping in the network but traffic blackholing will be seen.
Workaround: There is no workaround for this Issue.
- Issue 53147: Non default hop limit values advertised in the router advertisements are not honored on the VMware SD-WAN Edge. The hop limit value of the tunnel is always set to 64.
The default value of hop limit is 64. If the desire is to have a non-default hop limit value advertised through router advertisements, the Edge does not process the hop limit fields in the packet and the values remain as 64.
Workaround: There is no workaround to this issue.
- Issue 53219: After a VMware SD-WAN Hub Cluster rebalances, a few Spoke Edges may not have their RPF interface/IIF set properly.
On the affected Spoke Edges, multicast traffic will be impacted. What happens is that after a cluster rebalance, some of the Spoke Edge fail to send a PIM join.
Workaround: This issue will persist until the affected Spoke Edge has an Edge Service restart.
- Issue 53337: Packet drops may be observed with an AWS instance of a VMware SD-WAN Gateway when the throughput is above 3200 Mbps.
When traffic exceeds a throughput above 3200 Mbps and a packet size of 1300 bytes, packets drops are observed at RX and at IPv4 BH handoff.
Workaround: There is no workaround for this issue.
- Issue 53359: BGP/BFD session may fail during some DDoS attack scenarios.
If traffic is flooded from the client connected to the routed interface to the LAN client, the BGP/BFD session can fail. Also when real-time high priority traffic is flooded to the overlay destination, the BGP/BFD session can fail.
Workaround: There is no workaround for this issue.
- Issue 53687: When a VMWare SD-WAN Spoke Edge has a preference for a IPv6/v4 tunnel, the non-preferred v4/v6 tunnels MTU will influence the MTU seen in preferred tunnels too.
An Edge (Spoke or Hub) maintains a system level MTU, which is the minimum of all the link MTU's and this MTU is exchanged as the advertised MTU. Since a non-preferred link's MTU can still be considered for determining the system level MTU, a lower MTU can be advertised than the actual path MTU.
Workaround: There is no workaround for this issue.
- Issue 53830: On a VMware SD-WAN Edge, some of the routes in BGP view may not have the correct preference and advertise values when DCC flag is set to on, causing incorrect sorting order in the Edge's FIB.
When Distributed Cost Calculation (DCC) is turned on in a scaled scenario with a large number of routes on an Edge, when looking at an Edge diagnostic bundle for the log bgp_view some of the routes may not be correctly updated with the preference and advertise values. This issue, if found at all, would be a found in a few Edges as part of a large enterprise (100+ Spoke Edges connected to either Hub Edges or Hub Clusters).
Workaround: This issue can be addressed by either relearning the underlay BGP routes or performing a "Refresh" option on the OFC page of the VMware SD-WAN Orchestrator for the affected routes. Please note that performing a "Refresh" of a route would re-learn the routes from all the Edges in the enterprise.
- Issue 53934: In an enterprise where a VMware SD-WAN Hub Cluster is configured, if the primary Hub has Multihop BGP neighborships on the LAN side, the customer may experience traffic drops on a Spoke Edge when there is a LAN side failure or when BGP is turned off on all segments.
In a Hub cluster, the primary Hub has Multihop BGP neighborship with a peer device to learn routes. If the physical interface on the Hub by which BGP neighborship is established, goes down, then BGP LAN routes may not become zero despite BGP view being empty. This may cause Hub Cluster rebalancing to not happen. The issue may also be observed when BGP is turned off for all segments and when there are one or more Multihop BGP neighborships.
Workaround: Restart the Hub which had the LAN-side failure (or BGP turned off).
- Issue 54099: Fragmented IPv6 packets will be dropped by a VMware SD-WAN Edge.
Any fragmented IPv6 packet is going to get dropped by an Edge.
Workaround: There is no workaround for this issue.
- Issue 54378: IPv6 static address configured interface duplicates address detection (DAD) check fails due to NA drops.
Static address DAD check will not happen and if there is a duplicate address in the network for configured static address then that will not be detected.
Workaround: There is no workaround for this issue.
- Issue 54536: Duplicate address detection (DAD) check not triggered after a VMware SD-WAN Edge reboot.
If there is a duplicate address in the network, then that would not be detected if this DAD check is not performed after reboot.
Workaround: There is no workaround for this issue.
- Issue 54687: DHCPv6 solicit message is not sent by a VMware SD-WAN Edge after the server is provided with a configuration of T1 values greater than T2.
If a DHCPv6 server initially provides T1 value greater than T2, then the Edge does not accept the provided prefix, but even after this configuration has been corrected on the server, the Edge will not send a DHCPv6 solicit message after attempting three times. At that point only when the Edge's Dataplane Service is restarted will the issue get cleared.
Workaround: Restart the Edge's service.
- Issue 54731: Renew messages are sent with a high frequency until the rebind time (t2) is reached when a user changes the value of the IPv6 address range in the server.
When an IP address which has been assigned to a VMware SD-WAN Edge is removed from the valid range of addresses provided by a server, the client keeps sending renew messages to the server until T2 time is reached. This may result in a customer user observing a large quantity of DHCPv6 traffic.
Workaround: There is no workaround for this issue.
- Issue 56218: For a customer site deployed with a High-Availability topology or where HA has just been configured, when the Edges are upgrade from 3.2.x to 3.4.x, the Standby Edge may go down.
When HA is configured, or the HA Edges are upgraded from 3.2.2 to 3.4.x after a WAN setting is configured using the Local UI, the HA interface (e.g. LAN1 or GE1 depending on the Edge model) will be removed from the Standby Edge and HA status will be set to HA_FAILED on the VMware SD-WAN Orchestrator.
Workaround: Reboot the Standby Edge to recover it
- Issue 56454: Configure both IPv4 and IPv6 links as auto-discovered links on an interface and then have tunnels formed through the non-preferred link as well. The link stats do not display consolidated information of a IPv4 and IPv6 link.
When an interface has both IPv4 and IPv6 overlays configured as auto-discovered overlays and tunnels formed over both links, the links stats reflect only the status of the preferred link. The traffic information or the status of the non-preferred link is not correctly reflected. As a result, the statistics seen for the link on the Edge > Monitoring page which include the bandwidth and throughput should be used as a guideline to measure the performance of the tunnels formed over the preferred IP address family only.
Workaround: There is no workaround for this issue.
- Issue 57957: If a DPDK interface is changed from Autonegotiate=on to Autonegotiate=off, the Edge unloads the KNI driver and loads the Linux driver for that interface during the Edge Service restart sequence (from vc_dpdk.py).
After loading up the new Linux interface and nameif'ing it, vc_dpdk.py also needs to invoke "set_interface_neg.py" to apply the auto-negotiation settings. However, because of the new auto-negotiation settings and the Linux driver is reloaded, the bare metal interface is no longer under DPDK control.
- Issue 58453: Some Office365 packets are misclassified as SSL packets by the VMware SD-WAN Edge.
The VMware SD-WAN Deep Packet Inspection (DPI) engine is sometimes misclassifying packets that should be classified as Office365 as SSL instead. The impact is that these flows will be treated as SSL flows versus Office365 flows and that may mean they are treated with less priority, impacting the user experience.
- Issue 59970: Customer will observe traffic drop from a VMware SD-WAN Edge to the datacenter through a Zscaler IaaS, when switching from a primary to a secondary Gateway.
When the primary Gateway goes down and the Orchestrator switches to the secondary Gateway, the current traffic flow reverses path from a Zscaler Enforcement Noted (ZEN) does not work.
Workaround: The workaround would be to reinitiate all traffic flow. The Zscaler is notified about this issue and they confirmed that reverse traffic path is not working properly on their side.
- Issue 61882: When a security parameter configuration change is made (e.g., SA lifetime change) from the VMware SD-WAN Orchestrator, the customer may observe traffic drop for a period of time.
This has been seen on a large-scale deployment with +1000 Edges in a Hub/Spoke topology. If security parameters (lifetime, cryptography algorithms, authentication mode) are changed this will bring down current tunnels, which will then rebuild. In a large-scale deployment this can cause issues with traffic stability. The responder side (Hub Edge) may not be able to handle all the tunnels in time and this can cause a traffic drop. Eventually tunnels will establish, but it can take some time based on the existing number of tunnels.
Workaround: It is recommended to perform configuration changes in a maintenance window, since recovery time is unknown based on the existing number of tunnels.
- Issue 62685: If LAN side NAT is configured with the same outside IP for different LAN subnets with NAT type as source, traffic destined for the Cloud will not work.
For the outside IP used in LAN side NAT rules, we configure a static route and advertise it to the remote branches. For the return traffic to be routed to the correct the LAN subnet, route lookup should be done based on the Inside IP configured in the LAN side NAT rule instead of the next hop in the static route. But for the return traffic from cloud, the route lookup is done based on the next hop in the static route and traffic can get routed to the wrong LAN subnet.
Workaround: Use a different Outside IP for different LAN subnets.
- Issue 62725: A VMware SD-WAN Edge in a network which uses BGP may experience high memory usage under certain rare conditions.
If an Edge learns a BGP route with a next hop IP address which is different from the peer IP address, the next hop will be tracked for reachability by the Edge's Next Hop Tracking (NHT) module. If BGP is then turned off when the tracked IP address is unreachable on the Edge, the tracked NHT entry may not get deleted. In rare cases where there are a lot of stale NHT entries, high Edge memory usage can be seen.
Workaround: Reboot the Edge to delete the memory leaking NHT entries.
- Issue 62897: The debug command tcpdump does not work properly on a VMware SD-WAN Gateway.
Execute tcpdump command on eth0 or eth1 interface of Gateway, the output is not correct. tcpdump.sh and vctcdump are also not working. An attempted fix was attempted, an AppArmor complain profile for vctcpdump (based on the tcpdump profile) that allows tcpdump to inherit vctcpdump's confinement was added, but tcpdump still is not working. Essentially AppArmor causes stdout to stop working for tcpdump. This is a known issue with AppArmor.
Workaround: "pipe tcpdump output to cat. e.g. tcpdump -nnplei eth0 | cat"
- Issue 63125: When MTU is increased for any interface/link on a VMware SD-WAN Hub Edge, it will not be reflected in the path MTU on the Spoke Edge (for the paths with that Hub Edge).
If the user increases the MTU of an interface or link on a Hub, the Spoke Edge path does not pick up the changed MTU setting.
Workaround: Reboot the Spoke Edge, the increased MTU will be reflected in the path MTU on the Spoke.
- Issue 63362: On a site that uses an Enhanced High Availability topology, a DHCP/PPPoE configured interface stops sending traffic after the VMware SD-WAN Standby Edge is either rebooted or power cycled.
In an Enhanced HA setup, if DHCP/PPPoE is configured on a proxy interface (i.e., HA link state set to USE_PEER), it fails to get the address from the server after the Standby Edge reboots or power cycles.
Workaround: Either change the dynamic address to a static address type or do a forced HA failover to get the IP address from the server.
- Issue 64736: DHCPv6 IP's may not get flushed from a VMware SD-WAN Edge interface after configuring the interface from routed to switched
When an interface is converted from routed to switched, the IPv6 IP addresses should be flushed from the interface because IPv6 is not supported on the LAN. But in some instances, the IPv6 IP addresses are not getting flushed post-conversion to switched. These IP addresses even persist if the interface is bounced. The issue does not occur every time an interface is converted.
Workaround: Reboot the Edge to clear the IPv6 IP addresses from the newly converted switched interface.
- Issue 64909: Datacenter route for a Non SD-WAN Destination via Gateway is set to false when changing the Gateway IP address.
When the IP address is changed for the Primary Gateway of an NSD, the tunnel comes up, but if routes are checked on the Gateway, the route to the datacenter/peer is set to False.
Workaround: If the Gateway service is restarted, the datacenter route comes up, which is something that should be done in a window.
- Issue 19566:
After High Availability failover, the serial number of the standby VMware SD-WAN Edge may be shown as the active serial number in the Orchestrator.
- Issue 21342:
When assigning Partner Gateways per-segment, the proper list of Gateway Assignments may not show under the Operator option "View" Gateways on the VMware SD-WAN Edge monitoring list.
- Issue 24269:
Monitor > Transport > Loss not graphing observed WAN link loss while QoE graphs do reflect this loss.
- Issue 25932:
The VMware SD-WAN Orchestrator allows VMware SD-WAN Gateways to be removed from the Gateway Pool even when they are in use.
- Issue 32335:
The ‘End User Service Agreement’ (EUSA) page throws an error when a user is trying to accept the agreement.
Workaround: Ensure no leading or trailing spaces are found in Enterprise Name.
- Issue 32435:
A VMware SD-WAN Edge override for a policy-based NAT configuration is permitted for tuples which are already configured at the profile level and vice versa.
- Issue 32856:
Though a business policy is configured to use the Hub cluster to backhaul internet traffic, the user can unselect the Hub cluster from a profile on a VMware SD-WAN Orchestrator that has been upgraded from Release 3.2.1 to Release 3.3.x.
- Issue 32913:
After Enabling High Availability, Multicast details for the VMware SD-WAN Edge are not displayed on the Monitoring Page. A failover resolves the issue.
- Issue 34828:
Traffic cannot pass between a VMware SD-WAN Spoke Edge using release 2.x and a Hub Edge using release 3.3.1.
- Issue 35658:
When a VMware SD-WAN Edge is moved from one profile to another which has a different CSS setting (e.g. IPsec in profile1 to GRE in profile2), the Edge level CSS settings will continue to use the previous CSS settings (e.g. IPsec versus GRE).
Workaround: Turn off, and then turn on GRE at the Edge level to resolve the issue.
- Issue 35667:
When a VMware SD-WAN Edge is moved from one profile to another profile which has the same CSS setting but a different GRE CSS name (the same endpoints), some GRE tunnels will not show in monitoring.
Workaround: Turn off, and then turn on GRE at the Edge level to resolve the issue.
- Issue 36665:
If the VMware SD-WAN Orchestrator cannot reach the internet, user interface pages that require accessing the Google Maps API may fail to load entirely.
- Issue 38056:
The Edge-Licensing export.csv file does not show region data.
- Issue 38843:
When pushing an application map, there is no Operator event, and the Edge event is of limited utility.
- Issue 39633:
The Super Gateway hyper link does not work after a user assigns the Alternate Gateway as the Super Gateway.
- Issue 39790:
The VMware SD-WAN Orchestrator allows a user to configure a VMware SD-WAN Edge’s routed interface to have greater than the supported 32 subinterfaces, creating the risk that a user can configure 33 or more subinterfaces on an interface which would cause a Dataplane Service Failure for the Edge.
- Issue 40341:
Though the Skype application is properly categorized on the backend as Real Time traffic, when editing the Skype Business Policy on the VMware SD-WAN Orchestrator, the Service Class may erroneously display “Transactional”.
- Issue 41691:
User cannot change the 'Number of addresses' field although the DHCP pool is not exhausted on the Configure > Edge > Device page.
- Issue 43276:
User cannot change the Segment type when a VMware SD-WAN Edge or Profile has a partner gateway configured.
- Issue 44153:
The VMware SD-WAN Orchestrator does not consistently send alert emails to the email addresses configured in the 'Alerts and Notifications' section.
- Issue 47269:
The VMware SD-WAN 510-LTE interface may appear for Edge models that do not support an LTE interface.
- Issue 47713:
If a Business Policy Rule is configured while Cloud VPN is set to off, the NAT configuration must be reconfigured upon enabling Cloud VPN.
- Issue 47820:
If a VLAN is configured with DHCP set to off at the Profile level, while also having an Edge Override for this VLAN on that Edge with DHCP turned on, and there is an entry for the DNS server field set to none (no IP configured), the user will be unable to make any changed on the Configure > Edge > Device page and will get an error message of ‘invalid IP address []’ that does not explain or point to the actual problem.
- Issue 48085:
The VMware SD-WAN Orchestrator allows a user to delete a VLAN which is associated with an interface.
- Issue 48737:
On a VMware SD-WAN Orchestrator which is using the Release 4.0.0 new user interface, If a user is on a Monitor page and changes the Start & End time interval and then navigates between tabs, the Orchestrator does not update Start & End interval time to the new values.
- Issue 49225:
VMware SD-WAN Orchestrator does not enforce a limit of 32 total VLANs.
- Issue 49790:
When a VMware SD-WAN Edge is activated to Release 4.0.0, the activation is posted twice in Events.
Workaround: Ignore the duplicate event.
- Issue 50531:
When two Operators of differing privileges use the same browser window when accessing the New UI on a 4.0.0 Release version of the VMware SD-WAN Orchestrator, and the Operator with lesser privileges tries to login after the Operator with higher privileges, that lesser privileged Operator will observe multiple errors stating that the "user does not have privilege".
Note: There is no escalation in privileges for the Operator with lower privileges, only the display of error messages.
Workaround: The next operator may refresh that page prior to logging in to prevent seeing the errors, or each Operator may use different browser windows to avoid this display issue.
- Issue 51722: On the Release 4.0.0 VMware SD-WAN Orchestrator, the time range selector is no greater than two weeks for any statistic in the Monitor > Edge tabs.
The time range selector does not show options greater than "Past 2 Weeks" in Monitor > Edge tabs even if the retention period for a set of statistics is much longer than 2 weeks. For example, flow and link statistics are retained for 365 days by default (which is configurable), while path statistics are retained only for 2 weeks by default (also configurable). This issue is making all monitor tabs conform to the lowest retained type of statistic versus allowing a user to select a time period that is consistent with the retention period for that statistic.
Workaround: A user may use the "Custom" option in the time range selector to see data for more than 2 weeks.
- Issue 63726: On a site which has configured a Non SD-WAN Destination (NSD) via Gateway using a Zscaler type and assigned it to the global segment, if a user changes the assigned Zscaler NSD from the the global, to a non-global segment, the 0.0.0.0/0 route to the peer is missing on the VMware Edges.
On the VMware Gateway the route is present, but on the Edge the expected data center route is not present. The Orchestrator is not assigning the route to Edges on the newly assigned non-global segment.
Workaround: Remove the datacenter configuration from the parent segment and save the configuration. Wait for the tunnel to be completely torn down by checking events. Now assign a new datacenter configuration to the new non-global segment and save the configuration. New routes will be assigned to to the new non-global segment.
- Issue 64716: User is unable to generate reports on the VMware SD-WAN Orchestrator.
When the user attempts to generate a report it will fail and with 'Error' displayed in the Status column of the Reports page.
Workaround: There is no workaround for this issue.
- Issue 64847: On a VMware SD-WAN Orchestrator UI where the browser locale is set to a non-English language, when using the New UI the Top Level Menu pane is missing all options.
On the New UI, there is a "three-line" icon in the upper right corner of the screen and if a user clicks the "three-line" icon, it should trigger the opening of a "Top Level Menu" pane. These icons sometimes fail to navigate to the intended target.
Workaround: Reload the page to force the intended page to display.
- Issue 62934: For an enterprise using VMware Cloud Web Security, if a client user opens a Chrome browser in Incognito and attempts to download a file, the download may occasionally not be successful.
Incognito requires enabling 3rd party cookies. Turn on 3rd party cookies and retry the operation. On an unsuccessful download the user would observe a screen which reads either "Error occurred contact your administrator" or for files from a custom web server: "This page is not working". Occasionally some web servers or Files may have a variance in File signature, that the Cloud Web Security Service may not be able to recognize, and hence this issue.
Workaround: Turn on allowing 3rd party Cookies and retry. No known workaround for this issue if using an Incognito window.
- Issue 62978: For an enterprise using VMware Cloud Web Security, if a client user opens a Chrome browser in Incognito and attempts to download a file, the download may fail and if it does the user may see an error screen that lacks VMware branding.
In the above scenario where a file download fails (which is covered in Issue #62934) the error screen "Error occurred contact your administrator" should show VMware branding, but instead shows No branding. If a user encounters this, it should be understood this is not an expected outcome.
Workaround: Allow 3rd party cookies in browser.
- Issue 63149: When a customer deployment has overlapping subnets in a profile and configures a subnet for a VMware Cloud Web Security policy, and associates the Cloud Web Security policy to the profile and segment, Edge clients on that subnet will not be able to connect to the internet.
Should witness traffic failing, clients will not be able to access internetIf there are overlapping subnets configured for the LAN segments behind VMware SD-WAN Edges within the same segment, then the resources behind the Edges cannot have Cloud Web Security policies applied for the internet-bound traffic. This is because there is no way to uniquely identify the destination Edge for the return traffic from the internet to Cloud Web Security.
Workaround: Turn on LAN side NAT on the Edge and have a unique subnet represent the traffic originated from resources behind the Edge.
- Issue 64429: A customer enterprise using VMware Cloud Web Security, where a Cloud Web Security policy is configured and applied using internet backhaul, a UDP transfer to a server in Internet, initiated by a Client Behind Edge, with DF bit set, UDP transfer will fail.
Cloud Web Security will receive MTU sized packets, but will need to send ICMP unreachable message back to sender, as fragmentation is not allowed due to Do Not Fragment (DF) bit set. Cloud Web Security is executing DNAT (destination network address translation) to the wrong source IP address and is sending a "ICMP unreachable fragmentation needed" message to Client instead of sending it back to Server.
Workaround: TCP is expected to work, or use UDP without DF bit set.
- Issue 65001: For a customer using VMware Cloud Web Security, a user cannot configure the Inspection Engine to turn on/off File Hash checks when using the VMware Orchestrator to do so.
When a user is using the Orchestrator to configure the Cloud Web Security Inspection engine's File Hash check parameter for either "Action for Unknown File Download" and "Action for unknown document Download", these changes are not sent to the Inspection Engine and are not applied.
Workaround: There is no workaround for this issue.
- Issue 65115: For a customer using VMware Cloud Web Security, if SAML authentication is configured, a user may not be able access any websites.
With SAML (Security Assertion Markup Language) authentication turned on, if a user accesses any website it results in an authentication loop where access to the IdP (Identify Provider) itself requires authentication and fails.
Workaround: Add an SSL Exception in the security policy to allow access to the IdP login URL without authentication.
- Issue 64541: For a customer using VMware Secure Access, when using the option in Workspace ONE UEM Configuration to configure Tunnel Hostname within the Organization Group, if a user selects 'Yes', the hostname will be created in the UEM console automatically instead of being configured manually.
The user should have the option to configure the hostname manually and not just have it automatically created.
Workaround: The workaround is to manually set it in the UEM console.