Updated August 31st, 2021
VMware SD-WAN Orchestrator Version R410-20201028-GA
Check regularly for additions and updates to these release notes.
What's in the Release NotesThe release notes cover the following topics:
- Recommended Use
- New Features
- Orchestrator API Changes
- Revision History
- Resolved Issues
- Known Issues
This release is no longer recommended for use and customers are encouraged to upgrade to Release 4.2.1 or newer as soon as possible.
Release 4.1.0 Orchestrators, Gateways, and Hub Edges support all previous VMware SD-WAN Edge versions greater than or equal to Release 3.0.0.
VMware Edge Network Intelligence™
VMware Edge Network Intelligence™ is a vendor agnostic AIOps solution based on technology from Nyansa, which VMware acquired in February 2020. With VMware SD-WAN Release 4.1.0, VMware Edge Network Intelligence has been integrated with VMware SD-WAN™ to offer insights and visibility tied to users and Internet of Things (IoT) devices to provide a complete, end-to-end understanding of the entire edge network. With this integration, VMware Edge Network Intelligence can extract data from different vantage points for each application flow, including wireless controllers, LAN switches, network services, VMware SD-WAN Edges, VMware SD-WAN Edge clusters, as well as application performance metrics. The solution provides deep visibility into the client experience for every application with actionable insights for proactive remediation.
With this release, VMware Edge Network Intelligence data collection functionality is built into the VMware SD-WAN Edge firmware, enabling inline data collection. The integrated solution is available on all models of the VMware SD-WAN Edge, including the Virtual Edge appliances. The Zero Touch Provisioning feature of the VMware SD-WAN Edge takes care of provisioning VMware Edge Network Intelligence and is configured centrally using the VMware SD-WAN Orchestrator. Once the Edge is provisioned, the analytics functionality collects data, performs deep packet inspection of all traffic, identifies network applications and correlates traffic with user information. The metadata trends and analytics are then sent directly to the VMware Edge Network Intelligence Cloud Analytics Engine. Customers will be able to seamlessly access the VMware Edge Network Intelligence user interface from the VMware SD-WAN Orchestrator user interface.
Existing VMware SD-WAN customers can take advantage of VMware Edge Network Intelligence with a software upgrade and license addition. Please speak to an account manager to add this to an existing VMware SD-WAN platform.
VMware Edge Network Intelligence Use Cases
- Application assurance: Analyzes over 3,000 applications, tracking performance for fault detection and fault isolation. This helps IT teams determine the worst performing clients for an application, identify if the problem is systemic or isolated, and compare performance with other locations within the organization and with peers in the industry.
- Improved wireless and wired end-client experience: Significantly improves and quantifies the end user and IoT device experience at any location, helping pinpoint whether a perceived application problem is due to issues with the local Wi-Fi network, broadband network, WAN, network services, or with the application.
- Business continuity and work-from-home: Helps IT teams manage their remote workforce by collecting application data related to user experience directly from SaaS applications, and combining it with telemetry received from a client application installed on the end user device.
- Change verification and return on investment (ROI): Offers change verification by comparing performance before and after a change. This provides quantifiable data on whether a change has resolved an issue or whether a rollback is required, instead of waiting for users to open cases to find out. This helps justify the ROI.
- Fault isolation and recommendations: Isolates a fault to the client LAN, WAN, data center LAN, cloud, the internet, or the application. Using machine learning (ML) algorithms such as clustering, analyzes historical network data to determine where the problem occurs and makes recommendations and predictions to the IT team.
The complete 4.1.0 API reference is available via code.vmware.com.
Changes in this release are as follows:
- New Endpoints
- post /analytics/getAnalyticsSystemUrl
- post /enterprise/getAnalyticsConfiguration
- post /enterprise/insertOrUpdateAnalyticsConfiguration
- New Request Parameters
- post /edge/edgeProvision
- new request parameter: analyticsMode (in: body, type: string)
October 29th, 2020. First Edition.
June 4th, 2021. Second Edition.
Revised the Recommend Use section to read: "This release is no longer recommended for use and customers are encouraged to upgrade to Release 4.2.1 or newer as soon as possible."
The resolved issues are grouped as follows.Edge/Gateway Resolved Issues
Resolved in Version R410-20201028
The below issues have been resolved since Edge and Gateway version R400-20201002.
- Fixed Issue 33195:
PIM joins may fail on the VMware SD-WAN Hub when it has more than 1280 multicast neighbors.
Resolved in R410-20201028-GA
The below issues have been resolved since Orchestrator version R400-20201001-GA
- Fixed Issue 47279:
The IKE/IPSEC Template is not correct for the Non SD-WAN Destinations via Gateway type “Generic Firewall” (Policy Based VPN).
- Fixed Issue 47910:
When a Non SD-WAN Destinations via Gateway with type Checkpoint has its configuration modified through the Monitor > Network Services screen, the primary VPN goes down due to the VMware SD-WAN Orchestrator pushing a configuration update which includes the wrong NVS type.
Open Issues in Release 4.1.0
The known issues are grouped as follows.Edge/Gateway Known Issues
- Issue 14655:
Plugging or unplugging an SFP adapter may cause the device to stop responding on the Edge 540, Edge 840, and Edge 1000 and require a physical reboot.
Workaround: The Edge must be physically rebooted. This may be done either on the Orchestrator using Remote Actions > Reboot Edge, or by power-cycling the Edge.
- Issue 25504:
Static route costs greater than 255 may result in unpredictable route ordering.
Workaround: Use a route cost between 0 and 255
- Issue 25595:
A restart may be required for changes to static SLA on a WAN overlay to work properly.
Workaround: Restart Edge after adding and removing Static SLA from WAN overlay
- Issue 25742:
Underlay accounted traffic is capped at a maximum of the capacity towards the VMware SD-WAN Gateway, even if that is less than the capacity of a private WAN link which is not connected to the Gateway.
- Issue 25758:
USB WAN links may not update properly when switched from one USB port to another until the VMware SD-WAN Edge is rebooted.
Workaround: Reboot the Edge after moving USB WAN links from one port to another.
- Issue 25855:
A large configuration update on the Partner Gateway (e.g. 200 BGP-enabled VRFs) may cause latency to increase for approximately 2-3 seconds for some traffic via the VMware SD-WAN Gateway.
Workaround: No workaround available.
- Issue 25921:
VMware SD-WAN Hub High Availability failover takes longer than expected (up to 15 seconds) when there are three thousand branch Edges connected to the Hub.
- Issue 25997:
The VMware SD-WAN Edge may require a reboot to properly pass traffic on a routed interface that has been converted to a switched port.
Workaround: Reboot the Edge after making the configuration change.
- Issue 26421:
The primary Partner Gateway for any branch site must also be assigned to a VMware SD-WAN Hub cluster for tunnels to the cluster to be established.
- Issue 28175:
Business Policy NAT fails when the NAT IP overlaps with the VMware SD-WAN Gateway interface IP.
- Issue 31210:
VRRP: ARP is not resolved in the LAN client for the VRRP virtual IP address when the VMware SD-WAN Edge is master with a non-global CDE segment running on the LAN interface.
- Issue 32731:
Conditional default routes advertised via OSPF may not be withdrawn properly when the route is turned off. Re-enabling and disabling the route will retract it successfully.
- Issue 32960:
Interface “Autonegotiation” and “Speed” status might be displayed incorrectly on the Local Web UI for activated VMware SD-WAN Edges.
- Issue 32981:
Hard-coding speed and duplex on a DPDK-enabled port may require a VMware SD-WAN Edge reboot for the configurations to take effect as it requires disabling DPDK.
- Issue 34254:
When a Zscaler CSS is created and the Global Segment has FQDN/PSK settings configured, these settings are copied to Non-Global Segments to form IPsec tunnels to a Zscaler CSS.
- Issue 35778:
When there are multiple user-defined WAN links on a single interface, only one of those WAN links can have a GRE tunnel to Zscaler.
Workaround: Use a different interface for each WAN link that needs to build GRE tunnels to Zscaler.
- Issue 35807:
A DPDK routed interface will be disabled completely if the interface is disabled and re-enabled from the VMware SD-WAN Orchestrator.
- Issue 36923:
Cluster name may not be updated properly in the NetFlow interface description for a VMware SD-WAN Edge which is connected to that Cluster as its Hub.
- Issue 38682:
A VMware SD-WAN Edge acting as a DHCP server on a DPDK-enabled interface may not properly generate “New Client Device" events for all connected clients.
- Issue 38767:
When a WAN overlay that has GRE tunnels to Zscaler configured is changed from auto-detect to user-defined, stale tunnels may remain until the next restart.
Workaround: Restart the Edge to clear the stale tunnel.
- Issue 39134:
The System health statistic “CPU Percentage” may not be reported correctly on Monitor > Edge > System for the VMware SD-WAN Edge, and on Monitor > Gateways for the VMware SD-WAN Gateway.
Workaround: Users should use handoff queue drops for monitoring Edge capacity not CPU percentage.
- Issue 39374:
Changing the order of VMware SD-WAN Partner Gateways assigned to a VMware SD-WAN Edge may not properly set Gateway 1 as the local Gateway to be used for bandwidth testing.
- Issue 39608:
The output of the Remote Diagnostic “Ping Test” may display invalid content briefly before showing the correct results.
- Issue 39624:
Ping through a subinterface may fail when the parent interface is configured with PPPoE.
- Issue 39659:
On a site configured for Enhanced High Availability, with one WAN link on each VMware SD-WAN Edge, when the standby Edge has only PPPoE connected and the active has only non-PPPoE connected, a split brain state (active/active) may be possible if the HA cable fails.
- Issue 39753:
Disabling Dynamic Branch-to-Branch VPN may cause existing flows currently being sent using Dynamic Branch-to-Branch to stall.
- Issue 40096:
If an activated VMware SD-WAN Edge 840 is rebooted, there is a chance an SFP module plugged into the Edge will stop passing traffic even though the link lights and the VMware SD-WAN Orchestrator will show the port as 'UP'.
Workaround: Unplug the SFP module and then replug it back into the port.
- Issue 40421:
Traceroute is not showing the path when passing through a VMware SD-WAN Edge with an interface configured as a switched port.
- Issue 42278:
For a specific type of peer misconfiguration, the VMware SD-WAN Gateway may continuously send IKE init messages to a Non-SD-WAN peer. This issue does not disrupt user traffic to the Gateway; however, the Gateway logs will be filled with IKE errors and this may obscure useful log entries.
- Issue 42388:
On a VMware SD-WAN Edge 540, an SFP port is not detected after disabling and reenabling the interface from the VMware SD-WAN Orchestrator.
- Issue 42488:
On a VMware SD-WAN Edge that has a switched port with VRRP enabled, if the cable is disconnected and the Edge Service is restarted, the LAN connected routes are advertised.
Workaround: There is no workaround for this issue.
- Issue 42872:
Enabling Profile Isolation on a Hub profile where a Hub cluster is associated does not revoke the Hub routes from the routing information base (RIB).
- Issue 43373:
When the same BGP route is learnt from multiple VMware SD-WAN Edges, if this route is moved from preferred to eligible exit in the Overlay Flow Control, the Edge is not removed from the advertising list and continues to be advertised.
Workaround: Enable distributed cost calculation on the VMware SD-WAN Orchestrator.
- Issue 44832:
Traffic from one Non SD-WAN Destinations via Edge to another Non SD-WAN Destinations via Edge (i.e. 'hairpinning' or 'NAT loopback'), is dropped on the VMware SD-WAN Edge.
- Issue 44995:
OSPF routes are not revoked from VMware SD-WAN Gateways and VMware SD-WAN Spoke Edges when the routes are withdrawn from the Hub Cluster.
- Issue 45189:
With source LAN side NAT is configured, the traffic from a VMware SD-WAN Spoke Edge to a Hub Edge is allowed even without the static route configuration for the NAT subnet.
- Issue 45302:
In a VMware SD-WAN Hub Cluster, if one Hub loses connectivity for more than 5 minutes to all of the VMware SD-WAN Gateways common between itself and its assigned Spoke Edges, the Spokes may in rare conditions be unable to retain the hub routes after 5 minutes. The issue resolves itself when the Hub regains contact with the Gateways.
- Issue 46053:
BGP preference does not get auto-corrected for overlay routes when its neighbor is changed to an uplink neighbor.
Workaround: An Edge Service Restart will correct this issue.
- Issue 46137:
A VMware SD-WAN Edge running 3.4.x software does not initiate a tunnel with AES-GCM encryption even if the Edge is configured for GCM.
- Issue 46216:
On a Non SD-WAN Destinations via Gateway or Edge where the peer is an AWS instance, when the peer initiates Phase-2 re-key, the Phase-1 IKE is also deleted and forces a re-key. This means the tunnel is torn down and rebuilt, causing packet loss during the tunnel rebuild.
Workaround: To avoid tunnel destruction, configure the Non SD-WAN Destinations via Gateway/Edge or CSS IPsec rekey timer to less than 60 minutes. This prevents AWS from initiating the re-key.
- Issue 46391:
For a VMware SD-WAN Edge 3800, the SFP1 and SFP2 interfaces each have issues with Multi-Rate SFPs (i.e. 1/10G) and should not be used in those ports.
Workaround: Please use single rate SFP's per the KB article VMware SD-WAN Supported SFP Module List (79270). Multi-Rate SFPs may be used with SFP3 and SFP4.
- Issue 46628:
The GE5 and GE6 ports on a VMware SD-WAN Edge 620/640/680 do not detect a link if the ports are configured with 100 Mbps and duplex.
- Issue 46918:
A VMware SD-WAN Spoke Edge using the 3.4.2 Release does not update the private network id of a Cluster Hub node properly.
- Issue 47084:
A VMware SD-WAN Hub Edge cannot establish more than 750 PIM (Protocol-Independent Multicast) neighbors when it has 4000 Spoke Edges attached.
- Issue 47244:
On an activated VMware SD-WAN Edge 6x0 with DPDK enabled, some Copper SFPs, the Edge will show the link as 'UP' even when no cable is inserted on the VMware SD-WAN Orchestrator UI.
Workaround: Plugging and unplugging a cable removes the false state.
- Issue 47355:
When the same route is learned via local underlay BGP, Hub BGP and/or statically configured on the Partner Gateway, the sorting order of the routes is incorrect with the Hub BGP being preferred over the underlay BGP.
- Issue 47664:
In a Hub and Spoke configuration where Branch-to-Branch via Hub VPN is disabled, trying to U-turn Branch-to-Branch traffic using a summary route on an L3 switch/router will cause routing loops.
Workaround: Configure Cloud VPN to enable Branch-to-Branch VPN and select “Use Hubs for VPN”.
- Issue 47681:
When a host on the LAN side of a VMware SD-WAN Edge uses the same IP as that Edge’s WAN interface, the connection from the LAN host to the WAN does not work.
- Issue 47787:
A VMware SD-WAN Spoke Edge configured with a backhaul business policy incorrectly sends traffic via the VMware SD-WAN Gateway path if that flow is initiated from the Hub Edge to that Spoke Edge.
- Issue 48166:
A VMware SD-WAN Virtual Edge on KVM is not supported when using a Ciena virtualization OS and the Edge will experience recurring Dataplane Service Failures.
- Issue 48175:
A VMware SD-WAN Edge running Release 3.4.2 will form an OSPF adjacency on a non-global segment if the non-global segment has an interface configured in the same IP range as an interface configured on the global segment
- Issue 48488:
Business policy rule is not overridden if the outbound traffic box is not checked (for the Traffic initiated from remote peer and allowed by 1:1 NAT rule).
Workaround: Please check “Outbound Traffic” in 1:1 NAT.
- Issue 48502:
In some scenarios, a VMware SD-WAN Hub Edge being used to backhaul internet traffic may experience a Dataplane Service Failure due the improper handling of backhaul return packets.
- Issue 48530:
VMware SD-WAN Edge 6x0 models do not perform autonegotiation for triple speed (10/100/1000 Mbps) copper SFP's.
Workaround: Edge 520/540 supports triple speed copper SFPs but this model has been marked for End-of-Sale by Q1 2021.
- Issue 48666:
IPsec-fronted Gateway Path MTU calculation does not account for 61 Byte IPsec overhead, resulting in higher MTU advertisement to LAN client and subsequent IPsec packet fragmentation.
Workaround: There is no workaround for this issue.
- Issue 49172:
A Policy Based NAT rule configured with the same NAT subnet for two different VMware SD-WAN Edges does not work.
- Issue 49738:
In some cases, when a VMware SD-WAN Spoke Edge is configured to use multiple Hub Edges, the Spoke Edge may not form tunnels to one of the Hubs configured in the Hub list.
- Issue 50433:
When a secondary IP address is deleted from a routed interface on a VMware SD-WAN Edge, the corresponding route may not be removed from the peer Edges.
Workaround: Perform a Remote Actions > Restart Service for the Edge from which the secondary IP address was removed.
- Issue 50518:
On a VMware SD-WAN Gateway where PKI is enabled, if >6000 PKI tunnels attempt to connect to the Gateway, the tunnels may not all come up because inbound SAs do not get deleted.
Note: Tunnels using pre-shared key (PSK) authentication do not have this issue.
- Issue 19566:
After High Availability failover, the serial number of the standby VMware SD-WAN Edge may be shown as the active serial number in the Orchestrator.
- Issue 20900:
If the MaxMind geolocation service is enabled and cannot reach the MaxMind server, new VMware SD-WAN Edge activations will not work.
- Issue 21342:
When assigning Partner Gateways per-segment, the proper list of Gateway Assignments may not show under the Operator option "View" Gateways on the VMware SD-WAN Edge monitoring list.
- Issue 24269:
Monitor > Transport > Loss not graphing observed WAN link loss while QoE graphs do reflect this loss.
- Issue 25932:
The VMware SD-WAN Orchestrator allows VMware SD-WAN Gateways to be removed from the Gateway Pool even when they are in use.
- Issue 32335:
The ‘End User Service Agreement’ (EUSA) page throws an error when a user is trying to accept the agreement.
Workaround: Ensure no leading or trailing spaces are found in Enterprise Name.
- Issue 32435:
A VMware SD-WAN Edge override for a policy-based NAT configuration is permitted for tuples which are already configured at the profile level and vice versa.
- Issue 32856:
Though a business policy is configured to use the Hub cluster to backhaul internet traffic, the user can unselect the Hub cluster from a profile on a VMware SD-WAN Orchestrator that has been upgraded from Release 3.2.1 to Release 3.3.x.
- Issue 32913:
After Enabling High Availability, Multicast details for the VMware SD-WAN Edge are not displayed on the Monitoring Page. A failover resolves the issue.
- Issue 33026:
The ‘End User Service Agreement’ (EUSA) page does not reload properly after deleting the agreement.
- Issue 34828:
Traffic cannot pass between a VMware SD-WAN Spoke Edge using release 2.x and a Hub Edge using release 3.3.1.
- Issue 35658:
When a VMware SD-WAN Edge is moved from one profile to another which has a different CSS setting (e.g. IPsec in profile1 to GRE in profile2), the Edge level CSS settings will continue to use the previous CSS settings (e.g. IPsec versus GRE).
Workaround: Disable and then reenable GRE at the Edge level to resolve the issue.
- Issue 35667:
When a VMware SD-WAN Edge is moved from one profile to another profile which has the same CSS setting but a different GRE CSS name (the same endpoints), some GRE tunnels will not show in monitoring.
Workaround: Disable and then reenable GRE at the Edge level to resolve the issue.
- Issue 36665:
If the VMware SD-WAN Orchestrator cannot reach the internet, user interface pages that require accessing the Google Maps API may fail to load entirely.
- Issue 38056:
The Edge-Licensing export.csv file not show region data.
- Issue 38843:
When pushing an application map, there is no Operator event, and the Edge event is of limited utility.
- Issue 39633:
The Super Gateway hyper link does not work after a user assigns the Alternate Gateway as the Super Gateway.
- Issue 39790:
The VMware SD-WAN Orchestrator allows a user to configure a VMware SD-WAN Edge’s routed interface to have greater than the supported 32 subinterfaces, creating the risk that a user can configure 33 or more subinterfaces on an interface which would cause a Dataplane Service Failure for the Edge.
- Issue 40341:
Though the Skype application is properly categorized on the backend as Real Time traffic, when editing the Skype Business Policy on the VMware SD-WAN Orchestrator, the Service Class may erroneously display “Transactional”.
- Issue 41691:
User cannot change the 'Number of addresses' field although the DHCP pool is not exhausted on the Configure > Edge > Device page.
- Issue 43276:
User cannot change the Segment type when a VMware SD-WAN Edge or Profile has a partner gateway configured.
- Issue 44153:
The VMware SD-WAN Orchestrator does not consistently send alert emails to the email addresses configured in the 'Alerts and Notifications' section.
- Issue 46254:
During a VMware SD-WAN Edge activation, the VMware SD-WAN Orchestrator does not detect a changed WAN link MTU or the presence of a VLAN ID for DHCP configured interfaces.
- Issue 46482:
If a site using VMware SD-WAN Edge 540’s configured in High-Availability is upgraded to Edge software release 3.4.1, the VMware SD-WAN Orchestrator will display this site’s HA status as “Standby Failed”.
- Issue 47269:
The VMware SD-WAN 510-LTE interface may appear for Edge models that do not support an LTE interface.
- Issue 47713:
If a Business Policy Rule is configured while Cloud VPN is disabled, the NAT configuration must be reconfigured upon enabling Cloud VPN.
- Issue 47820:
If a VLAN is configured with DHCP disabled at the Profile level, while also having an Edge Override for this VLAN on that Edge with DHCP enabled, and there is an entry for the DNS server field set to none (no IP configured), the user will be unable to make any changed on the Configure > Edge > Device page and will get an error message of ‘invalid IP address ’ that does not explain or point to the actual problem.
- Issue 48085:
The VMware SD-WAN Orchestrator allows a user to delete a VLAN which is associated with an interface.
- Issue 48737:
On a VMware SD-WAN Orchestrator which is using the Release 4.0.0 new user interface, If a user is on a Monitor page and changes the Start & End time interval and then navigates between tabs, the Orchestrator does not update Start & End interval time to the new values.
- Issue 49225:
VMware SD-WAN Orchestrator does not enforce a limit of 32 total VLANs.
- Issue 49790:
When a VMware SD-WAN Edge is activated to Release 4.0.0, the activation is posted twice in Events.
Workaround: Ignore the duplicate event.
- Issue 50531:
When two Operators of differing privileges use the same browser window when accessing the New UI on a 4.0.0 Release version of the VMware SD-WAN Orchestrator, and the Operator with lesser privileges tries to login after the Operator with higher privileges, that lesser privileged Operator will observe multiple errors stating that the "user does not have privilege".
Note: There is no escalation in privileges for the Operator with lower privileges, only the display of error messages.
Workaround: The next operator may refresh that page prior to logging in to prevent seeing the errors, or each Operator may use different browser windows to avoid this display issue.
- For the Edge 500, 510, 520 and 610, maximum throughput is obtainable with analytics enabled.
- For all other models, there is a performance penalty of up to 20% when analytics are enabled. This penalty will be reduced in a subsequent release.
- Flow capacity is reduced by half when analytics are enabled due to the additional memory and processing required for analysis. This penalty will be reduced in a subsequent release.