This section covers the performance and scale architecture of the VMware SD-WAN Edge. It provides recommendations based on tests conducted on the various Edges configured with specific service combinations. It also explains performance and scale data points and how to use them.
Introduction
The tests represent common deployment scenarios to provide recommendations that apply to most deployments. The test data herein are not all-inclusive metrics, nor are they performance or scale limits. There are implementations where the observed performance exceeds the test results and others where specific services, extremely small packet sizes, or other factors can reduce performance below the test results.
Customers are welcome to perform independent tests, and results could vary. However, recommendations based on our test results are adequate for most deployments.
VMware SD-WAN EdgeVMware SD-WAN Edges are zero-touch, enterprise-class appliances that provide secure optimized connectivity to private, public, and hybrid applications as well as compute and virtualized services. VMware SD-WAN Edges perform deep application recognition of traffic flows, performance metrics measurements of underlay transport and apply end-to-end quality of service by applying packet-based link steering and on-demand application remediation, in addition to supporting other virtualized network services.
Throughput Performance Test Topologies
Test Methodology
This subsection details the performance and scale test methodology used to derive the results.
Performance Test MethodologyThe testing methodology for Edges uses the industry benchmarking standard RFC 2544 as a framework to execute throughput performance testing. There are specific changes to the type of traffic used and configurations set during testing, described below:
- Performance is measured using a fully operational SD-WAN network overlay (DMPO tunnels) test topology in order to exercise the SD-WAN features and obtain results that can be used to appropriately size WAN networks. Testing is conducted using stateful traffic that establishes multiple flows (connections) and are a mix of well-known applications. The number of flows depends on the platform model being tested. Platforms are divided by expected aggregate performance of under 1 Gbps and over 1 Gbps models. Typically, hundreds of flows are needed to fully exercise and determine max throughput of platforms expected to perform under 1 Gbps, and thousands of flows are used to exercise platforms of over 1 Gbps.
The traffic profiles simulate two network traffic conditions:
- Large Packet, a 1300-byte condition.
- IMIX, a mix of packet sizes that average to a 417-byte condition.
These traffic profiles are used separately to measure maximum throughput per profile.
- Performance results are recorded at a packet drop rate (PDR) of 0.01%. The PDR mark provides a more realistic performance result which accounts for normal packet drop that may occur within the SD-WAN packet pipeline in the device. A PDR of 0.01% does not impact application experience even in single link deployment scenarios.
- The device under test is configured with the following DMPO features; IPsec encrypted using AES-128 and SHA1 for hashing, Application Recognition, link SLA measurements, per-packet forwarding. Business Policy is configured to match all traffic as bulk/low priority to prevent DMPO NACK or FEC from executing and incorrectly altering the traffic generator’s packet count tracking.
Test Results
VMware SD-WAN Edge Performance and Scale Results
Performance metrics are based on the Test Methodology detailed above.
Switched Port Performance: VMware SD-WAN Edges are designed to be deployed as gateway routers between the LAN and the WAN. However, the Edges also provide the flexibility of meeting a variety of other deployment topologies. For example, SD-WAN Edges can have their interfaces configured to operate as switched ports—allowing the switching of LAN traffic between various LAN interfaces without the need for an external device.
An Edge with its interfaces configured as switched ports is ideal for small office deployments where high throughput is not required, as the additional layer of complexity required to handle traffic switching reduces the overall performance of the system. For most deployments, VMware recommends using all routed interfaces.
- The Edge device's Maximum Throughput is the sum of throughput across all interfaces of the Edge under test.
- Overall traffic is the “aggregate” of all traffic flows going to and from an Edge device.
VMware SD-WAN Edge | 510, 510N | 510-LTE | 520 | 520V | 540 | 610, 610C, 610N | 610-LTE | 620, 620C, 620N |
---|---|---|---|---|---|---|---|---|
Maximum Throughput Large Packet (1300-byte) | ||||||||
Routed Mode All Ports | 350 Mbps | 350 Mbps | 350 Mbps | 350 Mbps | 1 Gbps | 350 Mbps | 350 Mbps | 1.5 Gbps |
Switched Mode All Ports | 200 Mbps | 200 Mbps | 200 Mbps | 200 Mbps | 650 Mbps | 300 Mbps | 300 Mbps | 700 Mbps |
Maximum Throughput Internet Traffic (IMIX) | ||||||||
Routed Mode All Ports | 200 Mbps | 200 Mbps | 200 Mbps | 200 Mbps | 500 Mbps | 200 Mbps | 200 Mbps | 750 Mbps |
Switched Mode All Ports | 80 Mbps | 80 Mbps | 80 Mbps | 80 Mbps | 200 Mbps | 150 Mbps | 150 Mbps | 250 Mbps |
Other Scale Vectors | ||||||||
Maximum Tunnel Scale | 50 | 50 | 50 | 50 | 100 | 50 | 50 | 100 |
Flows Per Second | 2,400 | 2,400 | 2,400 | 2,400 | 4,800 | 2,400 | 2,400 | 4,800 |
Maximum Concurrent Flows | 240K | 240K | 240K | 240K | 480K | 240K | 240K | 480K |
Maximum Number of Routes | 100K | 100K | 100K | 100K | 100K | 100K | 100K | 100K |
Maximum Number of Segments | 128 | 128 | 128 | 128 | 128 | 128 | 128 | 128 |
Maximum Number of NAT Entries | 80K | 80K | 80K | 80K | 150K | 80K | 80K | 150K |
VMware SD-WAN Edge | 640, 640C, 640N | 680, 680C, 680N | 840 | 2000 | 3400, 3400C | 3800, 3800C | 3810 |
---|---|---|---|---|---|---|---|
Maximum Throughput Large Packet (1300-byte) | |||||||
Routed Mode All Ports | 3 Gbps | 6 Gbps | 4 Gbps | 10 Gbps | 7 Gbps | 10 Gbps | 10 Gbps |
Switched Mode All Ports | 1 Gbps | 1 Gbps | 1 Gbps | 1.2 Gbps | 1.2 Gbps | 1.2 Gbps | 1.2 Gbps |
Maximum Throughput Internet Traffic (IMIX) | |||||||
Routed Mode All Ports | 1 Gbps | 2 Gbps | 1.5 Gbps | 5 Gbps | 2.5 Gbps | 5 Gbps | 5 Gbps |
Switched Mode All Ports | 350 Mbps | 350 Mbps | 350 Mbps | 350 Mbps | 900 Mbps | 900 Mbps | 900 Mbps |
Other Scale Vectors | |||||||
Maximum Tunnel Scale | 400 | 800 | 400 | 6,000 | 4,000 | 6,000 | 6,000 |
Flows Per Second | 19,200 | 19,200 | 19,200 | 38,400 | 38,400 | 38,400 | 38,400 |
Maximum Concurrent Flows | 1.9M | 1.9M | 1.9M | 1.9M | 1.9M | 1.9M | 1.9M |
Maximum Number of Routes | 100K | 100K | 100K | 100K | 100K | 100K | 100K |
Maximim Number of Segments | 128 | 128 | 128 | 128 | 128 | 128 | 128 |
Maximum Number of NAT Entries | 650K | 650K | 650K | 960K | 960K | 960K | 960K |
- Large Packet performance is based on a large packet (1300-byte) payload with AES-128 encryption and DPI turned on.
- Internet Traffic (IMIX) performance is based on an average packet size of 417-byte payload with AES-128 encryption and DPI turned on.
Each WAN link (public or private) an Edge uses establishes two tunnels with another Edge or Gateway: one inbound, and one outbound. For example, if an Edge had four WAN links and connected to another Edge with four WAN links, that would be a total of 16 tunnels. In this example, no additional tunnels could be established between the two Edges if an additional WAN link is added to either Edge.
Edge Model | 520V | 620, 620C, 620N | 640, 640C, 640N | 680, 680C, 680N | 840 | 3400, 3400C | 3800, 3800C | 3810 |
---|---|---|---|---|---|---|---|---|
Max. Throughput With FW VNF (1300-byte) | 100 Mbps | 300 Mbps | 600 Mbps | 1 Gbps | 1 Gbps | 2 Gbps | 3 Gbps | 3 Gbps |
Edge Model | 510, 510N | 510-LTE | 520, 520v | 610, 610C, 610N | 610-LTE | 620, 620C, 620N |
---|---|---|---|---|---|---|
Maximum Throughput (IMIX) Across Enhanced HA Link | 90 Mbps | 90 Mbps | 100 Mbps | 200 Mbps | 200 Mbps | 500 Mbps |
Edge Model | 640, 640C, 640N | 680, 680C, 680N | 840 | 2000 | 3400, 3400C | 3800, 3800C | 3810 |
---|---|---|---|---|---|---|---|
Maximum Throughput (IMIX) Across Enhanced HA Link | 800 Mbps | 800 Mbps | 800 Mbps | 800 Mbps | 800 Mbps | 800 Mbps | 800 Mbps |
- There is a performance impact of up to 20% when analytics are enabled.
- Flow capacity is reduced by half when analytics are enabled due to the additional memory and processing required for analysis.
Virtual Edge
Edge Device | Maximum Throughput | Maximum Number of Tunnels | Flows Per Second | Maximum Concurrent Flows | Maximum Number of Routes | Maximum Number of Segments | |
---|---|---|---|---|---|---|---|
ESXi Virtual Edge (2-core, VMXNET3) | 2 Gbps (1300-byte) 800 Mbps (IMIX) |
50 | 2400 | 240K | 35K | 128 | |
KVM Virtual Edge (2-core, Linux Bridge) | 500 Mbps (1300-byte) 200 Mbps (IMIX) |
50 | 2400 | 240K | 35K | 128 | |
KVM Virtual Edge (2-core, SR-IOV) | 1.25 Gbps (1300-byte) 600 Mbps (IMIX) |
50 | 2400 | 240K | 35K | 128 | |
ESXi Virtual Edge (4-core, VMXNET3) | 2 Gbps (1300-byte) 1.5 Gbps (IMIX) |
400 | 19200 | 1.9M | 35K | 128 | |
ESXi Virtual Edge (4-core, SR-IOV) | 2 Gbps (1300-byte) 1.5 Gbps (IMIX) |
400 | 19200 | 1.9M | 35K | 128 | |
KVM Virtual Edge (4-core, Linux Bridge) | 1 Gbps (1300-byte) 350 Mbps (IMIX) |
400 | 4800 | 480K | 35K | 128 | |
KVM Virtual Edge (4-core, SR-IOV) | 2 Gbps (1300-byte) 1 Gbps (IMIX) |
400 | 19200 | 1.9M | 35K | 128 | |
ESXi Virtual Edge (8-core, VMXNET3) | 5 Gbps (1300-byte) 2.5 Gbps (IMIX) |
800 | 38400 | 1.9M | 35K | 128 | |
ESXi Virtual Edge (8-core, SR-IOV) | Version 3.4 or older: 5 Gbps (1300-byte) 2.5 Gbps (IMIX) |
Version 4.0 or newer: 9 Gbps (1300-byte) 4 Gbps (IMIX) |
800 | 38400 | 1.9M | 35K | 128 |
KVM Virtual Edge (8-core, SR-IOV | Version 3.4 or older: 3.5 Gbps (1300-byte) 1 Gbps (IMIX) |
Version 4.0 or newer: 9 Gbps (1300-byte) 3 Gbps (IMIX) |
800 | 38400 | 1.9M | 35K | 128 |
2 vCPU | 4vCPU | 8vCPU | 10vCPU | |
---|---|---|---|---|
Minimim Memory (DRAM) | 8 GB | 16GB | 32 GB | 32 GB |
Minimum Storage | 8 GB | 8 GB | 16 GB | 16 GB |
Supported Hypervisors | Software Version 3.4 or older:
Software version 4.0 and above:
|
|||
Supported Public Cloud | AWS, Azure, GCP, and Alibaba | |||
Support Network I/O | SR-IOV, VirtIO, VMXNET3 | |||
Recommended Host Settings | CPUs at 2.0 GHz or higher CPU Instruction set:
Hyper-threading disabled |
Public Cloud
AWS Instance Type | c5.large | c5.xlarge | c5.2xlarge |
---|---|---|---|
Maximum Throughput | 100 Mbps (1300-byte) 50 Mbps (IMIX) |
200 Mbps (1300-byte) 100 Mbps (IMIX) |
7 Gbps (1300-byte) 2.4 Gbps (IMIX) |
Maximum Tunnels | 50 | 400 | 800 |
Flows Per Second | 1,200 | 2,400 | 4,800 |
Maximum Concurrent Flows | 125,000 | 250,000 | 550,000 |
Maximum Number of Routes | 35,000 | 35,000 | 35,000 |
Maximum Number of Segments | 128 | 128 | 128 |
Azure VM Series | D2d v4 | D4d v4 | D8d v4 |
---|---|---|---|
Maximum Throughput | 100 Mbps (1300-byte) 50 Mbps (IMIX) |
200 Mbps (1300-byte) 100 Mbps (IMIX) |
1 Gbps (1300-byte) 450 Mbps (IMIX) |
Maximum Tunnels | 50 | 400 | 800 |
Flows Per Second | 1,200 | 2,400 | 4,800 |
Maximum Concurrent Flows | 125,000 | 250,000 | 550,000 |
Maximum Number of Routes | 35,000 | 35,000 | 35,000 |
Maximum Number of Segments | 128 | 128 | 128 |
Use of DPDK on VMware SD-WAN Edges
To improve packet throughput performance, VMware SD-WAN Edges take advantage of Data Plane Development Kit (DPDK) technology. DPDK is a set of data plane libraries and drivers provided by Intel for offloading TCP packet processing from the operating system kernel to processes running in user space and results in higher packet throughput. For more details, see https://www.dpdk.org/.
Edge hardware models 620 and higher and all virtual Edges use DPDK by default on their routed interfaces. Edges do not use DPDK on their switched interfaces. A user cannot activate or deactivate DPDK for an Edge interface.