The custom mode is used to configure the TCP Proxy Settings manually. When the TCP proxy profile is set to custom, administrators can use the NSX Advanced Load Balancer UI, CLI, or REST API to alter the TCP proxy profile default parameters described in the following section.

Timeout Parameters

Idle Connections - specified by the Idle Duration parameter, NSX Advanced Load Balancer terminates the connection. Any packet sent or received over the connection by the SE, client or server will reset the Idle Duration timer.

  • Select either TCP keepalive or Age Out Idle Connections to control the behavior of the idle connections.

    1. TCP keepalive: Periodically send a keepalive packet to the client that will reset the idle duration timer on successful client acknowledgment. The keepalive packet sent from the SE does not reset the timer.

    2. Age Out Idle Connections: Terminates the idle connections that have no keep-alive signal from the client, as specified by the Duration field. The NSX Advanced Load Balancer does not send out keepalives, though it still honors keepalive packets received from clients or servers.

  • Enter the Idle Duration in seconds (between 5-14400 seconds, or a 0 for an infinite timeout). This is the time before the TCP connection is eligible to be proactively closed by NSX Advanced Load Balancer. The timer resets when any packet is sent or received by the client, server, or SE.

Note:

  • Setting this value higher can be appropriate for long-lived connections that do not use keepalive packets. Higher settings can also increase the vulnerability of NSX Advanced Load Balancer to denial of service attacks, as the system will not proactively close out idle connections.

  • The Default value for Idle Duration is 600 seconds. The Range is 5-3600 seconds. (0 seconds for infinite timeout and deactivating proactive closing of idle connections).

  • Ignore Wait Time - When a connection between a SE and a client or the SE and a server is closed, the unique client or server IP: Port + service engine IP: port (called a 4-tuple) is placed in a TIME_WAIT state for some time. This 4-tuple cannot be reused till it is determinedthat there are no more delayed packets on the network that are still in flight or that are yet to be delivered. The Time Wait value defines the timeout period before this 4-tuple can be reused. Enter a value between 500 – 2000 ms or enable the Ignore Time Wait option to allow NSX Advanced Load Balancer to immediately reopen the 4-tuple connection, if it receives an SYN packet from the remote IP that matches the same 4-tuple. Default value is 2000 ms.

Retransmission Behavior Parameters

  1. Max Retransmissions - Enter a value between 3-8. It is the number of attempts at re-transmitting before closing the connection. The default value is 8.

  2. Max SYN Retransmissions - Enter a value between 3-8. It is the maximum number of attempts at re-transmitting an SYN packet before giving up. The default value is 8.

Buffer Management Parameters

1.Receive Windowinforms the sender how much data the NSX Advanced Load Balancer can buffer before sending a TCP acknowledgment. It can be a value in the range between 32 KBand 65536 KB.

2. Max Segment Size (MSS) is calculated by using the Maximum Transmission Unit (MTU) length for a network interface. The MSS determines the largest size of data that can be safely inserted into a TCP packet.

In some environments, the MSS must be smaller than the MTU. For example, traffic between the NSX Advanced Load Balancer and a client that is traversing a site-to-site VPN might require some space reserved for padding with encryption data. Click Use Network Interface MTU for Size to set the MSS based on the MTU size of the network interface. The MSS is set to MTU - 40 bytes to account for the IP and TCP headers. For an MTU of 1500 bytes, the MSS is set to 1460.

Alternatively, you can enter a custom value in the range 512–9000 bytes.

QoS & Traffic Engineering Parameters

Differentiated Service Code Point (DSCP) allows NSX Advanced Load Balancer to either pass an existing differentiated services code point parameter or specify a custom number. DSCP is an 8-bit field in the TCP header that can be used for classifying traffic.

The following parameters can be configured through the NSX Advanced Load Balancer CLI and REST API.

Congestion Control Parameters

  1. Aggressive Congestion Avoidance - Congestion window defines the amount of data a sender can reliably transmit without an ACK. The congestion window size keeps increasing up to the maximum receive window, or until the network reaches its congestion limit. In networks where there are no transmissions or timeouts observed, the NSX Advanced Load Balancer can choose higher initial congestion windows to avoid slow start and ramp up TCP connections faster. The following are the possible values for the field:

    1. Enabled — 10x.

    2. Disabled — 1x the size of the MSS.

    3. Default value is Disabled.

  2. CC Algo is the congestion control algorithm that governs the behavior for identifying and responding to detected network congestion. The following are the possible values for the field:

    1. New Reno is a versatile TCP congestion control algorithm for most networks.

    2. Cubic is designed for long fat networks (LFN), with high throughput and high latency.

    3. HTCP is recommended only for high-throughput and high-latency networks.

    The default value is New Reno.

  3. Congestion Recovery Scaling Factor defines the congestion window scaling factor after recovery and used with aggressive congestion avoidance. It can be in the range 0-8 and defaults to 2.

  4. Min Rexmt Timeout is in the range 50-5000 ms. TCP has built-in logic for ensuring that packets are received by the remote device, failing which the sender re-transmits the packets. This parameter sets the minimum time to wait before re-transmitting a packet.

  5. Reassembly Queue Size defines the size of the buffer used to reassemble TCP segments when the segments have arrived out of order, i.e. the maximum number of TCP segments that can be queued for reassembly. Lower values might lead to issues in downloading large content or handling bulk traffic. The value can be between 0-5k. Default value is 0 (provides unlimited queue size).

  6. Reorder Threshold controls the number of duplicate ACks required to trigger a re-transmission. A higher value means less number of re-transmissions caused by packet reordering. If out-of-order packets are common in the environment, it is advisable to use a higher number. The value can be in between 1-100. The default value is 8 for public clouds (For example, AWS, Azure, GCP) and 3 for others.

  7. Slow Start Scaling Factor is the congestion window scaling factor during slow start. It is different from the window scaling factor. This parameter is in effect only when aggressive congestion avoidance is enabled. The field value can be between 0-8. The default value is 1.

  8. Time Wait Delay is the time to wait before closing a connection in the TIME_WAIT state. The field can take the following values:

    1. Range — 500-2000ms.

    2. Special — 0 (for immediate close).

    3. Default — 0.

There are a few more optimization parameters that are enabled by default in the NSX Advanced Load Balancer TCP stack that cannot be changed by users. These parameters are described in the following section.

Unalterable Parameters

  1. Window Scaling Factor - Window scaling determines the amount of TCP data the receiver ( the SE) can buffer for a connection. The default initial window is 65535 bits. For modern TCP clients supporting this TCP extension, the window scaling factor increases this number significantly by doubling the window size by x number of times (where x is the scale factor). This is helpful for networks with high latency and high throughput, which describes most broadband Internet connections. The NSX Advanced Load Balancer window scale factor is 10, which implies that it can buffer up to 67,104,840 bits. The default value for the field is 10 (which means a buffer of up to 67,104,840 bits when the receive window is set to 65535).

  2. Selective ACK - With selective acknowledgments, the data receiver can inform the sender about all segments that have arrived successfully. So the sender needs to only re-transmit the segments that have actually been lost. Consider the scenario where the first five packets are successfully received, the sixth packet is lost and is not yet received and the packets seven to ten are successfully received. In this case, without SACK, the sender would re-transmit all packets starting from packet six, since it cannot figure out which packets were actually lost. This would lead to unnecessary re-transmits, further consuming bandwidth and impacting TCP performance. The default value for this field is Enabled.

  3. Limited Transmit Recovery - This parameter is used to more effectively recover lost segments when the congestion window of a connection is small, or when several segments are lost in a single transmission window. The limited transmit algorithm allows sending a new data segment in response to each of the first two duplicate acknowledgments that arrive at the sender. Transmitting these segments increases the probability that TCP can recover from a single lost segment using the fast re-transmit algorithm, instead of using a costly re-transmission timeout. The value for this field is Enabled.

  4. Delayed ACK - Instead of sending one ACK segment per data segment received, the NSX Advanced Load Balancer can improve efficiency by sending delayed ACKs. This is part of TCP congestion control. As per RFC, timestamp to delay ACK is less than 0.5 seconds, and in a stream of full-sized segments, an ACK would be available for at least every second segment.

Configuring MTU using the CLI

The Maximum Transmission Unit (MTU) can be configured as a global property, which sets the MTU across all SEs managed by the Controller cluster. By default, the MTU is learned using DHCP. This can be manually set using the CLI. The following command sets the MTU to 1500 bytes. Two examples illustrate the need to change MTU from the default:

  • If the installation is in an environment using VXLAN or some other type of overlay network (for example, OpenStack), the MTU must be reduced to accommodate the additional tunnel headers.

  • If the DHCP option sets the MTU to 9000 (jumbo), but the entire infrastructure (switches and routers) does not support jumbo MTU. It can happen in AWS environments.

configure serviceengineproperties
se_runtime_properties
global_mtu 1500
Overwriting the previously entered value for global_mtu
save
save
Note:

NSX Advanced Load Balancer SEs support MTU a maximum of 1500 bytes.

Protection from TCP Attacks

Apart from performance tuning parameters of TCP, the NSX Advanced Load Balancer also has in-built mechanisms to protect itself from some common TCP level attacks as explained in the following section. This list is not exhaustive and includes some common attacks. For more information, see DDoS Attack Mitigation.

Attack Type

Description

Mitigation

SYN flood

A form of denial-of-service attack in which an attacker sends a succession of SYN requests to a target system without acknowledging the SYN ACKs. This is done in an attempt to consume enough server resources and make the system unresponsive to legitimate traffic.

The NSX Advanced Load Balancer starts sending SYN cookies by default if the TCP table has half-open connections. There is currently no configuration to allow specific clients from this behavior. In a TCP fast path profile where there is no TCP proxy-ing, SYN protection can be enabled, causing the NSX Advanced Load Balancer to delay establishing a TCP session with the server, until a complete three-way handshake with the client has taken place. This protects the server from SYN flood or half-open states.

LAND attacks

This acts like a SYN flood attack. The difference is that the source and destination IP addresses are identical, which makes the IP stack process the same packet over and over again, potentially leading to a crash of the victimized system.

When this attack is detected, the NSX Advanced Load Balancer drops the packets at the dispatcher layer.

Port scan

An attacker launches a port scan by sending TCP packets on various ports to find out listening ports for next level of attacks. Most of these ports are non-listening ports.

When this attack is detected, the NSX Advanced Load Balancer drops the packets at the dispatcher layer.