NSX prepares the DVS selected by the user for VXLAN.

This requires NSX to create a DVPortgroup on the DVS for VTEP vmknics to use.

The teaming, load balancing method, MTU, and VLAN ID is chosen during VXLAN configuration. The teaming and load balancing methods must match the configuration of the DVS selected for the VXLAN.

The MTU must be set to be at least 1600 and not less than what is already configured on the DVS.

The number of VTEPs created depends on the teaming policy selected and the DVS configuration.

Common Issues During VXLAN Preparation

During the configuration of VXLAN, the typical kinds of issues that can be encountered are as follows:

  • Teaming method chosen for VXLAN does not match what can be supported by the DVS. See the VMware NSX for vSphere Network Virtualization Design Guide at https://communities.vmware.com/docs/DOC-27683.

  • Incorrect VLAN ID chosen for the VTEPs.

  • DHCP selected to assign VTEP IP addresses, but no DHCP server is available.

  • A vmknic is missing “force-Sync” the configuration.

  • A vmknic has a bad IP address.

Important Port Numbers

The VXLAN UDP port is used for UDP encapsulation. Prior to NSX 6.2.3, the default VXLAN port number was 8472. In NSX 6.2.3 the default VXLAN port number changed to 4789 for new installs. In NSX 6.2 and later installations that use a hardware VTEP, you must use VXLAN port number 4789. For information on changing the VXLAN port configuration, see "Change VXLAN Port" in the NSX Administration Guide.

Port 80 must be open from NSX Manager to the hosts. This is used to download the VIB/agent.

Port 443/TCP from, to, and among the ESXi hosts, and the vCenter Server.

Additionally, the following ports must be open on NSX Manager:

  • 443/TCP: Required for downloading the OVA file on the ESXi host for deployment, for using REST APIs, and for the NSX Manager user interface.

  • 80/TCP: Required for initiating a connection to the vSphere SDK and for messaging between NSX Manager and NSX host modules.

  • 1234/TCP: Requred for communication between ESXi Host and NSX Controller Clusters.

  • 5671: Required for Rabbit MQ (a messaging bus technology).

  • 22/TCP: Required for console access (SSH) to the CLI. By default, this port is closed.

If the hosts in your clusters were upgraded from vCenter Server version 5.0 to 5.5, you must open ports 80 and 443 on those hosts for Guest Introspection installation to be successful.

Control plane status displays as disabled if the host does not have any active VMs which need a controller connection

Use the show logical-switch commands to view VXLAN details on the host. For details, refer to NSX Command Line Interface Reference.

The show logical-switch host hostID verbose command will display status of control plane as disabled if the host has not been populated with any VMs which require a connection to the controller cluster for forwarding table information.

Network count:  18
VXLAN network:  32003
Multicast IP:   0.0.0.0
Control plane:  Disabled  <<========
MAC entry count:        0
ARP entry count:        0
Port count:     1

Netcpa of a host will not connect to controllers unless there is a need to fetch some information from the controller. In this case the host did not had any VM and that is the reason there was no connection to the controller.

Error while configuring VXLAN gateway

When configuring VXLAN using a static IP pool (at Networking & Security > Installation > Host Preparation > Configure VXLAN and the configuration fails to set an IP pool gateway on the VTEP, the VXLAN configuration status enters the Error (RED) state for the host cluster. The error message is “VXLAN Gateway cannot be set on host” and the error status is “VXLAN_GATEWAY_SETUP_FAILURE”.

In the REST API call, GET https://<nsxmgr-ip>/api/2.0/nwfabric/status?resource=<cluster-moid>, the status of VXLAN is as follows:

<nwFabricFeatureStatus>
<featureId>com.vmware.vshield.nsxmgr.vxlan</featureId>
  <featureVersion>5.5</featureVersion>
  <updateAvailable>false</updateAvailable>
  <status>RED</status>
  <message>VXLAN Gateway cannot be set on host</message>
  <installed>true</installed>
  <enabled>true</enabled>
  <errorStatus>VXLAN_GATEWAY_SETUP_FAILURE</errorStatus>
</nwFabricFeatureStatus>

Workaround: To fix the error, there are two options.

  • Option 1: Remove VXLAN configuration for the host cluster, fix the underlying gateway setup in the IP pool by making sure the gateway is properly configured and reachable, and then reconfigure VXLAN for the host cluster.

  • Option 2: Perform the following steps.

    1. Fix the underlying gateway setup in the IP pool by making sure the gateway is properly configured and reachable.

    2. Put the host (or hosts) into maintenance mode to ensure no VM traffic is active on the host.

    3. Delete the VXLAN VTEPs from the host.

    4. Take the host out of maintenance mode. Taking hosts out of maintenance mode triggers the VXLAN VTEP creation process on NSX Manager. NSX Manager will try to re-create the required VTEPs on the host.