Setting Up VMware vSphere Network

VMware Greenplum on VMware vSphere uses the three networks described below.

Internal network

The internal network is identified by the port group gp-virtual-internal. It is used by Greenplum for internal communications. It requires a new VLAN ID and a number of non-routable contiguous static IP addresses that will be used for:
- Greenplum Database coordinator and segment hosts
Determine the number of required IP addresses based on the number of coordinator hosts and segment hosts in your environment. For example, for a four ESXi host configuration, it requires 67 non-routable contiguous static IP addresses.
External network

The external network is identified by the port group gp-virtual-external. It is used by Greenplum for external traffic through the coordinator and standby coordinator nodes. It requires a new VLAN ID and 4 routable static IP addresses that will be used for:
- Greenplum Coordinator
- Greenplum Standby
- Greenplum Virtual Appliance
- Greenplum Base template
Load Backup and Restore network

The load backup and restore network is identified by the port group gp-virtual-etl-bar. It is used by Greenplum for ETL traffic and for backup and restore traffic. It requires a new VLAN ID and a number of non-routable contiguous static IP addresses that will be used for:
- Greenplum Database coordinator and segment hosts
Determine the number of required IP addresses based on the number of coordinator hosts and segment hosts in your environment. For example, for a four ESXi host configuration, it requires 67 non-routable contiguous static IP addresses.

Note that ETL or backup traffic must take place within the internal network gp-virtual-etl-bar which is non-routable.

Verifying the Distributed Virtual Switch Settings

Check the Distributed Virtual Switch settings to make sure that the Maximum Transmission Unit (MTU) is set to 9000 Bytes:

On the VMware vSphere Client Home page, click Networking and navigate to your distributed switch.
Navigate to Configure → Properties → Advanced.
Verify that MTU is set to 9000 Bytes. If it is set to any other value, edit the value and set it to 9000 Bytes, and click OK.

Creating the Distributed Port Groups

The table below summarizes the distributed port groups that the Virtual Distributed Switch must have configured. Some of them might already be present in your VMware vSphere environment.

Port Group Name	Description
gp‑virtual‑internal	Used for Greenplum cluster internal communications, including interconnect, dispatch and mirroring. This network is usually air-gapped, and does not have internet connection. All IP addresses are statically assigned.
gp‑virtual‑external	Used for Greenplum cluster user connections from outside the cluster. The Greenplum coordinator host exposes `$PGPORT` on this network. It is also used to connect to the coordinator host and standby coordinator host for DBA or troubleshooting purposes. This network usually has DHCP enabled.
gp‑virtual‑etl‑bar	Used for ETL and backup/restore operations for the Greenplum cluster.
VMware vSphere‑management	Used by vCenter to manage the ESXi hosts. It supports the VMkernel port of management.
VMware vSphere‑vmotion	Used for VMware vSphere HA and DRS. It supports the VMkernel port of vMotion.
vsphere‑vsan	Used for vSAN connections. It supports the VMkernel port of vSAN.
vsphere‑vcenter	Dedicated management network, including vCenter, DHCP, DNS, and NTP services. Usually the IP addresses are statically assigned.
vsphere‑vm	Used by non Greenplum virtual machines to connect to the company network. It usually has internet connectivity. This network usually has DHCP enabled.

Create the above port groups in your VMware vSphere environment:

On the VMware vSphere Client Home page, click Networking and navigate to the distributed switch for this environment.
Click on the arrow next to your distributed switch to view the list of existing port groups. Skip the creation of any port groups listed above that already exist.
Right-click the distributed switch and select Distributed port group > New distributed port group.
Specify the name of the new distributed port group (use the table above), then click Next.
On the Configure settings page, make sure you select VLAN as the VLAN type, then select the appropriate VLAN ID for each port group. You should have this information provided by your network administrator.
Click Next, confirm your settings and click Finish.

Configuring the Distributed Port Groups

Principles of VMware vSphere Distributed Switch to Uplink Assignment

When creating and configuring the distributed port groups listed above, you must adhere to the following principles:

Always use active/standby uplinks to optimize the performance by avoiding a single point of failure.
Always separate the active/standby uplinks across two different NICs and two different switches to ensure high availability.
The two uplinks of the active/standby pair must connect to two physical switches which must be interlinked to ensure high availability.
The port group vsphere-vsan must be always on its own active uplink.
The port group gp-virtual-internal must be always on its own active uplink.
The port group gp-virtual-etl-bar must be always on its own active uplink.
Multiple standby uplinks can be overlapped with the dedicate active uplink. However, there is a risk of degraded performance if the active link goes offline.

Based on the principles above, and depending on your Dell EMC VxRail topology, the configuration of the port groups may vary. The next section describes the recommended topology of Dell EMC VxRail and documents how the port groups must be configured for this particular topology.

Using Four 100GbE Links

This Dell EMC VxRail topology consists of Dell EMC VxRail nodes with two 100GbE Network Cards each with two 100GbE ports. Two ports are connected to two TOR switches, and one optional connection to management switch for Integrated Dell Remote Access Controller (iDRAC). You can find more information about this topology on page 109 of the Dell EMC VxRail Network Planning Guide (Figure 53).

Configure the port groups in your VMware vSphere environment:

On the VMware vSphere Client Home page, click Networking and navigate to your distributed switch.
Right-click the distributed port and select Edit settings.
On the Teaming and failover page, under Failover order, add the corresponding uplink.
Click OK.

You must configure the following port groups on the Virtual Distributed Switch to match the following table:

Port Group Name	uplink1 (vmnic0)	uplink2 (vmnic1)	uplink3 (vmnic2)	uplink4 (vmnic3)
vsphere-management	Active	Unused	Unused	Standby
vsphere-vmotion	Standby	Unused	Unused	Active
vsphere-vm	Standby	Unused	Unused	Active
vsphere-vcenter	Standby	Unused	Unused	Active
vsphere-vsan	Unused	Active	Standby	Unused
gp-virtual-internal	Unused	Standby	Active	Unused
gp-virtual-external	Active	Unused	Unused	Standby
gp-virtual-etl-bar	Standby	Unused	Unused	Active

For example, the Failover order configuration for port group gp-virtual-internal should look like this:

Enabling VMware vSphere Distributed Switch (vDS) Health Check

Since the distributed switch and the physical switches are configured separately, settings such as the MTU, VLAN, and teaming settings might present configuration discrepancies. To help narrow down any network configuration issues, VMware vSphere provides a vDS Health Check to detect network configuration inconsistencies between the distributed switch and physical switches. Please refer to the VMware vSphere Documentation for more details.

To enable the VMware vSphere Distributed Switch Health Check from vCenter:

Click Menu → Networking.
Select your distributed switch.
Click Configure → Settings → Health Check.
Click the Edit button on the right pane.
For VLAN and MTU, select Enabled and leave Interval as default (1 minute).
For Teaming and Failover, select Enabled and leave Interval as default (1 minute).
Click OK.

There are some known limitations of Health Check:

It does not check the LAG ports.
It could cause network performance degradation if you have many uplinks, VLANs and hosts.

You may want to enable vDS Health Check only during deployment and testing phases, and turn it back off once you have verified that there are no configuration inconsistencies.

Next Steps

Continue configuring Dell EMC VxRail with VMware vSphere by Setting Up vSphere DRS and HA.