Following networking security best practices helps ensure the integrity of your vSphere deployment.
General vSphere Networking Security Recommendations
Following general network security recommendations is the first step in securing your vSphere networking environment. You can then move on to special areas, such as securing the network with firewalls or using IPsec.
Recommendations to Secure a vSphere Networking Environment
- Spanning Tree Protocol (STP) detects and prevents loops from forming in the network topology. VMware virtual switches prevent loops in other ways, but do not support STP directly. When network topology changes occur, some time is required (30–50 seconds) while the network relearns the topology. During that time, no traffic is allowed to pass. To avoid these problems, network vendors have created features for switch ports to continue forwarding traffic. For more information, see the VMware knowledge base article at https://kb.vmware.com/s/article/1003804. Consult your network vendor documentation for the proper network and networking hardware configurations.
- Ensure that Netflow traffic for a Distributed Virtual Switch is only sent to authorized collector IP addresses. Netflow exports are not encrypted and can contain information about the virtual network. This information increases the potential for sensitive information to be viewed and captured in transit by attackers. If Netflow export is required, verify that all Netflow target IP addresses are correct.
- Ensure that only authorized administrators have access to virtual networking components by using the role-based access controls. For example, give virtual machine administrators only access to port groups in which their virtual machines reside. Give network administrators access to all virtual networking components but no access to virtual machines. Limiting access reduces the risk of misconfiguration, whether accidental or malicious, and enforces key security concepts of separation of duties and least privilege.
- Ensure that port groups are not configured to the value of the native VLAN. Physical switches are often configured with a native VLAN, and that native VLAN is often VLAN 1 by default. ESXi does not have a native VLAN. Frames with VLAN specified in the port group have a tag, but frames with VLAN not specified in the port group are not tagged. This can cause a problem because virtual machines that are tagged with a 1 end up belonging to native VLAN of the physical switch.
For example, frames on VLAN 1 from a Cisco physical switch are untagged because VLAN 1 is the native VLAN on that physical switch. However, frames from the ESXi host that are specified as VLAN 1 are tagged with a 1. As a result, traffic from the ESXi host that is destined for the native VLAN is not routed correctly because it is tagged with a 1 instead of being untagged. Traffic from the physical switch that is coming from the native VLAN is not visible because it is not tagged. If the ESXi virtual switch port group uses the native VLAN ID, traffic from virtual machines on that port is not visible to the native VLAN on the switch because the switch is expecting untagged traffic.
- Ensure that port groups are not configured to VLAN values reserved by upstream physical switches. Physical switches reserve certain VLAN IDs for internal purposes and often disallow traffic configured to these values. For example, Cisco Catalyst switches typically reserve VLANs 1001–1024 and 4094. Using a reserved VLAN might result in a denial of service on the network.
- Ensure that port groups are not configured to VLAN 4095 except for Virtual Guest Tagging (VGT). Setting a port group to VLAN 4095 activates VGT mode. In this mode, the virtual switch passes all network frames to the virtual machine without modifying the VLAN tags, leaving it to the virtual machine to deal with them.
- Restrict port-level configuration overrides on a distributed virtual switch. Port-level configuration overrides are deactivated by default. When overrides are activated, you can use different security settings for a virtual machine than the port-group level settings. Certain virtual machines require unique configurations, but monitoring is essential. If overrides are not monitored, anyone who gains access to a virtual machine with a less secure distributed virtual switch configuration might attempt to exploit that access.
- Ensure that distributed virtual switch port mirror traffic is sent only to authorized collector ports or VLANs. A vSphere Distributed Switch can mirror traffic from one port to another to allow packet capture devices to collect specific traffic flows. Port mirroring sends a copy of all specified traffic in unencrypted format. This mirrored traffic contains the full data in the packets captured and can result in total compromise of that data if misdirected. If port mirroring is required, verify that all port mirror destination VLAN, port, and uplink IDs are correct.
Labeling vSphere Networking Components
Identifying the different components of your vSphere networking architecture is critical and helps ensure that no errors are introduced as your network expands.
Follow these best practices:
- Ensure that port groups are configured with a clear network label. These labels serve as a functional descriptor for the port group and help you identify each port group's function as the network becomes more complex.
- Ensure that each vSphere Distributed Switch has a clear network label that indicates the function or IP subnet of the switch. This label serves as a functional descriptor for the switch, just as physical switches require a host name. For example, you can label the switch as internal to show that it is for internal networking. You cannot change the label for a standard virtual switch.
Document and Check the vSphere VLAN Environment
Check your VLAN environment regularly to avoid addressing problems. Fully document the VLAN environment and ensure that VLAN IDs are used only once. Your documentation can help with troubleshooting and is essential when you want to expand the environment.
Procedure
Adopting Network Isolation Practices in vSphere
Network isolation practices bolster network security in your vSphere environment.
Isolate the vSphere Management Network
The vSphere management network provides access to the vSphere management interface on each component. Services running on the management interface provide an opportunity for an attacker to gain privileged access to the systems. Remote attacks are likely to begin with gaining access to this network. If an attacker gains access to the management network, it provides the staging ground for further intrusion.
Strictly control access to management network by protecting it at the security level of the most secure VM running on an ESXi host or cluster. No matter how the management network is restricted, administrators must have access to this network to configure the ESXi hosts and vCenter Server system.
Place the vSphere management port group in a dedicated VLAN on a common standard switch. Production (VM) traffic can share the standard switch if the vSphere management port group's VLAN is not used by production VMs.
Check that the network segment is not routed, except to networks where other management-related entities are found. Routing a network segment might make sense for vSphere Replication. In particular, make sure that production VM traffic cannot be routed to this network.
- To access the management network in especially sensitive environments, configure a controlled gateway or other controlled method. For example, require that administrators connect to the management network through a VPN. Allow access to the management network only to trusted administrators.
- Configure bastion hosts that run management clients.
Isolate Storage Traffic
Ensure that IP-based storage traffic is isolated. IP-based storage includes iSCSI and NFS. Virtual machines might share virtual switches and VLANs with the IP-based storage configurations. This type of configuration might expose IP-based storage traffic to unauthorized virtual machine users.
IP-based storage frequently is not encrypted. Anyone with access to this network can view IP-based storage traffic. To restrict unauthorized users from viewing IP-based storage traffic, logically separate the IP-based storage network traffic from the production traffic. Configure the IP-based storage adapters on separate VLANs or network segments from the VMkernel management network to limit unauthorized users from viewing the traffic.
Isolate vMotion Traffic
vMotion migration information is transmitted in plain text. Anyone with access to the network over which this information flows can view it. Potential attackers can intercept vMotion traffic to obtain the memory contents of a VM. They might also stage an MITM attack in which the contents are modified during migration.
Separate vMotion traffic from production traffic on an isolated network. Set up the network to be nonroutable, that is, make sure that no layer-3 router is spanning this and other networks, to prevent outside access to the network.
Use a dedicated VLAN on a common standard switch for the vMotion port group. Production (VM) traffic can use the same standard switch if the vMotion port group’s VLAN is not used by production virtual machines.
Isolate vSAN Traffic
When configuring your vSAN network, isolate vSAN traffic on its own Layer 2 network segment. You can perform this isolation by using dedicated switches or ports, or by using a VLAN.
Use Virtual Switches with the vSphere Network Appliance API Only If Required
Do not configure your host to send network information to a virtual machine unless you are using products that use the vSphere Network Appliance API (DvFilter). If the vSphere Network Appliance API is enabled, an attacker might attempt to connect a virtual machine to the filter. This connection might provide access to the network of other virtual machines on the host.
If you are using a product that uses this API, verify that the host is configured correctly. See the sections on DvFilter in the Developing and Deploying vSphere Solutions, vServices, and ESX Agents documentation. If your host is set up to use the API, make sure that the value of the Net.DVFilterBindIpAddress parameter matches the product that uses the API.
