Configure the initial networking on ESXi hosts to prepare for the vSAN Ready Node cluster.

Procedure

  1. At the ESXi console, press F2, log in as root and enable the ESXi shell and SSH access under Troubleshooting Options.
  2. Exit the ESXi console and log in as root to the ESXi shell (ALT-F1).
  3. Determine the name of the two 10 GBASE NICs to use by checking the link status with ESXCLI.

    The following ESXCLI command output is condensed. [root@localhost:~] esxcli network nic list

    Name   PCI Device   Driver Admin Status Link Status Speed Duplex MAC Address
    ------ ------------ ------ ------------ ----------- ----- ------ ----------------
    vmnic4 0000:03:00.0 ixgbe  Up           Up          10000 Full   a0:36:9f:73:00:f8
    vmnic5 0000:03:00.1 ixgbe  Up           Up          10000 Full   a0:36:9f:73:00:fa
    
  4. Determine the uplink on vSwitch0, and, if necessary, replace the current uplink on vSwitch0 with an active vmnic.

    [root@localhost:~] esxcli network vswitch standard list | grep Uplinks

    Uplinks: vmnic0

    [root@localhost:~] esxcli network vswitch standard uplink add --uplink-name=vmnic4 --vswitch-name=vSwitch0

    [root@localhost:~] esxcli network vswitch standard uplink remove --uplink-name=vmnic0 --vswitch-name=vSwitch0

    [root@localhost:~] esxcli network vswitch standard list | grep Uplinks

    Uplinks: vmnic4

  5. Set vmk0 to use the static IP address for esxi-01 and set the default VMkernel gateway.

    [root@localhost:~] esxcli network ip interface ipv4 set --interface-name=vmk0 --ipv4=172.16.0.11 --netmask=255.255.255.0 --type=static

    [root@localhost:~] esxcli network ip interface ipv4 get

    Name  IPv4 Address  IPv4 Netmask   IPv4 Broadcast  Address Type  DHCP DNS
    ----  ------------  -------------  --------------  ------------  --------
    vmk0  172.16.0.11   255.255.255.0  172.16.0.255     STATIC           false
    
    

    [root@localhost:~] esxcli network ip route ipv4 add --gateway=172.16.0.1 --network=default

    [root@localhost:~] esxcli network ip route ipv4 list

    Network    Netmask        Gateway    Interface  Source
    ---------  -------------  ---------  ---------  ------
    default    0.0.0.0        172.16.0.1 vmk0       MANUAL
    172.16.0.0 255.255.255.0  0.0.0.0    vmk0       MANUAL
  6. Set the VLAN ID on the Management Network and VM Network port groups.

    [root@localhost:~] esxcli network vswitch standard portgroup set --portgroup-name="Management Network" --vlan-id=1000

    [root@localhost:~] esxcli network vswitch standard portgroup set --portgroup-name="VM Network" --vlan-id=1000

    [root@localhost:~] esxcli network vswitch standard portgroup list

    Name                Virtual Switch  Active Clients  VLAN ID
    ------------------  --------------  --------------  -------
    Management Network  vSwitch0                     1     1000
    VM Network          vSwitch0                     0     1000
    
  7. Connect to the host remotely with SSH and log in as root.
  8. Set the hostname and domain name of the system and configure DNS.

    [root@localhost:~] esxcli system hostname set --host=esxi-01 --domain=vrn.local

    [root@esxi-01:~] esxcli system hostname get

    Domain Name: vrn.local
    Fully Qualified Domain Name: esxi-01.vrn.local
    Host Name: esxi-01
    

    [root@esxi-01:~] esxcli network ip dns server add --server=172.16.1.3

    [root@esxi-01:~] esxcli network ip dns server list

    DNSServers: 172.16.1.3

    [root@esxi-01:~] esxcli network ip dns search add --domain=vrn.local

    [root@esxi-01:~] esxcli network ip dns search list

    DNSSearch Domains: vrn.local

  9. Configure and enable the NTP daemon, allow it a moment to start, and verify synchronization.

    [root@esxi-01:~] echo "server 172.16.1.3" >> /etc/ntp.conf

    [root@esxi-01:~] cat /etc/ntp.conf

    restrict 127.0.0.1
    restrict default kod nomodify notrap
    driftfile /etc/ntp.drift
    server 172.16.1.3
    
    

    [root@esxi-01:~] esxcli network firewall ruleset set --ruleset-id=ntpClient --enabled=true

    [root@esxi-01:~] chkconfig ntpd on

    [root@esxi-01:~] /etc/init.d/ntpd start

    Starting ntpd

    [root@esxi-01:~] ntpq -p

     remote      refid               st t when poll reach   delay   offset  jitter
    ==============================================================================
    *vrn-dc1.vrn 172.16.100.100       4 u   18   64    3    2.425    0.089   7.999
  10. Increase the number of hosts allowed in the vSAN cluster to a maximum of 64.

    [root@esxi-01:~] esxcli system settings advanced set --option=/VSAN/goto11 --int-value=1

    [root@esxi-01:~] esxcli system settings advanced list --option=/VSAN/goto11

    Path: /VSAN/goto11
    Type: integer
    Int Value: 1
    Default Int Value: 0
    Min Value: 0
    Max Value: 1
    String Value:
    Default String Value:
    Valid Characters:
    Description: Allow VSAN clusters to grow up to 64 nodes. (requires reboot)
    
  11. Set TCP/IP heap size to 1024 MiB.

    [root@esxi-01:~] esxcli system settings advanced set --option=/Net/TcpipHeapMax --int-value=1024

    [root@esxi-01:~] esxcli system settings advanced list --option=/Net/TcpipHeapMax

    Path: /Net/TcpipHeapMax
    Type: integer
    Int Value: 1024
    Default Int Value: 512
    Min Value: 32
    Max Value: 1536
    String Value:
    Default String Value:
    Valid Characters:
    Description: Max megabytes the tcpip module heap can grow to. (REQUIRES REBOOT!)
    
  12. Verify that CBRC cache size is 2048 MiB.

    [root@esxi-01:~] esxcli system settings advanced list --option=/CBRC/DCacheSize

    Path: /CBRC/DCacheSize
    Type: integer
    Int Value: 2048
    Default Int Value: 2048
    Min Value: 2048
    Max Value: 2048
    String Value:
    Default String Value:
    Valid Characters:
    Description: Size of CBRC Data Cache in MB. This cannot be changed if CBRC.Enable is set to 1.
    
  13. Set CBRC cache mem reserved to 1024 MiB.

    [root@esxi-01:~] esxcli system settings advanced set --option=/CBRC/DCacheMemReserved --int-value=1024

    [root@esxi-01:~] esxcli system settings advanced list --option=/CBRC/DCacheMemReserved

    Path: /CBRC/DCacheMemReserved
    Type: integer
    Int Value: 1024
    Default Int Value: 400
    Min Value: 100
    Max Value: 2048
    String Value:
    Default String Value:
    Valid Characters:
    Description: Memory consumed by CBRC Data Cache (in MB)
  14. If the SAS HBA of the system has a driver requirement, firmware requirement, or both for vSAN use, perform the updates according to the manufacturer's documentation.
  15. Enter maintenance mode, reboot the system, log in as root over SSH, and exit maintenance mode.

    [root@esxi-01:~] esxcli system maintenanceMode set --enable true

    [root@esxi-01:~] esxcli system shutdown reboot --reason=reasons

    ...

    [root@esxi-01:~] esxcli system maintenanceMode set --enable false