You can install NSX Edge devices in an automated fashion on bare metal or as a VM using PXE. Note that PXE boot installation is not supported for NSX Manager and NSX Controller. This includes automatically configuring networking settings, such as IP address, gateway, network mask, NTP, and DNS.

About this task

This procedure demonstrates how to set up a PXE server on Ubuntu. PXE is made up of several components: DHCP, HTTP, and TFTP.

DHCP dynamically distributes IP settings to NSX-T components, such as NSX Edge. In a PXE environment, the DHCP server allows NSX Edge to request and receive an IP address automatically.

TFTP is a file-transfer protocol. The TFTP server is always listening for PXE clients on the network. When it detects any network PXE client asking for PXE services, it provides the NSX-T component ISO file and the installation settings contained in a preseed file.

After the PXE server is ready, the procedure shows how to install NSX Edge with a preseeded configuration file.

Prerequisites

  • Verify that the system requirements are met. See System Requirements.

  • Verify that the required ports are open. See Ports and Protocols.

  • Plan your IPv4 IP address scheme. In this release of NSX-T, IPv6 is not supported.

  • A PXE server must be available in your deployment environment. The PXE server can be set up on any Linux distribution. The PXE server must have two interfaces, one for external communication and another for providing DHCP IP and TFTP services.

    If you have multiple management networks, you can add static routes to the other networks from the NSX-T appliance.

Procedure

  1. (Optional) : Create a kickstart file to set up a new TFTP or DHCP services on an Ubuntu server.

    A kickstart file is a text file that contains CLI commands that you run on the appliance after the first boot.

    Name the kickstart file based on the PXE server it is pointing to. For example:

    nsxcli.install

    and must be copied to your web server, for example at /var/www/html/nsx-edge/nsxcli.install.

    In the kickstart file, you can add CLI commands.

    For example:

    To configure the IP address of the management interface:

    stop dataplane
    set interface eth0 <ip-cidr-format> plane mgmt
    start dataplane

    To change the admin user password:

    set user admin password <new_password> old-password <old-password>

    Note that if you specify a password in the preseed.cfg file, use the same password in the kickstart file. Otherwise, use the default password, which is "default".

    To join the NSX Edge with the management plane:

    join management-plane <mgr-ip> thumbprint <mgr-thumbprint> username <mgr-username> password <mgr password>
  2. Create two interfaces, one for management and another for DHCP and TFTP services.

    Make sure that the DHCP/TFTP interface is in the same subnet that the NSX Edge will reside in.

    For example, if the NSX Edge management interfaces are going to be in the 192.168.210.0/24 subnet, place eth1 in that same subnet.

    # The loopback network interface
    auto lo
    iface lo inet loopback
    
    # PXE server's management interface
    auto eth0
    iface eth0 inet static
      address 192.168.110.81
      gateway 192.168.110.1
      netmask 255.255.255.0
      dns-nameservers 192.168.110.10
    
    # PXE server's DHCP/TFTP interface
    auto eth1
    iface eth1 inet static
      address 192.168.210.82
      gateway 192.168.210.1
      netmask 255.255.255.0
      dns-nameservers 192.168.110.10
    
  3. Install DHCP server software.
    sudo apt-get install isc-dhcp-server -y
  4. Edit the /etc/default/isc-dhcp-server file, and add the interface that provides DHCP service.
    INTERFACES="eth1"
  5. (Optional) : If you want this DHCP server to be the official DHCP server for the local network, uncomment the authoritative; line in the /etc/dhcp/dhcpd.conf file.
    ...
    authoritative;
    ...
  6. In /etc/dhcp/dhcpd.conf, define the DHCP settings for the PXE network.

    For example:

    subnet 192.168.210.0 netmask 255.255.255.0 {
       range 192.168.210.90 192.168.210.95;
       option subnet-mask 255.255.255.0;
       option domain-name-servers 192.168.110.10;
       option routers 192.168.210.1;
       option broadcast-address 192.168.210.255;
       default-lease-time 600;
       max-lease-time 7200;
    }
  7. Start the DHCP service.
    sudo service isc-dhcp-server start
  8. Verify that the DHCP service is running.
    service --status-all | grep dhcp
  9. Install Apache, TFTP, and other components that are required for PXE booting.
    sudo apt-get install apache2 tftpd-hpa inetutils-inetd
  10. Verify that TFTP and Apache are running.
    service --status-all | grep tftpd-hpa
    service --status-all | grep apache2
  11. Add the following lines to the /etc/default/tftpd-hpa file.
    RUN_DAEMON="yes"
    OPTIONS="-l -s /var/lib/tftpboot"
  12. Add the following line to the /etc/inetd.conf file.
    tftp    dgram   udp    wait    root    /usr/sbin/in.tftpd /usr/sbin/in.tftpd -s /var/lib/tftpboot
  13. Restart the TFTP service.
    sudo /etc/init.d/tftpd-hpa restart
  14. Copy or download the NSX Edge installer ISO file to where it needs to be.
  15. Mount the ISO file and copy the install components to the TFTP server and the Apache server.
    sudo mount -o loop ~/nsx-edge.<build>.iso /mnt
    cd /mnt
    sudo cp -fr install/netboot/* /var/lib/tftpboot/
    sudo mkdir /var/www/html/nsx-edge
    sudo cp -fr /mnt/* /var/www/html/nsx-edge/
  16. (Optional) : Edit the /var/www/html/nsx-edge/preseed.cfg file to modify the encrypted passwords.

    You can use a Linux tool such as mkpasswd to create a password hash.

    sudo apt-get install whois
    sudo mkpasswd -m sha-512
    
    Password: 
    $6$SUFGqs[...]FcoHLijOuFD
    1. To modify the root password, edit /var/www/html/nsx-edge/preseed.cfg and search for the following line:
      d-i passwd/root-password-crypted password $6$tgmLNLMp$9BuAHhN...
    2. Replace the hash string.

      You do not need to escape any special character such as $, ', ", or \.

    3. Add the usermod command to preseed.cfg to set the password for root, admin, or both.

      For example, search for the echo 'VMware NSX Edge' line and add the following command.

      usermod --password '\$6\$VS3exId0aKmzW\$U3g0V7BF0DXlmRI.LR0v/VgloxVotEDp00bO2hUF8u/' root; \
      usermod --password '\$6\$VS3exId0aKmzW\$U3g0V7BF0DXlmRI.LR0v/VgloxVotEDp00bO2hUF8u/' admin; \

      The hash string is an example. You must escape all special characters. The root password in the first usermod command replaces the password that is set in d-i passwd/root-password-crypted password $6$tgm....

      If you use the usermod command to set the password, the user is not prompted to change the password at the first login. Otherwise, the user must change the password at the first login.

  17. Add the following lines to the /var/lib/tftpboot/pxelinux.cfg/default file.

    Be sure to replace 192.168.210.82 with the IP address of your TFTP server.

    label nsxedge
        kernel ubuntu-installer/amd64/linux
        ipappend 2
        append netcfg/dhcp_timeout=60 auto=true priority=critical vga=normal partman-lvm/device_remove_lvm=true netcfg/choose_interface=auto debian-installer/allow_unauthenticated=true preseed/url=http://192.168.210.82/nsx-edge/preseed.cfg mirror/country=manual mirror/http/hostname=192.168.210.82 nsx-kickstart/url=http://192.168.210.82/nsx-edge/nsxcli.install mirror/http/directory=/nsx-edge initrd=ubuntu-installer/amd64/initrd.gz mirror/suite=xenial --
  18. Add the following lines to the /etc/dhcp/dhcpd.conf file.

    Be sure to replace 192.168.210.82 with the IP address of your DHCP server.

    allow booting;
    allow bootp;
    
    next-server 192.168.210.82; #Replace this IP address
    filename "pxelinux.0";
  19. Restart the DHCP service.
    sudo service isc-dhcp-server restart
    Note:

    If an error is returned, for example: "stop: Unknown instance: start: Job failed to start", run sudo /etc/init.d/isc-dhcp-server stop and then sudo /etc/init.d/isc-dhcp-server start. The sudo /etc/init.d/isc-dhcp-server start command returns information about the source of the error.

  20. Use the bare-metal install instructions or the ISO install instructions to complete the installation.
  21. Power on the NSX Bare Metal Host.
  22. At the boot menu, select nsxedge.

    The network is configured, partitions are created, and the NSX Edge components are installed.

    When the NSX Edge login prompt appears, you can log in as admin or root.

    By default, the root login password is vmware, and the admin login password is default.

  23. (Optional) : For optimal performance, reserve memory for the NSX-T component.

    A memory reservation is a guaranteed lower bound on the amount of physical memory that the host reserves for a virtual machine, even when memory is overcommitted. Set the reservation to a level that ensures the NSX-T component has sufficient memory to run efficiently. See System Requirements.

  24. Open the console of the NSX Edge to track the boot process.

    If the console window does not open, make sure that pop-ups are allowed.

  25. After the NSX Edge is completely booted, log in to the CLI and run the get interface eth0 command to verify that the IP address was applied as expected.
    nsx-edge-1> get interface eth0 
    
    Interface: eth0
      Address: 192.168.110.37/24
      MAC address: 00:50:56:86:62:4d
      MTU: 1500
      Default gateway: 192.168.110.1
      Broadcast address: 192.168.110.255
      ...
    

    If needed, run the set interface eth0 ip <CIDR> gateway <gateway-ip> plane mgmt command to update the management interface. Optionally, you can start the SSH service with the start service ssh command.

  26. Verify that the NSX Edge appliance has the required connectivity.

    If you enabled SSH, make sure that you can SSH to your NSX Edge.

    • You can ping your NSX Edge.

    • NSX Edge can ping its default gateway.

    • NSX Edge can ping the hypervisor hosts that are in the same network as the NSX Edge.

    • NSX Edge can ping its DNS server and its NTP server.

  27. Troubleshoot connectivity problems.
    Note:

    If connectivity is not established, make sure the VM network adapter is in the proper network or VLAN.

    By default, the NSX Edge datapath claims all virtual machine NICs except the management NIC (the one that has an IP address and a default route). If DHCP assigns the wrong NIC as management, complete the tasks to correct the problem.

    1. Log in CLI and type the stop service dataplane command.
    2. Type the set interface eth0 dhcp plane mgmt command.
    3. Place eth0 into the DHCP network and wait for an IP address to be assigned to eth0.
    4. Type the start service dataplane command.

      The datapath fp-ethX ports used for the VLAN uplink and the tunnel overlay are shown in the get interfaces and get physical-port commands on the NSX Edge.

What to do next

Join the NSX Edge with the management plane. See Join NSX Edge with the Management Plane.