You can install NSX Edge devices in an automated fashion on bare metal or as a VM using PXE. Note that PXE boot installation is not supported for NSX Manager and NSX Controller. This includes automatically configuring networking settings, such as IP address, gateway, network mask, NTP, and DNS.

About this task

This procedure demonstrates how to set up a PXE server on Ubuntu. PXE is made up of two components: DHCP and TFTP.

DHCP dynamically distributes IP settings to NSX-T components, such as NSX Edge. In a PXE environment, the DHCP server allows NSX Edge to request and receive an IP address automatically.

TFTP is a file-transfer protocol. The TFTP server is always listening for PXE clients on the network. When it detects any network PXE client asking for PXE services, it provides the NSX-T component ISO file and the installation settings contained in a preseed file.

After the PXE server is ready, the procedure shows how to install NSX Edge with a preseeded configuration file.

Prerequisites

  • Verify that the system requirements are met. See System Requirements.

  • Verify that the required ports are open. See Ports and Protocols.

  • If you don't already have one, create the target VM port group network. Most deployments place NSX appliances on a management VM network.

    If you have multiple management networks, you can add static routes to the other networks from the NSX appliance. Prepare management VM port group on which NSX appliances will communicate.

  • Plan your IPv4 IP address scheme. In this release of NSX-T, IPv6 is not supported.

  • A PXE server must be available in your deployment environment. The PXE server can be set up on any Linux distribution. The PXE server must have two interfaces, one for external communication and another for providing DHCP IP and TFTP services.

Procedure

  1. (Optional) : Create a kickstart file.

    A kickstart file is a text file that contains CLI commands that you would generally run on the appliance after the first boot.

    The kickstart file must be named

    nsxcli.install

    and must be copied to your web server, for example at /var/www/html/nsx-edge/nsxcli.install.

    In the kickstart file, you can add the desired CLI commands.

    For example:

    To configure the IP address of the management interface:

    stop dataplane
    set interface eth0 <ip-cidr-format> plane mgmt
    start dataplane
    

    To change the admin user password:

    set user admin password <password>
    

    Note that if you specify a password in the preseed.cfg file, use the same password in the kickstart file. Otherwise, use the default password, which is "default".

    To join the NSX Edge with the management plane:

    join management-plane <mgr-ip> thumbprint <mgr-thumbprint> username <mgr-username> password <mgr password>
  2. Create two interfaces, one for management and another for DHCP and TFTP services.

    Make sure that the DHCP/TFTP interface is in the same subnet that the NSX Edge will reside in.

    For example, if the NSX Edge management interfaces are going to be in the 192.168.210.0/24 subnet, place eth1 in that same subnet.

    # The loopback network interface
    auto lo
    iface lo inet loopback
    
    # PXE server's management interface
    auto eth0
    iface eth0 inet static
      address 192.168.110.81
      gateway 192.168.110.1
      netmask 255.255.255.0
      dns-nameservers 192.168.110.10
    
    # PXE server's DHCP/TFTP interface
    auto eth1
    iface eth1 inet static
      address 192.168.210.82
      gateway 192.168.210.1
      netmask 255.255.255.0
      dns-nameservers 192.168.110.10
    
  3. Install DHCP server software.
    sudo apt-get install isc-dhcp-server -y
  4. Edit the /etc/default/isc-dhcp-server file, and add the interface that provides DHCP service.
    INTERFACES="eth1"
  5. (Optional) If you want this DHCP server to be the official DHCP server for the local network, uncomment the authoritative; line in the /etc/dhcp/dhcpd.conf file.
    ...
    authoritative;
    ...
  6. In /etc/dhcp/dhcpd.conf, define the DHCP settings.

    For example:

    subnet 192.168.210.0 netmask 255.255.255.0 {
       range 192.168.210.90 192.168.210.95;
       option subnet-mask 255.255.255.0;
       option domain-name-servers 192.168.110.10;
       option routers 192.168.210.1;
       option broadcast-address 192.168.210.255;
       default-lease-time 600;
       max-lease-time 7200;
    }
  7. Start the DHCP service.
    sudo service isc-dhcp-server start
  8. Make sure the DHCP service is running.
    service --status-all | grep dhcp
  9. Install Apache, TFTP, and other components that are required for PXE booting.
    sudo apt-get install apache2 tftpd-hpa inetutils-inetd
  10. Make sure that TFTP and Apache are running.
    service --status-all | grep tftpd-hpa
    service --status-all | grep apache2
  11. Add the following lines to the /etc/default/tftpd-hpa file.
    RUN_DAEMON="yes"
    OPTIONS="-l -s /var/lib/tftpboot"
  12. Add the following line to the /etc/inetd.conf file.
    tftp    dgram   udp    wait    root    /usr/sbin/in.tftpd /usr/sbin/in.tftpd -s /var/lib/tftpboot
  13. Restart the TFTP service.
    sudo /etc/init.d/tftpd-hpa restart
  14. Copy or download the NSX Edge installer ISO file to where it needs to be.
  15. Mount the ISO file and copy the install components to the TFTP server and the Apache server.
    sudo mount -o loop ~/nsx-edge.<build>.iso /mnt
    cd /mnt
    sudo cp -fr install/netboot/* /var/lib/tftpboot/
    sudo mkdir /var/www/html/nsx-edge
    sudo cp -fr /mnt/* /var/www/html/nsx-edge/
  16. (Optional) Edit the /var/www/html/nsx-edge/preseed.cfg file to modify the encrypted passwords.

    You can use a Linux tool such as mkpasswd to create a password hash.

    sudo apt-get install whois
    sudo mkpasswd -m sha-512
    
    Password: 
    $6$SUFGqs[...]FcoHLijOuFD

    To modify the root password, edit /var/www/html/nsx-edge/preseed.cfg and search for the following line:

    d-i passwd/root-password-crypted password $6$tgmLNLMp$9BuAHhN...

    Replace the hash string. You do not need to escape any special character such as $, ', ", or \.

    You can also add the usermod command to preseed.cfg to set the password for root, admin, or both. For example, you can add the following two lines:

    usermod --password '\$6\$VS3exId0aKmzW\$U3g0V7BF0DXlmRI.LR0v/VgloxVotEDp00bO2hUF8u/' root; \
    usermod --password '\$6\$VS3exId0aKmzW\$U3g0V7BF0DXlmRI.LR0v/VgloxVotEDp00bO2hUF8u/' admin; \

    The hash string is only an example. You must escape all special characters. The root password in the first usermod command replaces the password that is set in d-i passwd/root-password-crypted password $6$tgm....

    If you use the usermod command to set the password, the user is not prompted to change the password at the first login. Otherwise, the user must change the password at the first login.

  17. Add the following lines to the /var/lib/tftpboot/pxelinux.cfg/default file.

    Be sure to replace 192.168.210.82 with the IP address of your TFTP server.

    label nsxedge
        kernel ubuntu-installer/amd64/linux
        ipappend 2
        append netcfg/dhcp_timeout=60 auto=true priority=critical vga=normal partman-lvm/device_remove_lvm=true netcfg/choose_interface=auto debian-installer/allow_unauthenticated=true preseed/url=http://192.168.210.82/nsx-edge/preseed.cfg mirror/country=manual mirror/http/hostname=192.168.210.82 nsx-kickstart/url=http://192.168.210.82/nsx-edge/nsxcli.install mirror/http/directory=/nsx-edge initrd=ubuntu-installer/amd64/initrd.gz mirror/suite=trusty --
  18. Add the following lines to the /etc/dhcp/dhcpd.conf file.

    Be sure to replace 192.168.210.82 with the IP address of your DHCP server.

    allow booting;
    allow bootp;
    
    next-server 192.168.210.82; #Replace this IP address
    filename "pxelinux.0";
  19. Restart the DHCP service.
    sudo service isc-dhcp-server restart
    Note:

    If an error is returned (for example: "stop: Unknown instance: start: Job failed to start"), run sudo /etc/init.d/isc-dhcp-server stop and then sudo /etc/init.d/isc-dhcp-server start. The sudo /etc/init.d/isc-dhcp-server start command returns information about the source of the error.

  20. Use the bare-metal install instructions or the ISO install instructions to complete the installation.
  21. Power on the VM.
  22. At the boot menu, select nsxedge.

    The network is automatically configured, partitions are created, and the NSX Edge components are installed.

    When the NSX Edge login prompt appears, you can log in as admin or root.

    By default, the root login password is vmware, and the admin login password is default.

  23. For optimal performance, reserve memory for the NSX component.

    A memory reservation is a guaranteed lower bound on the amount of physical memory that the host reserves for a virtual machine, even when memory is overcommitted. Set the reservation to a level that ensures the NSX component has sufficient memory to run efficiently. See System Requirements.

Results

Open the console of the NSX Edge to track the boot process. If the window doesn’t open, make sure that pop-ups are allowed.

After the NSX Edge is completely booted, log in to the CLI and run the get interface eth0 command to verify that the IP address was applied as expected.

nsx-edge-1> get interface eth0 

Interface: eth0
  Address: 192.168.110.37/24
  MAC address: 00:50:56:86:62:4d
  MTU: 1500
  Default gateway: 192.168.110.1
  Broadcast address: 192.168.110.255
  ...

If needed, run the set interface eth0 ip <CIDR> gateway <gateway-ip> plane mgmt command to update the management interface. Optionally, you can start the SSH service with the start service ssh command.

Ensure that your NSX Edge appliance has the required connectivity.

  • Make sure that you can ping your NSX Edge.

  • Make sure that the NSX Edge can ping its default gateway.

  • Make sure that your NSX Edge can ping the hypervisor hosts that are in the same network as the NSX Edge.

  • Make sure that the NSX Edge can ping its DNS server and its NTP server.

  • If you enabled SSH, make sure that you can SSH to your NSX Edge.

Note:

If connectivity is not established, make sure the VM network adapter is in the proper network or VLAN.

By default, the NSX Edge datapath claims all virtual machine NICs except the management NIC (the one that has an IP address and a default route). If DHCP assigns the wrong NIC as management, you can correct this as follows:

  1. stop service dataplane

  2. set interface eth0 dhcp plane mgmt

  3. Place eth0 into the DHCP network and wait for an IP address to be assigned to eth0.

  4. start service dataplane

The datapath fp-ethX ports used for the VLAN uplink and the tunnel overlay are shown in the get interfaces and get physical-port commands on the NSX Edge.

What to do next

Join the NSX Edge with the management plane. See Join NSX Edge with the Management Plane.