The vSphere Bitfusion OVA file contains the compressed open virtualization format files that compose the vSphere Bitfusion server. After setting up your vSphere environment, download the vSphere Bitfusion OVA file and deploy it as a vSphere appliance.

You can deploy virtual machines, virtual appliances, and vApps in Open Virtual Format (OVF) and Open Virtual Appliance (OVA). Deploying an OVF or OVA template allows you to add pre-configured virtual machines or vApps to your vCenter Server or ESXi inventory. Deploying an OVF or OVA template is similar to deploying a virtual machine from a template. However, you can deploy an OVF or OVA template from any local file system accessible from the vSphere Client, or from a remote web server.

vSphere Bitfusion is distributed as an OVA file. To learn more about OVA or OVF files, see the vSphere Virtual Machine Administration documentation.

Figure 1. Primary vSphere Bitfusion server deployment workflow

Workflow diagram that depicts the deployment process of the primary vSphere Bitfusion server

Start the vSphere Bitfusion appliance deployment

To begin the vSphere Bitfusion deployment, you specify details about the primary vSphere Bitfusion server, such as host name, location, and storage within your vSphere environment.

You follow this procedure to install the primary vSphere Bitfusion server and is the first step to install vSphere Bitfusion. To install secondary vSphere Bitfusion servers, use the vSphere Bitfusion plug-in, that is added to your vCenter Server instance during the installation of the primary server. See How to install subsequent vSphere Bitfusion servers.

Prerequisites

  • Download the vSphere Bitfusion OVA file from https://my.vmware.com/downloads/.
  • Verify that the vSphere environment on which you are deploying the vSphere Bitfusion appliance meets the minimum system requirements. See System requirements for vSphere Bitfusion server.
  • Verify that you can log into the vSphere Client as an administrator.
  • Verify that the ESXi hosts on which you want to deploy the vSphere Bitfusion appliances are running.

Procedure

  1. Log in to vSphere Client as an administrator.
  2. In the vSphere Client, right-click the ESXi host on which to deploy the vSphere Bitfusion appliance and select Deploy OVF Template.
  3. On the Select an OVF template page, enter the URL of the OVA file or browse to the file, and click Next.
  4. On the Select a name and folder page, enter a name for the vSphere Bitfusion virtual machine and select a location for your deployment, and click Next.
  5. On the Select a compute resource page, select a resource on which to run the deployed VM template and click Next.
  6. On the Review details page, verify the OVF template details and click Next.
    The Review details page displays a warning cautioning that the vSphere Bitfusion OVF uses advanced configuration values that might pose a security risk. The configuration values that trigger the alert are pciPassthru.use64bitMMIO = true and pciPassthru.64bitMMIOSizeGB = 256. The first parameter enables PCI passthrough for GPU devices, that require 16 GB or more of memory mapping, and the second parameter configures a memory mapped I/O (MMIO) size of 256 GB. You can adjust this value later in the settings of the vSphere Bitfusion virtual machine.
  7. On the Select storage page, define where and how to store the files for the deployed OVF template, and click Next.
  8. On the Select networks page, select the network to use with Network Adapter 1 and click Next.
    The network you use with Network Adapter 1 carries management traffic. You can use the same network concurrently for data traffic or add additional network adapters later.

Customize the vSphere Bitfusion appliance OVF template

As part of the vSphere Bitfusion appliance deployment, you must specify several custom parameters in the Deploy OVF Template dialog box.

On the Customize template page of the Deploy OVF Template dialog box, you customize the deployment properties of the OVF template.

Prerequisites

  • Verify that you have the vCenter Server TLS certificate thumbprint. See How to obtain the vCenter Server TLS certificate thumbprint.
  • Verify that the DNS and NTP services you use are set up properly in your environment.
    • If you use DHCP, verify that it provides DNS and NTP addresses.
    • If you do not use DHCP, specify the DNS and NTP server addresses in the OVF template.
    Note: Clock synchronization is important for the functionality of vSphere Bitfusion.

Procedure

  1. In the Bitfusion Server Setup section, specify a hostname for the server.
    Valid characters for hostnames are the ASCII characters A through Z (both upper- and lower-case), the digits 0 to 9, and the hyphen (-). A hostname cannot start with a hyphen. The hostname is stored in /etc/hostname.
  2. In the Bitfusion Server Setup section, enter the user name and password for the vCenter Server instance on which you are deploying the vSphere Bitfusion OVF template.
  3. In the Bitfusion Server Setup section, enter your vCenter Server TLS certificate thumbprint.
    All letters in the thumbprint must be uppercase, and a colon symbol must be used between every two symbols. For example, 6E:A5:99:4E:3B:B7:3B:D9:CC:12:5F:96:D2:08:8A:29:2A:3F:94:2F.
  4. In the Credentials section, specify a customer password.
    After the deployment is complete, you use the customer user account to log into the vCenter Server appliance using the console shell or SSH.
  5. (Optional) In the NVIDIA Packages section, select the Download and Install NVIDIA packages check box to accept the NVIDIA license.
    By accepting the NVIDIA license, vSphere Bitfusion downloads and installs the NVIDIA driver, CUDA libraries, and NVIDIA Fabric Manager during the first boot of the virtual machine.
    Note: If you are operating vSphere Bitfusion in an environment without access to the Internet, for example, by using an air-gapped network, do not select the check box. You must manually download and install the NVIDIA software after deploying the vSphere Bitfusion appliance.
  6. In the Network Adapter section, specify the networking configuration for your environment.
    You must specify the configuration for Network Adapter 1 that is used for management and data traffic. Network Adapter 1 must be connected to a network that communicates with the vCenter Server instance.

    Network Adapters 2, 3, and 4 are optional and are used for data traffic only. Each network adapter must be connected to a separate network. vSphere Bitfusion chooses the network that is most efficient for data transfers to the vSphere Bitfusion server.

    You can use PVRDMA adapters, which improves the network performance and connections between your vSphere Bitfusion servers and clients. For more information, see Using vSphere Bitfusion with PVRDMA in the Installing VMware vSphere Bitfusion documentation.

    Option Description
    IPv4 Address Enter the IPv4 address of the network adapter. If you are using DHCP, leave this text box blank.
    Note: IPv6 is not supported.
    Network CIDR Prefix Enter the network Classless Inter-Domain Routing (CIDR) settings.

    For example, if your network uses a /24 netmask, select 24 (255.255.255.0) from the drop-down menu.
    MTU Enter an MTU size. The default value is 1500. For optimal performance, specify an MTU size that is equal to the maximum MTU size supported by your network hardware.
    Note: If you set an MTU size greater than 1500, verify that the network switches in your data center are enabled for jumbo frames.
    Gateway Enter the network gateway address to use with the appliance. If you are using DHCP, leave this text box blank.
    DNS Enter the DNS server address to use with the appliance. If you are using DHCP, leave this text box blank.
    DNS Search Domains Enter the DNS search domain address to use with the appliance. Separate multiple addresses with a space character. If you are using DHCP, leave this text box blank.
    NTP Enter the NTP server address to use with the appliance. Separate multiple addresses with a space character. If you are using DHCP and the DHCP server supports sending NTP server information, leave this text box blank.
    Configure Network Adapter Select the check box if you want to configure Network Adapter 2. Repeat for each subsequent network adapter.
  7. Click Next.
  8. On the Ready to complete page, review the vSphere Bitfusion server configuration and click Finish.

Results

A new task for creating the vSphere Bitfusion appliance appears in the Recent Tasks pane. After the task finishes, the new appliance is created on the selected resource.

Pass through a GPU to the vSphere Bitfusion virtual machine

VMDirectPath I/O allows the guest operating system to access the GPU directly, bypassing the ESXi hypervisor. By using passthrough devices, you can use resources more efficiently and improve the performance of your vSphere Bitfusion environment. Enabling passthrough of the GPU provides a level of performance on vSphere close to that of its native system.

Prerequisites

  • Verify that you have the privileges that you need for the task that you plan to perform.
    • Verify that you have the Virtual machine.Configuration.Add or remove device privilege.
    • Verify that you have the Virtual machine.Configuration.Advanced configuration privilege.
    • If you plan to increase the memory reservation when you edit a virtual machine, verify that you have the Virtual machine.Configuration.Change resource privilege.
    • Verify that you have the Virtual machine.Configuration.Change Memory privilege.
  • Verify that the virtual machine of the vSphere Bitfusion server is powered off.
  • To use DirectPath, verify that Intel Virtualization Technology for Directed I/O (VT-d) or AMD I/O Virtualization Technology (IOMMU) is enabled in the BIOS of the ESXi host.
  • Verify that the GPU PCI devices are connected to the host and marked as available for passthrough. See How to enable a GPU for passthrough.
  • If your ESXi host is configured to boot from a USB device, or if the active coredump partition is configured to be on a USB device or SD cards connected through USB channels, deactivate the USB controller for passthrough.
    Note: VMware does not support a USB controller passthrough for ESXi hosts that boot from USB devices or SD cards connected through USB channels. A configuration in which the active coredump partition is configured to be on a USB device or SD card connected through USB channels is also not supported. For information, see http://kb.vmware.com/kb/1021345.

Procedure

  1. Add a GPU device.
    1. In the vSphere Client, right-click the vSphere Bitfusion virtual machine in the inventory and select Edit Settings.
    2. On the Virtual Hardware tab, click the Add New Device button.
    3. From the drop-down menu, under Other Devices, select PCI Device.
    4. Expand the New PCI device section and select the access type.
    5. In the New PCI device section, select a GPU from the PCI device drop-down menu.
      Note: By default, the same PCI device address is listed for each new GPU. When adding multiple devices, you must select the PCI addresses of each individual device.
    6. Click OK.
  2. Configure the CPU and memory resources of the ESXi host.
    If the ESXi host is a dedicated vSphere Bitfusion server, set the CPU and memory to their maximum values. If the host machine is not dedicated to vSphere Bitfusion, specify the minimum CPU value as the number of GPUs multiplied by 4, and the minimum memory as 1.5 times that of the aggregated GPU card memory or 32 GB, whichever is higher.
    1. In the vSphere Client, right-click the vSphere Bitfusion virtual machine and select Edit Settings.
    2. Expand the CPU section and edit the resources.
    3. Expand the Memory section and edit the resources.
    4. Under Memory, select the Reserve all guest memory (All locked) check box.
    5. Click OK.
  3. (Optional) Adjust the memory mapped I/O (MMIO) size.
    By default, the vSphere Bitfusion installer configures an MMIO size of 256 GB. If you need to reserve a larger amount of memory for MMIO, you can change the MMIO size in the advanced options of the VM. To calculate how much actual memory you must reserve for MMIO, consider the following MMIO memory calculations for two and three cards, each with 16 GB of memory.
    • 2 x 16 GB = 32. Round 32 GB to the next power of 2, and the memory mapped I/O size that is needed is 64 GB.
    • 3 x 16 GB = 48. Round 48 GB to the next power of 2, and memory mapped I/O size that is needed is 64 GB.
    1. In the vSphere Client, right-click the vSphere Bitfusion virtual machine and select Edit Settings.
    2. Click VM Options, and expand the Advanced section.
    3. Under Configuration Parameters, click Edit Configuration.
    4. In the Configuration Parameters dialog box, locate the parameter pciPassthru.64bitMMIOSizeGB, and enter the MMIO size in gigabytes.
    5. Click OK.
  4. (Optional) Take a snapshot of the virtual machine.
    Snapshots capture the state of the virtual machine at the time you take the snapshot. If an error occurs when you start the virtual machine, you can recover your vSphere Bitfusion installation from the snapshot.
    1. In the vSphere Client, right-click the vSphere Bitfusion virtual machine and select Snapshots > Take Snapshot.
    2. Enter a name and description for the snapshot.
    3. Click Create.
  5. In the vSphere Client, right-click the vSphere Bitfusion virtual machine and select Power > Power On.

Results

After you start the primary vSphere Bitfusion server, vSphere Bitfusion registers a vSphere Bitfusion plug-in in your vCenter Server instance, resulting in a single vSphere Bitfusion cluster that contains one vSphere Bitfusion server.

A new task for installing the vSphere Bitfusion server appears in the Recent Tasks pane. After the task finishes, the vSphere Bitfusion appliance is created on the selected resources.

Note: After the virtual machine powers on, allow it to run for ten or more minutes before performing any further configuration tasks or operations. During this time the virtual machine registers with vCenter Server and, downloads and installs the NVIDIA driver if you choose this installation option.

What to do next

  • If you chose not to download and install the NVIDIA driver during first boot, you must install the software manually. See How to install the NVIDIA driver in vSphere Bitfusion.
  • You can add more network adapters for data traffic to your vSphere Bitfusion. For more information, see Modifying the Network Settings of a vSphere Bitfusion Server in the Using VMware vSphere Bitfusion.

Verify that the vSphere Bitfusion plug-in registers with vCenter Server

After deploying the vSphere Bitfusion server, verify that the vSphere Bitfusion plug-in appears in the vSphere Client.

Prerequisites

  • If you chose not to install the NVIDIA driver as part of the vSphere Bitfusion deployment, restart the vSphere Bitfusion virtual machine (VM).

Procedure

  1. (Optional) To update all data in the current vSphere Client view, click the vSphere Client refresh icon (vSphere Client refresh icon).
  2. In the vSphere Client, select Menu (vSphere Client menu icon) > Bitfusion.
    The vSphere Bitfusion plug-in loads.

What to do next

If the vSphere Bitfusion plug-in is registered properly, you can deploy vSphere Bitfusion clients and subsequent servers.

If the vSphere Bitfusion plug-in is not working or is not available, verify that you configured properly the NTP, DNS, GUID, and SHA1 settings in the OVF template. Additionally, to check for error messages, you can launch the web console of the vSphere Bitfusion server VM.