VIA images the rack components in a pre-determined order, which is determined by the availability of network route to the different components of the rack. All switches are imaged first. This enables VIA to access the servers through the switches for imaging. The imaging order is as follows.

  1. Management switch

    The management switch is the main access gateway through which the Cloud Foundation management data is routed. The management ports of the ToR switches, Spine switches, and the physical servers are connected to the management switch. The data ports of the ToR switches are also connected to the management switch. This enables VIA to communicate with the servers over both management and data network through the management switch. VIA is also connected to the rack through a designated port on the management switch. It is therefore required that the management switch is the first component imaged by VIA in order to obtain access to the other components of the rack. VIA currently uses an IPMI connection to image the management switch.

  2. Spine switches (rack 2 only) and ToR switches

    Spine and ToR switches are imaged in parallel.

    Spine switches inter-connect multiple racks enabling a scale out architecture for the datacenter. They create an stretched L2 backplane between racks.

    ToR switches provide connectivity to servers in each rack out to spine switches. The first pair of ToR switches provide connectivity to your datacenter network.

  3. Servers

    The management ports on the servers become accessible to the management switch during the course of imaging/configuration, which in turn make the management ports accessible to VIA through the management switch. Once all the switches are imaged and configured, the data ports of the servers become accessible to VIA through the ToR switches, which then proceeds to image the servers in parallel.

For each component that is being imaged, the following tasks are performed.

  1. Discovery

    Rack components are discovered using the DHCP service running on the VIA VM. The DHCP Service uses the device type information to identify the device being discovered. Apart from the device type information, the DHCP service also uses hardware vendor specific strings to determine whether the switch being imaged is a management, ToR, or Spine switch.

    The first component to be discovered is the management switch. The DHCP service hands out a pre-determined IP address for the management switch followed by a PXE image specific to the management switch.

    After the management switch is imaged, the ToR and Spine switches are discovered and imaged. The management switch also discovers the IPMI network of the servers. This allows VIA to initiate imaging of the servers. The ToR switch enables discovery of the data network of the servers which is used to receive the installation image delivered by the DHCP service.

  2. Image installation

    Image installation refers to installing software on the components to make them operational. The software depends on the component type - an Operating System for switches and a Hypervisor for servers.

  3. Configuration

    This step in the imaging process ensures that the components of the rack work like a homogenous system. Configuration of each rack component is different. If any configuration step fails for the management, ToR, or spine switches, imaging stops at that point and cannot proceed. If a configuration step fails for the server, imaging for that server cannot be completed but the remaining servers in the rack can be imaged.

    Table 1. Management Switch Configuration

    Number

    Step Name

    Description

    1

    Apply license

    Apply the relevant license to the installed image

    2

    Configure ports

    1. Configure the ports which allow the management switch to connect to the management interfaces of the ToR and spine switches and the servers.

    2. Bridge the ports connected to VIA with the ports connected to the management interfaces of the ToR and spine switches.

    3. Create separate subnets for the management network and data network of the rack.

    3

    Update interface

    Ensure that only the management interfaces of ToR and spine switches are enabled while the management interfaces of the servers and the data network interfaces of the ToR switches are disabled before initiating the imaging of ToR switches.

    4

    Setup persistent network

    1. Wait till all ToRs are imaged and bridge the ports connected to VIA with the ports connected to the management interfaces of the ToR and spine switches and servers to enable VIA to listen to DHCP requests.

    2. Setup Spanning Tree Protocol (STP) on the IPMI management interfaces of the ToR and spine switches and servers.

    3. Setup STP on the Cloud Foundation management interfaces of the ToR switches and the interfaces connected to VIA.

    4. Enable LACP on ToR data interfaces.

    5. Create separate subnets for the management network and data network for the rack.

    5

    Setup IPMI DHCP

    Set up a DHCP service to discover the IPMI network of the servers.

    6

    Host Power Cycle

    Discover all servers and ensure that the minimum required servers are available to ensure that SDDC Manager can be deployed. If the requirement is met, VIA initiates a power cycle of all servers to initiate their imaging. If the required number of servers are not detected, imaging is aborted.

    7

    Change Password

    Change the default password to connect to the switch and stores the new password in a password store.

    8

    Generate Manifest

    Generate device manifest, which contains the current state of the imaging activity for each rack component.

    Table 2. ToR Switch Configuration

    Number

    Step

    Description

    1

    Apply license

    Apply the relevant license to the installed image.

    2

    Configure ports

    Configure all ports on the switch to operate in Full Duplex mode with auto negotiation enabled and at 1000Mb/s.

    6

    Change password

    Change the default password to connect to the switch and stores the new password in a password store.

    7

    Generate Manifest

    Generate device manifest, which contains the current state of the imaging activity for each rack component.

    Table 3. Spine Switch Configuration

    Number

    Step

    Description

    1

    Apply license

    Apply the relevant license to the installed image.

    5

    Change password

    Change the default password to connect to the switch and stores the new password in a password store.

    6

    Generate Manifest

    Generate device manifest, which contains the current state of the imaging activity for each rack component.

    Table 4. Node 0 Configuration

    Number

    Step

    1

    Wait for kickstart delivery.

    2

    Check host status.

    3

    Install VIBs.

    4

    Run storage configuration script.

    5

    Check VSAN setup.

    6

    Reboot host.

    7

    Post- ESXi installation configuration.

    8

    Verify disk status.

    9

    Create user task.

    10

    Check VSAN status after reboot.

    11

    Deploy LCM.

    12

    Shutdown LCM.

    13

    Take LCM snapshot.

    14

    Deploy LCM backup VM.

    15.

    Shutdown backup LCM VM.

    16

    Take backup LCM snapshot.

    17

    Deploy ISVMs.

    18

    Shutdown ISVMs.

    19

    Take ISVM snapshot.

    20

    Deploy VRM.

    21

    Post VRM installation configuration.

    22

    Set VM startup shutdown order.

    23

    Upload bundle ISO.

    24

    Add ISO to VRM.

    25

    Collect inventory.

    26

    Import SSH public keys.

    27

    Copy PRM manifest.

    28

    Copy HMS IB inventory.

    29

    Create VRM snapshot.

    30

    Reboot VRM.

    Table 5. Configuration on Remaining Nodes

    Number

    Step

    Description

    1

    Install Custom VIBs

    Install any custom VIBs that may be necessary to enable vendor specific devices on the server.

    2

    Reboot server

    Reboot the server to complete the installation process.

    3

    Apply licence

    Apply ESXi licence.

    4

    Create user

    Create a new ESXi user with Administrator role.

    5

    Generate manifest

    Generate device manifest, which contains the imaging status of the device, the IP address assigned, the software used to image it, etc.

    This is performed on all components irrespective of whether the previous steps were successful or not. This allows VIA to track the status of imaging of any given component during any stage of the imaging process.

    Imaging is a multi-step process.