This topic explains how to build and use custom OVA images as the basis for cluster nodes deployed by Tanzu Kubernetes Grid on vSphere.

Prerequisites

Before you begin this procedure, you must have:

Build an Image with Kubernetes Image Builder

To build a custom OVA image for vSphere on your local workstation:

  1. Clone the Kubernetes Image Builder tool.

    git clone https://github.com/kubernetes-sigs/image-builder.git
    
  2. Change directory to the cloned image-builder repository.

    cd image-builder/images/capi
    
  3. Install required dependencies for vSphere.

    make deps-ova
    
  4. If the make deps-ova output states that the command installed binaries to the .local/bin directory, then move these files into your $PATH. For example:

    mv .local/bin/packer /usr/local/bin
    
  5. Determine the Image Builder configuration version that you want to build from.

    • Search the VMware {code} Sample Exchange for TKG Image Builder to list the available versions.
    • Each version corresponds to the Kubernetes version that Image Builder uses. For example, image-builder-1.19.1-cfg.zip builds a Kubernetes v1.19.1 image.
    • If you need to create a management cluster, which you must do when you first install Tanzu Kubernetes Grid, choose the default Kubernetes version of your Tanzu Kubernetes Grid version. For example, in Tanzu Kubernetes Grid v1.2.0, the default Kubernetes version is v1.19.1.
  6. Download the configuration code zip file, and unpack its contents. You can unpack it within the image-builder directory structure or elsewhere.

  7. Within this extracted Image Builder configuration directory, in the subdirectory that contains build-image.sh, kubernetes.json and other files, create a file called vsphere.json with the following contents:

    {
        "vcenter_server": "",
        "datacenter": "",
        "username": "administrator@vsphere.local",
        "password": "",
        "datastore": "",
        "folder": "",
        "cluster": "",
        "network": "VM Network",
        "insecure_connection": "false",
        "create_snapshot": "true",
        "convert_to_template": "true"
    }
    
    • vcenter_server: Specify the IP address or FQDN of your vCenter.
    • datacenter: Specify an appropriate vSphere datacenter.
    • username and password: Provide a vSphere account and password that has access to create VMs and templates.
    • datastore: Specify the vSphere datastore to use for VM creation and installation media.
    • cluster: Specify an appropriate compute resource.
    • network: Enter an appropriate VM network that has access to internet resources.
    • insecure_connection: Set to true if your vCenter uses a self-signed certificate.
    • create_snapshot: Set to true if you plan on using the built template to generate other templates later.
    • convert_to_template: Set to true if you plan on using the built template to generate other templates later.
  8. In the kubernetes.json file, confirm or edit the Kubernetes version setting kubernetes_semver and source setting kubernetes_http_source.

    • For open source, upstream kubernetes_semver values such as v1.19.1, kubernetes_http_source must refer to an open source bucket.
    • Downstream, VMware Kubernetes versions such as v1.19.1+vmware.2 have a +vmware suffix and require kubernetes_http_source to point to VMware's build artifactory.
  9. Edit the osstp.json file in the same directory, and replace the . in the custom_role_names value with the full path to the ansible_customize directory. For example, you can use pwd.

  10. Set an environment variable PACKER_VAR_FILES to point to the .json files in your configuration directory:

    export PACKER_VAR_FILES="`pwd`/kubernetes.json `pwd`/osstp.json `pwd`/vsphere.json"
    
  11. From the image-builder/images/capi directory, run the following command to start the build process:

    For Photon3:

    make build-node-ova-vsphere-photon-3
    

    For Ubuntu 18.04

    make build-node-ova-vsphere-ubuntu-1804
    

    For Red Hat Enterprise Linux 7

    1. Download the Red Hat Enterprise Linux 7.7 ISO from Red Hat Customer Portal with SHA256 checksum of 88b42e934c24af65e78e09f0993e4dded128d74ec0af30b89b3cdc02ec48f028, and store it in your root with the filename /rhel-server-7.7-x86_64-dvd.iso. The filename must be exactly as shown and the checksum must match.
    2. Export a username and password with access to register a RHEL7 system with Red Hat Subscription Manager:

      export RHSM_USER='your username here'
      export RHSM_PASS='your password here'
      
    3. Build the image:

      make build-node-ova-vsphere-rhel-7
      

References

In the Kubernetes Image Builder documentation, see:

Import the OVA to vSphere

  1. Before you deploy a cluster with the built custom image, you must first import the built OVA. You can find OVA in the ./output directory. It is important to import this OVA, because the file has been tagged with the necessary parameters for Tanzu Kubernetes Grid to recognize the image as valid. You may have to delete the template left behind on vSphere before you import the OVA file.

  2. After the OVA has been imported to vSphere, you must convert the OVA to a template before you deploy a cluster.

  3. If you have more than one template that was built for a particular version of Kubernetes on vSphere (for example, photon3-1.19.1+vmware.2, ubuntu1804-1.19.1+vmware.2, and rhel7-1.19.1+vmware.2), then you can export the template name to use when deploying Tanzu Kubernetes Grid to vSphere:

    export VSPHERE_TEMPLATE='name of the template in vSphere'
    

What to Do Next

check-circle-line exclamation-circle-line close-line
Scroll to top icon