This topic explains how to build and use custom AMI images as the basis for cluster nodes deployed by Tanzu Kubernetes Grid on Amazon EC2.

Prerequisites

Before you begin this procedure, you must have:

Build an Image with Kubernetes Image Builder

To build a Tanzu Kubernetes Grid image for AWS on your local workstation:

  1. Clone the Kubernetes Image Builder tool.

    git clone https://github.com/kubernetes-sigs/image-builder.git
    
  2. Change directory to the cloned image-builder repository.

    cd image-builder/images/capi
    
  3. Install required dependencies for AWS.

    make deps-ami
    
  4. If the make deps-ami output states that the command installed binaries to the .local/bin directory, move these files into your $PATH. For example:

    mv .local/bin/packer /usr/local/bin
    
  5. Determine the Image Builder configuration version that you want to build from.

    • Search the VMware {code} Sample Exchange for TKG Image Builder to list the available versions.
    • Each version corresponds to the Kubernetes version that Image Builder uses. For example, image-builder-1.19.1-cfg.zip builds a Kubernetes v1.19.1 image.

    • If you need to create a management cluster, which you must do when you first install Tanzu Kubernetes Grid, choose the default Kubernetes version of your Tanzu Kubernetes Grid version. For example, in Tanzu Kubernetes Grid v1.2.0, the default Kubernetes version is v1.19.1.

  6. Download the configuration code zip file, and unpack its contents. You can unpack it within the image-builder directory structure or elsewhere.

  7. Within this extracted Image Builder configuration directory, in the subdirectory that contains build-image.sh, kubernetes.json and other files, create a file called aws.json with the following contents:

    {
    "ami_regions": "ap-south-1,eu-west-3,eu-west-2,eu-west-1,ap-northeast-2,ap-northeast-1,sa-east-1,ca-central-1,ap-southeast-1,ap-southeast-2,eu-central-1,us-east-1,us-east-2,us-west-1,us-west-2",
    "aws_region": "us-east-1",
    "aws_access_key": "",
    "aws_secret_key": ""
    }
    
  8. Edit the ami_regions to only include the regions where you would like to copy the built image.
  9. Edit the aws_region to specify the region where you would like to build the image build.
  10. Fill in your aws_access_key and aws_secret_key, and save the aws.json file.
  11. Edit the osstp.json file, and replace the . in the custom_role_names value with the full path to the ansible_customize directory. For example, you can use pwd.
  12. Set the PACKER_VAR_FILES environment variable.
export PACKER_VAR_FILES="`pwd`/kubernetes.json `pwd`/osstp.json `pwd`/aws.json"
  1. From the image-builder/images/capi directory, run the following command to start the build process:

    For Amazon Linux 2:

    make build-ami-amazon-2
    

    For Ubuntu 18.04

    make build-ami-ubuntu-1804
    
  2. Parse and record the ami-id values listed in the output. You insert these values later in the BOM file. The output has a different ami-id for each of the ami_regions that you set previously.

References

In the Kubernetes Image Builder documentation, see:

Edit the Tanzu Kubernetes Grid Bill of Materials

After your custom image is available in AWS, update the bill of materials (BOM) YAML file to point to the new image:

  1. In ~/.tkg/bom, find the BOM file that matches your Image Builder configuration version, which is based on its Kubernetes version:

    • If a Kubernetes version is not the default for a Tanzu Kubernetes Grid version, its BOM file is named for the Kubernetes version. For example, bom-1.18.8+vmware.1.yaml can use an image created with image-builder-1.18.8-cfg.
    • If a Kubernetes version is the default for a Tanzu Kubernetes Grid version, then its BOM file is named for the Tanzu Kubernetes Grid version. For example, bom-1.2.0+vmware.1.yaml can use an image created with image-builder-1.19.1-cfg because v1.19.1 is the default Kubernetes version in Tanzu Kubernetes Grid v1.2.0.
    • In all cases, the components.kubernetes.version field in a BOM file indicates its Kubernetes version.
  2. In the BOM file, locate the entire ami: key.

  3. Using ami-id values recorded from the image-builder output, replace the existing ami-id with the new values for each region where Image Builder copied the image.

    ami:
      ap-northeast-1:
        id: <new ami id here>
      ap-northeast-2:
        id: <new ami id here>
      ...
    

What to Do Next

After you configure the BOM file with your custom AMIs, you can deploy your management cluster or a workload cluster to AWS:

check-circle-line exclamation-circle-line close-line
Scroll to top icon