RSS
 

Prepare an Internet-Restricted Environment

You can deploy Tanzu Kubernetes Grid management clusters and workload clusters in environments that are not connected to the Internet, such as:

  • Proxied environments
  • Airgapped environments, with no physical connection to the Internet

This topic explains how to deploy management clusters to internet-restricted environments on vSphere or AWS.

To follow this procedure, you can either use a single machine or different machines as the bootstrap machines in the online and offline environments.

You do not need to follow these procedures if you are using Tanzu Kubernetes Grid in a connected environment that can pull images over an external Internet connection.

(Optional) Download the Harbor OVA

If you want to install the Harbor registry, download the Harbor OVA file:

  1. Go to VMware Customer Connect and log in with your VMware Customer Connect credentials.
  2. Visit the Tanzu Kubernetes Grid downloads page.
  3. In the VMware Tanzu Kubernetes Grid row, click Go to Downloads.
  4. In the Select Version drop-down, select 2.2.0.
  5. Under Product Downloads, scroll to the section labeled Harbor OVA, and click Download Now.
  6. (Optional) Verify that your downloaded files are unaltered from the original. VMware provides a SHA-1, a SHA-256, and an MD5 checksum for each download. To obtain these checksums, click Read More under the entry that you want to download. For more information, see Using Cryptographic Hashes.

General Prerequisites

Before you can deploy management clusters and workload clusters in an Internet-restricted environment, you must have:

  • An Internet-connected Linux bootstrap machine that:
    • Is not inside the internet-restricted environment or can access the domains listed in Proxy Server Allowlist.
    • Has minimum 2 GB RAM, 2 vCPU and 45 GB hard disk space.
    • Has the Tanzu CLI installed. See Install the Tanzu CLI and Other Tools to download, unpack, and install the Tanzu CLI binary on your Internet-connected system.
  • A way for cluster VMs to access images in the private registry:
    • Proxied environments: An egress proxy server that lets cluster VMs access the registry.
      • When you deploy a management cluster in this proxied environment, set TKG_*_PROXY variables in the cluster configuration file to the proxy server’s address, and set TKG_PROXY_CA_CERT to the proxy server’s CA if its certificate is self-signed. See Configure Proxies.
    • Airgapped environments: A USB thumb drive or other medium for bringing the private registry behind an airgap, after the registry is populated with images.
  • A USB thumb drive or other portable offline storage device.

vSphere Prerequisites and Architecture

vSphere Architecture

An internet-restricted Tanzu Kubernetes Grid installation on vSphere has firewalls and communication between major components as shown here.

Note: The following diagram describes the scenario where different bootstrap machines are used in the online and offline environments.

Diagram: Airgapped TKG on vSphere

On vSphere, in addition to the general prerequisites above, you must:

  • Upload to vSphere the OVAs from which node VMs are created. See Import the Base Image Template into vSphere in Deploy Management Clusters to vSphere.

    After the VM is created, if you cannot log in with the default username/password, reset the password using Gnu GRUB, as described in Resetting a Lost Root Password if it is Photon OS.

  • Log in to the jumpbox as root, and enable remote ssh as follows:

    1. Open the file /etc/ssh/sshd_config in an editor. nano /etc/ssh/sshd_config
    2. Add a line in the Authentication section of the file that says PermitRootLogin yes. In this case the line exist, remove the “#”.
    3. Save the updated /etc/ssh/sshd_config file.
    4. Restart the SSH server using service sshd restart

  • Install and configure a private Docker-compatible container registry such as Harbor, Docker, or Artifactory as follows. This registry runs outside of Tanzu Kubernetes Grid and is separate from any registry deployed as a shared service for clusters:

    • Install the registry within your firewall.
    • You can configure the container registry with SSL certificates signed by a trusted CA, or with self-signed certificates.
    • The registry must not implement user authentication. For example, if you use a Harbor registry, the project must be public, and not private.
    • To install Harbor on vSphere:

  • Configure an offline subnet to use as the internet-restricted environment, and associate it with the jumpbox.

  • Set up the DHCP server to allocate private IP’s to the new instance.

  • Create a vSphere distributed switch on a data center to handle the networking configuration of multiple hosts at a time from a central place.

AWS Prerequisites and Architecture

AWS Architecture

A proxied Tanzu Kubernetes Grid installation on Amazon Web Services (AWS) has firewalls and communication between major components as shown here. Security Groups (SG) are automatically created between the control plane and workload domains, and between the workload components and control plane components.

Diagram: Airgapped TKG on AWS

For a proxied installation on AWS, in addition to the general prerequisites above, you also need:

  • An AWS VPC with no internet gateway (“offline VPC”) configured as described below.
    • Your internet-connected bootstrap machine must be able to access IP addresses within this offline VPC. For more information, see VPC Peering.
  • A private Docker-compatible container registry such as Harbor, Docker, or Artifactory installed and configured as follows. This registry runs outside of Tanzu Kubernetes Grid and is separate from any registry deployed as a shared service for clusters:
    • Install the registry within your firewall.
    • You can configure the container registry with SSL certificates signed by a trusted CA, or with self-signed certificates.
    • The registry must not implement user authentication. For example, if you use a Harbor registry, the project must be public, and not private.
    • To install Harbor:
      1. Download the Harbor OVA.
      2. Follow the Harbor Installation and Configuration instructions in the Harbor documentation.
  • A Linux bootstrap VM running within your offline VPC, provisioned similarly to the internet-connected machine above.
    • The offline bootstrap VM must be able to reach cluster VMs created by Tanzu Kubernetes Grid directly, without a proxy.

After you create the offline VPC, you must add following endpoints to it (VPC endpoint enables private connections between your VPC and supported AWS services):

  • Service endpoints:
    • sts
    • ssm
    • ec2
    • ec2messages
    • elasticloadbalancing
    • secretsmanager
    • ssmmessages

To add the service endpoints to your VPC:

  1. In the AWS console, browse to VPC Dashboard > Endpoints.
  2. For each of the above services
    1. Click Create Endpoint.
    2. Search for the service and select it under Service Name.
    3. Select your VPC and its Subnets.
    4. Enable DNS Name for the endpoint.
    5. Select a Security group that allows VMs in the VPC to access the endpoint.
    6. Select Policy > Full Access.
    7. Click Create endpoint.

Preparing an Internet-Restricted Environment

To prepare an internet-restricted environment for deploying Tanzu Kubernetes Grid, you can choose one of the following options:

  • A single bootstrap machine for the online and offline environments.

  • Different bootstrap machines for the online and offline environments.

Based on the option that you select, follow the steps described in the following table:

Single Bootstrap Machine Different Bootstrap Machines
Step 1: Install the Isolated Cluster Plugin on the Online Machine Yes
 Yes
Step 2: Download the Images to the Online Machine Yes
 Yes
Step 3: Download the Tanzu CLI Image for Linux to the Online Machine No
 Yes
Step 4: Copy the Files to the Offline Machine No
 Yes
Step 5: Unpack the Files on the Offline Machine No
 Yes
Step 6: Install the Isolated Cluster Plugin on the Offline Machine No
 Yes
Step 7: Log in to the Private Registry on the Offline Machine Yes
 Yes
Step 8: Upload the Images to the Private Registry Yes
 Yes

Step 1: Install the Isolated Cluster Plugin on the Online Machine

Follow this step only if you have not installed the isolated-cluster plugin by running the tanzu plugin sync command on the online machine.

  1. Log in to the Internet-connected Linux bootstrap machine.

  2. Install the isolated-cluster plugin:

    tanzu plugin sync

Step 2: Download the Images to the Online Machine

Important

Before performing this step, ensure that the disk partition where you download the images has 45 GB of available space.

Download the image bundle on your Internet-connected Linux bootstrap machine:

tanzu isolated-cluster download-bundle --source-repo <SOURCE-REGISTRY> --tkg-version <TKG-VERSION> --ca-certificate <SECURITY-CERTIFICATE>

Where:

  • SOURCE-REGISTRY is the IP address or the hostname of the registry where the images are stored.
  • TKG-VERSION is the version of Tanzu Kubernetes Grid that you want to deploy in the proxied or air-gapped environment.

  • SECURITY-CERTIFICATE is the security certificate of the registry where the images are stored. To bypass the security certificate validation, use --insecure, instead of --ca-certificate. Both the strings are optional. If you do not specify any value, the system validates the default server security certificate.

    The following is an example:

    tanzu isolated-cluster download-bundle --source-repo projects.registry.vmware.com/tkg --tkg-version v2.2.0

The image bundle in the form of TAR files, along with the publish-images-fromtar.yaml file, is downloaded to the online machine. The YAML file defines the mapping between the images and the TAR files.

Step 3: Download the Tanzu CLI Image for Linux to the Online Machine

Download the Tanzu CLI image for Linux (ZIP file) to the online machine:

  1. On the online machine, go to VMware Customer Connect and log in with your VMware Customer Connect credentials.
  2. Visit the Tanzu Kubernetes Grid downloads page.
  3. In the VMware Tanzu Kubernetes Grid row, click Go to Downloads.
  4. In the Select Version drop-down, select 2.2.0.

  5. Under Product Downloads, scroll to the section labeled VMware Tanzu CLI 2.2 CLI, and locate VMware Tanzu CLI for Linux.

  6. Click Download Now. The tanzu-cli-bundle-linux-amd64.tar.gz ZIP file is downloaded to the local machine.

  7. (Optional) Verify that your downloaded files are unaltered from the original. VMware provides a SHA-1, a SHA-256, and an MD5 checksum for each download. To obtain these checksums, click Read More under the entry that you want to download. For more information, see Using Cryptographic Hashes.

Step 4: Copy the Files to the Offline Machine

Copy the following files to the offline machine, which is the bootstrap machine in the proxied or air-gapped environment, through a USB thumb drive or other storage medium:

  • The Image TAR files
  • The YAML files
  • The tanzu-cli-bundle-linux-amd64.tar.gz ZIP file

Step 5: Unpack the Files on the Offline Machine

On the offline bootstrap machine, unpack the TAR files into the tanzu directory:

tar -xvf tanzu-cli-bundle-linux-amd64.tar.gz -C $HOME/tanzu
cd $HOME/tanzu/cli
sudo install core/v0.29.0/tanzu-core-linux_amd64 /usr/local/bin/tanzu
tar -xvf tanzu-framework-plugins-standalone-linux-amd64.tar.gz

Step 6: Install the Isolated Cluster Plugin on the Offline Machine

Install the isolated-cluster plugin on the offline bootstrap machine:

tanzu plugin install isolated-cluster --local standalone-plugins/

Step 7: Log in to the Private Registry on the Offline Machine

Log in to the private registry on the offline machine through Docker:

docker login <URL>

Where URL is the URL to the private repository where the images will be stored in the proxied or air-gapped environment.

Step 8: Upload the Images to the Private Registry

Upload the images bundle to the offline machine:

tanzu isolated-cluster upload-bundle --source-directory <SOURCE-DIRECTORY> --destination-repo <DESTINATION-REGISTRY> --ca-certificate <SECURITY-CERTIFICATE>

Where:

  • SOURCE-DIRECTORY is the path to the location where the image TAR files are stored.
  • DESTINATION-REGISTRY is the path to the private registry where the images will be hosted in the air-gapped environment.
  • SECURITY-CERTIFICATE is the security certificate of the private registry where the images will be hosted in the proxied or air-gapped environment. To bypass the security certificate validation, use --insecure, instead of --ca-certificate. Both the strings are optional. If you do not specify any value, the system validates the default server security certificate.

The following is an example:

tanzu isolated-cluster upload-bundle --source-directory ./ --destination-repo hostname1 --ca-certificate /tmp/registryca.crt

What to Do Next

Configure the registry, to where you moved the images, by using the following configuration file reference variables:

  • TKG_CUSTOM_IMAGE_REPOSITORY
  • TKG_CUSTOM_IMAGE_REPOSITORY_CA_CERTIFICATE or TKG_CUSTOM_IMAGE_REPOSITORY_SKIP_TLS_VERIFY.

For more information on these configuration variables, see Private Image Registry Configuration.

Note: You can also use the tanzu config set env.CONFIG-VARIABLE command to configure the registry and persist the environment variables when you use the Tanzu CLI in the future. For more information, see tanzu config set in VMware Tanzu CLI Reference.

Your Internet-restricted environment is now ready for you to deploy or upgrade Tanzu Kubernetes Grid management clusters and start deploying workload clusters on vSphere or AWS.

check-circle-line exclamation-circle-line close-line
Scroll to top icon