Workflow for Provisioning TKG Clusters Using Kubectl

Follow this workflow to provision a TKG Service cluster declaratively using kubectl commands and a cluster specification defined in YAML.

This workflow supports provisioning a TKG cluster declaratively using kubectl and YAML.

Prerequisites

Verify or complete the following prerequisites before starting the provisioning workflow:

Install or update your environment to the latest Supervisor version. See Running TKG Service Clusters.
Create or update a content library with the latest Tanzu Kubernetes releases. See Administering Kubernetes Releases for TKG Service Clusters.
Create and configure a vSphere Namespace for hosting TKG clusters. See Configuring vSphere Namespaces for Hosting TKG Service Clusters.

Procedure

Install the Kubernetes CLI Tools for vSphere.

Install the Kubernetes CLI Tools for vSphere.
Authenticate with Supervisor using kubectl.
```
kubectl vsphere login --server=SUPERVISOR-CONTROL-PLANE-IP-ADDRESS-or-FQDN --vsphere-username USERNAME
```
Note: FQDN for Supervisor can only be used if enabled. Refer to the Supervisor configuration documentation for details.
Verify successful login to the Supervisor.
You should see a message similar to the following:
```
Logged in successfully.

You have access to the following contexts:
   192.197.2.65
   tkg2-cluster-namespace
```
Where 192.197.2.65 is the Supervisor context and tkg2-cluster-namespace is the context for the vSphere Namespace where you plan to provision the TKG cluster.

Verify that the target vSphere Namespace is the current context.

kubectl config get-contexts

CURRENT   NAME                    CLUSTER         AUTHINFO                                        NAMESPACE
          192.197.2.65            192.197.2.65    wcp:10.197.154.65:[email protected]
*         tkg2-cluster-namespace  10.197.154.65   wcp:10.197.154.65:[email protected]   tkg2-cluster-namespace

If the target vSphere Namespace is not the current context, switch to it.

kubectl config use-context tkg2-cluster-namespace

List the VM classes that are available in the vSphere Namespace.
```
kubectl get virtualmachineclass
```
You can only use VM classes bound to the target namespace. If you do not see any VM classes, verify that the default VM classes have been associated with the vSphere Namespace. See also Troubleshoot VM Class Errors.
Get the available persistent volume storage classes.
```
kubectl describe namespace VSPHERE-NAMESPACE-NAME
```
The command returns details about the namespace, including the storage class in the form tkg2-storage-policy.storageclass.storage.k8s.io/requests.storage. The first token of the string is the storage class name, in this example tkg2-storage-policy. The command kubectl describe storageclasses also returns available storage classes, but requires vSphere administrator permissions.
List the available Tanzu Kubernetes releases.
You can use either of the following commands to perform this operation:
```
kubectl get tkr
```
```
kubectl get tanzukubernetesreleases
```
This command returns the TKRs that are available to you in this vSphere Namespace, and shows their compatibility with the Supervisor you are deploying on. You can only use those releases that are returned by this command. If you do not see any releases, or the releases you want, verify that you have done the following: a) Created a TKR content library; b) Synchronized the content library with the desired OVA files; and c) Associated the content library with the vSphere Namespace where you are provisioning TKG cluster.
Craft the YAML file for provisioning the TKG cluster.
1. Determine the type of cluster you will create and review its API and features:
  - TanzuKubernetesCluster: Using the TanzuKubernetesCluster v1alpha3 API
  - Cluster: Using the Cluster v1beta1 API
2. Start with one of the example YAMLs for provisioning the cluster.
  For example:
  - v1alpha3 Example: Default TanzuKubernetesCluster
  - v1beta1 Example: Default Cluster
3. Save your YAML file as tkg2-cluster-1.yaml, or similar.
4. Populate the YAML using the information you gleaned from the output of the preceding commands, including:
  - The name of the cluster, such as tkg2-cluster-1
  - The target vSphere Namespace
  - Bound VM classes, such as guaranteed-medium
  - Storage class for cluster nodes and persistent volumes
  - The number of control plane and worker nodes (replicas)
  - The Tanzu Kubernetes release specified by the TKR NAME string, such as v1.25.7+vmware.3-fips.1-tkg.1
5. Customize the TGK cluster YAML as needed.
  - Add separate volumes for high-churn components, such as containerd
  - Specify a default persistent storage class for cluster nodes and persistent volumes
  - Customize cluster networking, including the CNI, pod and service CIDRs
6. Use a YAML syntax checker and verify that the YAML is valid.
The result of this step is a valid YAML for provisioning the TKG cluster.

Provision the TKG cluster by running the following command.

kubectl apply -f tkg2-cluster-1.yaml

Expected result:

tanzukubernetescluster.run.tanzu.vmware.com/tkg2-cluster-1 created

Monitor the provisioning of the TKG cluster.

kubectl get tanzukubernetesclusters

kubectl get tkc

Of, if you create a Cluster using the v1beta1 API:

kubectl get cluster

Initially the READY status is False as the cluster is being provisioned. After a few minutes it should be True.

NAME              CONTROL PLANE   WORKER   TKR NAME                            AGE   READY   TKR COMPATIBLE   UPDATES AVAILABLE
tkg2-cluster-1    3               6        v1.25.7+vmware.3-fips.1-tkg.1       49m   True    True

Run additional commands to view details about the cluster.

kubectl get tanzukubernetescluster,cluster,virtualmachinesetresourcepolicy,virtualmachineservice,kubeadmcontrolplane,machinedeployment,machine,virtualmachine

kubectl describe tanzukubernetescluster tkg2-cluster-1

Monitor the deployment of cluster nodes using the vSphere Client.
In the vSphere inventory for Hosts and Clusters, you should see the cluster node VMs deployed in the target vSphere Namespace.

Once all TKG cluster nodes are in a READY state, log in to the cluster using the vSphere Plugin for kubectl.

kubectl vsphere login --server=SUPERVISOR-CONTROL-PLANE-IP-ADDRESS-or-FQDN \
--vsphere-username USERNAME \
--tanzu-kubernetes-cluster-name CLUSTER-NAME \
--tanzu-kubernetes-cluster-namespace NAMESPACE-NAME

For example:

kubectl vsphere login --server=192.197.2.65 --vsphere-username [email protected] \
--tanzu-kubernetes-cluster-name tkg2-cluster-1 --tanzu-kubernetes-cluster-namespace tkg2-cluster-namespace

Note: The log in command will only succeed once the control plane nodes are running and the authentication service plugin has started. If worker nodes are in the process of being created, log in may spotty. It is recommended that you log in once all cluster nodes are READY.

Switch context to the TKG cluster so that it is the current context.
On successful login to the TKG cluster, you should see a message similar to the following.
```
Logged in successfully.

You have access to the following contexts:
   192.197.2.65
   tkg2-cluster-namespace
   tkg2-cluster-1
```
Where 192.197.2.65 is the Supervisor context, tkg2-cluster-namespace is the vSphere Namespace context, and tkg2-cluster-1 is the TKG cluster context.
Switch to the TKG cluster context.
```
kubect config use-context tkg2-cluster-1
```

Check TKG cluster resources.

kubectl get nodes

kubectl get namespaces

kubectl get pods -A

kubectl cluster-info

kubectl api-resources

Exercise the TKG cluster by deploying a test pod and verify that it works as expected.

See Test TKG Cluster Provisioning Using Kubectl.