Single-Node Clusters on vSphere

Tanzu Kubernetes Grid (TKG) supports single-node clusters. Single-node clusters are workload clusters on which hosted workloads run alongside control plane infrastructure on a single ESXi host.

To further minimize the footprint of a single-node cluster, you can create it from a tiny Tanzu Kubernetes release (TKr), which has a Photon or Ubuntu Tiny OVA for its base OS. Such clusters are called minimal single-node clusters.

Single-node clusters are class-based workload clusters that run on vSphere and are deployed by standalone management clusters.

Single-node clusters are fully supported for Telco Cloud Automation (TCA).

Note
You cannot use Tanzu Mission Control (TMC) to create and manage single-node clusters, but this capability is planned for a future release of TMC.

Single-node cluster use cases include:

Minimal single-node clusters
- Resource-constrained host environments such as far edge cell sites and other edge locations
Standard single-node clusters
- Experimental development and test environments

Compatibility

Single-node clusters are supported for the following environments and components:

Category	Supported Options
Infrastructure	vSphere 7, vSphere 8
Node OS	Ubuntu 20.04, Photon 3
Node size	small
Package	Cert Manager, Fluent Bit, Multus, Prometheus, Whereabouts
Control plane endpoint provider	Kube-Vip^*
Workload load balancer	Kube-Vip^*
Workload cluster type	Class-based
CNI	Antrea, Calico
Connectivity mode	Online, Internet-restricted

^*NSX Advanced Load Balancer (ALB) is not supported with single-node clusters in TKG v2.3.

Prerequisites

A vSphere account
A TKG standalone management cluster running TKG v2.3.x on vSphere
A bootstrap machine with the following installed, as described in Install the Tanzu CLI and Kubernetes CLI for Use with Standalone Management Clusters:
- Tanzu CLI
- kubectl

Create a Minimal Single-Node Cluster

To create a single-node workload cluster on vSphere that uses a tiny Tanzu Kubernetes release (TKr) to minimize its footprint:

Prepare the OVA:
1. Go to the Broadcom Support Portal and log in with your VMware customer credentials.
2. Go to the Tanzu Kubernetes Grid downloads page.
3. In the version drop-down, select 2.3.1.
4. Download the Tiny OVA to use for your single-node cluster:
  - Photon v3 Kubernetes v1.26.8 Tiny OVA
  - Ubuntu 2004 Kubernetes v1.26.8 Tiny OVA
5. Import the Tiny OVA into your vSphere environment and convert it into a VM template as described in Import the Base Image Template into vSphere.

Create the single-node workload cluster.

Note
To create minimal single-node clusters, you must run the tanzu cluster create command with a Kubernetes-style object spec. If you start with a flat cluster configuration file, you must follow a two-step process described in Create a Class-Based Cluster to generate the object spec, and then edit it as described below before running tanzu cluster create a second time to create the cluster.

Set environment variables as set in this example:

export CLUSTER_NAME='workload-snc'
export CLUSTER_NAMESPACE='default'
export CLUSTER_CIDR='100.96.0.0/11'
export SERVICE_CIDR='100.64.0.0/13'
export VSPHERE_CONTROL_PLANE_ENDPOINT=10.185.11.134
export VSPHERE_SERVER=10.185.12.154
export VSPHERE_USERNAME='[email protected]'
export VSPHERE_PASSWORD=<encoded:QWRtaW4hMjM=>
export VSPHERE_DATACENTER='/dc0'
export VSPHERE_DATASTORE='/dc0/datastore/sharedVmfs-0'
export VSPHERE_FOLDER='/dc0/vm'
export VSPHERE_NETWORK='/dc0/network/VM Network'
export VSPHERE_RESOURCE_POOL='/dc0/host/cluster0/Resources'
export VSPHERE_SSH_AUTHORIZED_KEY=ssh-rsa AAAAB3[...]tyaw== [email protected]
export VSPHERE_TLS_THUMBPRINT=47:F5:83:8E:5D:36:[...]:72:5A:89:7D:29:E5:DA
export VSPHERE_CONTROL_PLANE_NUM_CPUS='2'
export VSPHERE_CONTROL_PLANE_MEM_MIB='4096'
export VSPHERE_CONTROL_PLANE_DISK_GIB='20'
export TKG_CUSTOM_IMAGE_REPOSITORY='projects.registry.vmware.com/tkg'
export OS_NAME='photon'
export TKG_CUSTOM_IMAGE_REPOSITORY_CA_CERTIFICATE="LS0tL[...]0tLQo="

Create a manifest vsphere-snc.yaml with Cluster and Secret object specs referencing the above variables:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  annotations:
    tkg.tanzu.vmware.com/cluster-controlplane-endpoint: ${VSPHERE_CONTROL_PLANE_ENDPOINT}
    run.tanzu.vmware.com/resolve-tkr: 'tkr.tanzu.vmware.com/tiny'
  labels:
    tkg.tanzu.vmware.com/cluster-name: ${CLUSTER_NAME}
  name: ${CLUSTER_NAME}
  namespace: ${CLUSTER_NAMESPACE}
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - ${CLUSTER_CIDR}
    services:
      cidrBlocks:
      - ${SERVICE_CIDR}
  topology:
    class: tkg-vsphere-default-v1.1.0
    controlPlane:
      metadata:
        annotations:
          run.tanzu.vmware.com/resolve-os-image: image-type=ova,os-name=${OS_NAME}
      replicas: 1
    variables:
    - name: controlPlaneTaint
      value: false
    - name: auditLogging
      value:
        enabled: false
    - name: apiServerEndpoint
      value: ${VSPHERE_CONTROL_PLANE_ENDPOINT}
    - name: aviAPIServerHAProvider
      value: false
    - name: imageRepository
      value:
        host: ${TKG_CUSTOM_IMAGE_REPOSITORY}
    - name: trust
      value:
        additionalTrustedCAs:
        - data: ${TKG_CUSTOM_IMAGE_REPOSITORY_CA_CERTIFICATE}
          name: imageRepository
    - name: vcenter
      value:
        cloneMode: fullClone
        datacenter: ${VSPHERE_DATACENTER}
        datastore: ${VSPHERE_DATASTORE}
        folder: ${VSPHERE_FOLDER}
        network: ${VSPHERE_NETWORK}
        resourcePool: ${VSPHERE_RESOURCE_POOL}
        server: ${VSPHERE_SERVER}
        storagePolicyID: ""
        tlsThumbprint: ${VSPHERE_TLS_THUMBPRINT}
    - name: user
      value:
        sshAuthorizedKeys:
        - ${VSPHERE_SSH_AUTHORIZED_KEY}
    - name: controlPlane
      value:
        machine:
          diskGiB: ${VSPHERE_CONTROL_PLANE_DISK_GIB}
          memoryMiB: ${VSPHERE_CONTROL_PLANE_MEM_MIB}
          numCPUs: ${VSPHERE_CONTROL_PLANE_NUM_CPUS}
    version: v1.26.8+vmware.1-tiny.2
---
apiVersion: v1
kind: Secret
metadata:
  name: ${CLUSTER_NAME}
  namespace: ${CLUSTER_NAMESPACE}
stringData:
  password: ${VSPHERE_PASSWORD}
  username: ${VSPHERE_USERNAME}
EOF

Note the following:

The metadata.annotations setting for run.tanzu.vmware.com/resolve-tkr
The topology.variables setting for controlPlaneTaint
There is no topology.workers block, only topology.controlPlane
For prior versions of TKG, the topology.version should be v1.26.5+vmware.1-tiny.2 for v2.3.1, v1.25.7+vmware.1-tiny.1 for v2.2.0, v1.24.10+vmware.1-tiny.1 for v2.1.1 and v1.24.9+vmware.1-tiny.2 for v2.1.0.

(Optional) To configure the cluster to use Calico as the CNI instead of the default Antrea CNI, follow the instructions for single-node clusters in Calico CNI for Supervisor or Single-Node Class-Based Workload Clusters.

Apply the Cluster object manifest:

tanzu cluster create -f vsphere-snc.yaml

Create a Standard Single-Node Cluster

To create a single-node workload cluster on vSphere that uses a standard Photon or Ubuntu TKr:

Create a flat configuration file for the workload cluster as described in vSphere with Standalone Management Cluster Configuration Files.
Run tanzu cluster create with the --dry-run flag to convert the flat configuration file into a Kubernetes-style Cluster object spec as described in Create an Object Spec.
Edit the Cluster object spec to include the following settings:
- Under topology.controlPlane:
```
replicas: 1
```
- No topology.workers block; if present, delete it.
- Under topology.variables:
```
- name: controlPlaneTaint
  value: false
```
- (Optional) to set maximum pod counts, under topology.variables include max-pods settings as extra arguments for the control plane or worker kubelets. For example, to set both control plane and worker pod counts to a maximum of 254:
```
- name: controlPlaneKubeletExtraArgs
  value:
    max-pods: "254"
- name: workerKubeletExtraArgs
  value:
    max-pods: "254"
```
Run tanzu cluster create with the modified Cluster object spec as described in Create a Class-Based Cluster from the Object Spec.