Create a node pool on the ESXi host with the STS card.

Prerequisites

Workload clusters are already deployed.

Procedure

  1. Navigate to Infrastructure > CaaS Infrastructure > Cluster Instances in the TCA UI.
  2. Select the workload cluster where you want to deploy the Node Pool.
  3. Under Worker Nodes within the workload cluster, add a Node Pool and select the ESXi host with the STS card.

    Note: Ensure that a label is added to the node pool. Labels are required for instantiating the timing pod.


    Node Pool Details

What to do next

After creating the node pool, do the following tasks:

  1. Log in to the workload cluster and verify that the node is labeled. For example, 'telco.vmware.com.node-restriction.kubernetes.io/sts-silicom=true'.

    Sample output of 'kubectl get nodes': 
capv@wc-sts2-new-master-control-plane-rcfnd [ ~ ]$ kubectl get nodes -A
NAME STATUS ROLES AGE VERSION
wc-sts2-new-master-control-plane-rcfnd Ready control-plane,master 25d v1.23.8+vmware.2
wc-sts2-new-np1-766b448bdd-kdngf Ready <none> 25d v1.23.8+vmware.2
wc-sts2-new-np2-7b5949fc5-79vnk Ready <none> 23h v1.23.8+vmware.2
capv@wc-sts2-new-master-control-plane-rcfnd [ ~ ]$ kubectl get nodes wc-sts2-new-np2-7b5949fc5-79vnk --show-labels
NAME STATUS ROLES AGE VERSION LABELS
wc-sts2-new-np2-7b5949fc5-79vnk Ready <none> 23h v1.23.8+vmware.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=vsphere-vm.cpu-32.mem-52gb.os-photon,beta.kubernetes.io/os=linux,key=value,kubernetes.io/arch=amd64,kubernetes.io/hostname=wc-sts2-new-np2-7b5949fc5-79vnk,kubernetes.io/os=linux,node.kubernetes.io/instance-type=vsphere-vm.cpu-32.mem-52gb.os-photon,telco.vmware.com.node-restriction.kubernetes.io/sts-silicom=true,telco.vmware.com/nodepool=wc-sts2-new-np2
capv@wc-sts2-new-master-control-plane-rcfnd [ ~ ]$

  2. On the mgmt cluster, verify that the PCI groups and associated PCI devices appear in the output of esxinfoprofile and esxinfo.

    Sample outputs of 'esxinfoprofile' and 'esxinfo':
capv@mc-sts2-new-master-control-plane-846jv [ ~ ]$ kubectl get esxinfoprofile -n tca-system sts2-ptp-vf -o yaml
apiVersion: acm.vmware.com/v1alpha1
kind: EsxInfoProfile
metadata:
  creationTimestamp: "2022-09-16T18:57:56Z"
  generation: 4
  name: sts2-ptp-vf
  namespace: tca-system
  resourceVersion: "18633363"
  uid: 4b21c68e-e230-4cf3-91c8-110e40ccf8fb
spec:
  pciGroups:
  - filters:
    - alias: vmnic13
      sriovEnabled: true
    name: ptp8
  - filters:
    - alias: vmnic12
      sriovEnabled: true
    name: ptp7
  - filters:
    - alias: vmnic11
      sriovEnabled: true
    name: ptp6
  - filters:
    - alias: vmnic10
      sriovEnabled: true
    name: ptp5
  - filters:
    - deviceIds:
      - "0x1591"
      indices:
      - 0
      vendorId: "0x8086"
    name: ptp1
  - filters:
    - alias: vmnic9
      sriovEnabled: true
    name: ptp4
  - filters:
    - alias: vmnic8
      sriovEnabled: true
    name: ptp3
  - filters:
    - alias: vmnic7
      sriovEnabled: true
    name: ptp2
  - filters:
    - alias: vmnic[0-9]
      sriovEnabled: true
    name: sriov
  - filters:
    - deviceIds:
      - "0xd5c"
      sriovEnabled: true
      vendorId: "0x8086"
    name: acc100
  - filters:
    - deviceIds:
      - "0x1591"
      - "0x1592"
      - "0x1593"
      - "0x1599"
      - "0x159A"
      - "0x159B"
      sriovEnabled: true
      vendorId: "0x8086"
    - sriovEnabled: false
    name: ptp
  reservation:
    minCoresForCPUReservationPerNumaNode: 3
    reservedCoresPerNumaNode: 1
    reservedMemoryPerNumaNode: 512
capv@mc-sts2-new-master-control-plane-846jv [ ~ ]$

capv@mc-sts2-new-master-control-plane-846jv [ ~ ]$ kubectl get esxinfo -A
NAMESPACE    NAME                          AGE
tca-system   10.208.66.98                  25d
tca-system   w2-hs6-h0211.eng.vmware.com   25d
capv@mc-sts2-new-master-control-plane-846jv [ ~ ]$ kubectl get esxinfo -n tca-system w2-hs6-h0211.eng.vmware.com -o yaml
apiVersion: acm.vmware.com/v1alpha2
kind: EsxInfo
metadata:
  creationTimestamp: "2022-09-16T18:57:56Z"
  generation: 9
  name: w2-hs6-h0211.eng.vmware.com
  namespace: tca-system
  resourceVersion: "18633367"
  uid: f80a57d9-b51f-40ef-b681-fc01a620b219
spec:
  profile: sts2-ptp-vf
  revision: "1665429735"
  vcSecret: vc-p1016.eng.vmware.com
status:
  conditions:
  - lastTransitionTime: "2022-10-10T19:22:10Z"
    message: EsxInfo is updated with spec revision 1665429735
    reason: StatusUpToDate
    status: "True"
    type: Ready
  observedGeneration: 9
  pciGroups:
  - name: ptp8
    pciIds:
    - deviceId: "0X1591"
      id: 0000:88:00.7
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  - name: ptp7
    pciIds:
    - deviceId: "0X1591"
      id: 0000:88:00.6
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  - name: ptp6
    pciIds:
    - deviceId: "0X1591"
      id: 0000:88:00.5
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  - name: ptp5
    pciIds:
    - deviceId: "0X1591"
      id: 0000:88:00.4
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  - name: ptp1
    pciIds:
    - deviceId: "0X1591"
      id: 0000:88:00.0
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  - name: ptp4
    pciIds:
    - deviceId: "0X1591"
      id: 0000:88:00.3
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  - name: ptp3
    pciIds:
    - deviceId: "0X1591"
      id: 0000:88:00.2
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  - name: ptp2
    pciIds:
    - deviceId: "0X1591"
      id: 0000:88:00.1
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  - name: sriov
    pciIds:
    - deviceId: "0X159B"
      id: 0000:5e:00.1
      name: Ethernet Controller E810-XXV for SFP
      numa: 0
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.1
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.2
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.3
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.4
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.5
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.6
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.7
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  - name: acc100
    pciIds: []
  - name: ptp
    pciIds:
    - deviceId: "0X159B"
      id: 0000:5e:00.1
      name: Ethernet Controller E810-XXV for SFP
      numa: 0
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.1
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.2
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.3
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.4
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.5
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.6
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.7
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
    - deviceId: "0X1591"
      id: 0000:88:00.0
      name: Ethernet Controller E810-C for backplane
      numa: 1
      vendorId: "0X8086"
  profileGeneration: 4
  profileName: sts2-ptp-vf
  reservation:
    minCoresForCPUReservationPerNumaNode: 3
    reservedCoresPerNumaNode: 1
    reservedMemoryPerNumaNode: 512
  revision: "1665429735"
capv@mc-sts2-new-master-control-plane-846jv [ ~ ]$