This topic lists resources for securing Tanzu Kubernetes Grid infrastructure.

Ports and Protocols

Networking ports and protocols used by Tanzu Kubernetes Grid are listed in the VMware Ports and Protocols tool.

For each internal communication path, VMware Ports and Protocols lists:

  • Product
  • Version
  • Source
  • Destination
  • Ports
  • Protocols
  • Purpose
  • Service Description
  • Classification (Outgoing, Incoming, or Bidirectional)

You can use this information to configure firewall rules.

Tanzu Kubernetes Grid Firewall Rules

Name Source Destination Service Purpose
workload-to-mgmt tkg-workload-cluster-network tkg-management-cluster-network TCP:6443* Allow workload cluster to register with management cluster
mgmt-to-workload tkg-management-cluster-network tkg-workload-cluster-network TCP:6443*, 5556 Allow management network to configure workload cluster
allow-pinniped tkg-workload-cluster-network tkg-management-cluster-network TCP:31234 Allow Pinniped concierge on workload cluster to access Pinniped supervisor on management cluster, which may be running behind a NodePort or LoadBalancer service
allow-mgmt-subnet tkg-management-cluster-network tkg-management-cluster-network all Allow all internal cluster communication
allow-wl-subnet tkg-workload-cluster-network tkg-workload-cluster-network all Allow all internal cluster communication
jumpbox-to-k8s Jumpbox IP tkg-management-cluster-network, tkg-workload-cluster-network TCP:6443* Allow Jumpbox to create management cluster and manage clusters.
dhcp any NSX-T: any / no NSX-T: DHCP IP DHCP Allows hosts to get DHCP addresses.
to-harbor tkg-management-cluster-network, tkg-workload-cluster-network, jumpbox IP Harbor IP HTTPS Allows components to retrieve container images
to-vcenter tkg-management-cluster-network, tkg-workload-cluster-network, jumpbox IP vCenter IP HTTPS Allows components to access vSphere to create VMs and Storage Volumes
dns-ntp-outbound tkg-management-cluster-network, tkg-workload-cluster-network, jumpbox IP DNS, NTP servers DNS, NTP Core services
ssh-to-jumpbox any Jumpbox IP SSH Access from outside to the jumpbox
(For NSX ALB**) ako-to-avi tkg-management-cluster-network, tkg-workload-cluster-network Avi Controller** TCP:443 Allow Avi Kubernetes Operator (AKO) and AKO Operator (AKOO) access to Avi Controller
deny-all any any all deny by default

*With NSX Advanced Load Balancer, you can override the port 6443 setting with the VSPHERE_CONTROL_PLANE_ENDPOINT_PORT cluster configuration variable.

**For additional firewall rules required for NSX Advanced Load Balancer, formerly known as Avi Vantage, see Protocol Ports Used by Avi Vantage for Management Communication in the Avi Networks documentation.

CIS Benchmarking for Clusters

To assess cluster security, you can run Center for Internet Security (CIS) Kubernetes benchmark tests on clusters deployed by Tanzu Kubernetes Grid.

For Tanzu Kubernetes Grid clusters that do not pass all sections of the tests, see explanations and possible remediations in the table below.

Expected Test Failures for CIS Benchmark Inspection on Kubernetes Clusters Provisioned by Tanzu Kubernetes Grid v1.3.1

Section Test Description Explanation
1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)

The data directory (/var/lib/etcd) is owned by root:root.

To provision clusters, Tanzu Kubernetes Grid uses Cluster API which, in turn, uses the kubeadm tool to provision Kubernetes.

kubeadm makes etcd run containerized as a static pod, therefore the directory does not need to be set to a particular user.

kubeadm configures the directory to not be readable by non-root users.

1.2.6 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)

This flag is not set by default.

1.2.16 Ensure that the admission control plugin PodSecurityPolicy is set (Automated)

This flag is not set by default.

Remediation:

Using VMware Tanzu Mission Control, you can make the deployments to your clusters more secure by implementing constraints that govern what deployed pods can do. Security policies, implemented using OPA Gatekeeper, allow you to restrict certain aspects of pod execution in your clusters, such as privilege escalation, Linux capabilities, and allowed volume types. See Pod Security Management in the Tanzu Mission Control documentation.

Alternatively, this admission controller flag can be enabled natively in Kubernetes. For more information, please refer to Pod Security Policies in the VMware Tanzu Developer Center.

4.2.6 Ensure that the --protect-kernel-defaults argument is set to true (Scored)

This flag is not set by default.

The CIS is concerned that without kernel default protection set, a pod might be run in the cluster that is a mismatch for the security posture of the cluster as a whole. This is a low-likelihood occurrence.

1.2.21 Ensure that the --profiling argument is set to false (Automated)

This flag is not set by default.

1.3.2 Ensure that the --profiling argument is set to false (Automated)

This flag is not set by default.

1.4.1 Ensure that the --profiling argument is set to false (Automated)

This flag is not set by default.

For help remediating these this items, engage with VMware Tanzu support.

For benchmarking clusters in Tanzu Mission Control, see the Expected Test Failures for CIS Benchmark Inspection on Provisioned Tanzu Kubernetes Clusters table under About the CIS Benchmark Inspection and Provisioned Tanzu Kubernetes Clusters, in the Tanzu Mission Control documentation.

check-circle-line exclamation-circle-line close-line
Scroll to top icon