Security and Compliance

This topic lists resources for securing Tanzu Kubernetes Grid infrastructure and complying with network security policies.

Ports and Protocols

Networking ports and protocols used by Tanzu Kubernetes Grid are listed in the VMware Ports and Protocols tool.

For each internal communication path, VMware Ports and Protocols lists:

  • Product
  • Version
  • Source
  • Destination
  • Ports
  • Protocols
  • Purpose
  • Service Description
  • Classification (Outgoing, Incoming, or Bidirectional)

You can use this information to configure firewall rules.

Tanzu Kubernetes Grid Firewall Rules

Name Source Destination Service(:Port) Purpose
workload-to-mgmt workload cluster network management cluster network TCP:6443* Allow workload cluster to register with management cluster
mgmt-to-workload management cluster network workload cluster network TCP:6443*, 5556 Allow management network to configure workload cluster
allow-mgmt-subnet management cluster network management cluster network all Allow all internal cluster communication
allow-wl-subnet workload cluster network workload cluster network all Allow all internal cluster communication
jumpbox-to-k8s jumpbox IP management cluster network, workload cluster network TCP:6443* Allow Jumpbox to create management cluster and manage clusters.
dhcp any NSX: any / no NSX: DHCP IP DHCP Allows hosts to get DHCP addresses.
to-harbor management cluster network, workload cluster network, jumpbox IP Harbor IP HTTPS Allows components to retrieve container images
to-vcenter management cluster network, workload cluster network, jumpbox IP vCenter IP HTTPS Allows components to access vSphere to create VMs and Storage Volumes
dns-ntp-outbound management cluster network, workload cluster network, jumpbox IP DNS, NTP servers DNS, NTP Core services
ssh-to-jumpbox any jumpbox IP SSH Access from outside to the jumpbox
For External Identity Provider
allow-pinniped workload cluster network management cluster network TCP:31234 Allow Pinniped concierge on workload cluster to access Pinniped supervisor on management cluster, which may be running behind a NodePort or LoadBalancer service
bootstrap-allow-pinniped Bootstrap machine** management cluster network TCP:31234 Allow the bootstrap machine to access Pinniped supervisor on management cluster, which may be running behind a NodePort or LoadBalancer service
For NSX ALB***
avi-nodeport Avi SE any workload cluster TCP:30000-32767 Allow NSX ALB SE (service engine) traffic to pods, for clusters in NodePort mode (default) for both L4 and L7 Virtual Services
avi-nodeportlocal Avi SE any workload cluster TCP:61000-62000 Allow NSX ALB SE traffic to pods, for clusters in NodePortLocal mode for both L4 and L7 Virtual Services
ako-to-avi management cluster network, workload cluster network Avi Controller** TCP:443 Allow Avi Kubernetes Operator (AKO) and AKO Operator (AKOO) access to Avi Controller
Default deny-all rule
deny-all any any all deny by default

*You can override the port 6443 setting with the CLUSTER_API_SERVER_PORT or for environments with NSX Advanced Load Balancer, VSPHERE_CONTROL_PLANE_ENDPOINT_PORT cluster configuration variable.

** Bootstrap machine is where the Tanzu CLI commands are run. It can be a jumpbox (a remote machine to which you establish a connection through SSH), your local machine, or a CI/CD host.

***For additional firewall rules required for NSX Advanced Load Balancer, formerly known as Avi Vantage, see Protocol Ports Used by Avi Vantage for Management Communication in the Avi Networks documentation.

CIS Benchmarking for Clusters

To assess cluster security, you can run Center for Internet Security (CIS) Kubernetes benchmark tests on clusters deployed by Tanzu Kubernetes Grid.

For Tanzu Kubernetes Grid clusters that do not pass all sections of the tests, see explanations and possible remediations in the table below.

Expected Test Failures for CIS Benchmark Inspection on Kubernetes Clusters Provisioned by Tanzu Kubernetes Grid v1.5.1

Section Test Description Explanation
1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)

The data directory (/var/lib/etcd) is owned by root:root.

To provision clusters, Tanzu Kubernetes Grid uses Cluster API which, in turn, uses the kubeadm tool to provision Kubernetes.

kubeadm makes etcd run containerized as a static pod, therefore the directory does not need to be set to a particular user.

kubeadm configures the directory to not be readable by non-root users.

1.2.6 Ensure that the –kubelet-certificate-authority argument is set as appropriate (Scored)

This flag is not set by default.

1.2.16 Ensure that the admission control plugin PodSecurityPolicy is set (Automated)

This flag is not set by default.

Remediation:

Using VMware Tanzu Mission Control, you can make the deployments to your clusters more secure by implementing constraints that govern what deployed pods can do. Security policies, implemented using OPA Gatekeeper, allow you to restrict certain aspects of pod execution in your clusters, such as privilege escalation, Linux capabilities, and allowed volume types. See Pod Security Management in the Tanzu Mission Control documentation

Alternatively, this admission controller flag can be enabled natively in Kubernetes. For more information, please refer to Pod Security Policies in the VMware Tanzu Developer Center

4.2.6 Ensure that the –protect-kernel-defaults argument is set to true (Scored)

This flag is set false by default.

The CIS is concerned that without kernel default protection set, a pod might be run in the cluster that is a mismatch for the security posture of the cluster as a whole. This is a low-likelihood occurrence.

1.2.21 Ensure that the –profiling argument is set to false (Automated)

This flag is not set by default.

1.3.2 Ensure that the –profiling argument is set to false (Automated)

This flag is not set by default.

1.4.1 Ensure that the –profiling argument is set to false (Automated)

This flag is not set by default.

For help remediating these this items, engage with VMware Tanzu support.

For benchmarking clusters in Tanzu Mission Control, see the Expected Test Failures for CIS Benchmark Inspection on Provisioned Tanzu Kubernetes Clusters table under About the CIS Benchmark Inspection and Provisioned Tanzu Kubernetes Clusters, in the Tanzu Mission Control documentation.

check-circle-line exclamation-circle-line close-line
Scroll to top icon