Security and Compliance

This topic lists resources for securing Tanzu Kubernetes Grid infrastructure and complying with network security policies.

Ports and Protocols

Networking ports and protocols used by Tanzu Kubernetes Grid are listed in the VMware Ports and Protocols tool.

For each internal communication path, VMware Ports and Protocols lists:

Product
Version
Source
Destination
Ports
Protocols
Purpose
Service Description
Classification (Outgoing, Incoming, or Bidirectional)

You can use this information to configure firewall rules.

Tanzu Kubernetes Grid Firewall Rules

Name	Source	Destination	Service(:Port)	Purpose
workload-to-mgmt	workload cluster network	management cluster network	TCP:6443*	Allow workload cluster to register with management cluster
mgmt-to-workload	management cluster network	workload cluster network	TCP:6443*, 5556	Allow management network to configure workload cluster
allow-mgmt-subnet	management cluster network	management cluster network	all	Allow all internal cluster communication
allow-wl-subnet	workload cluster network	workload cluster network	all	Allow all internal cluster communication
jumpbox-to-k8s	jumpbox IP	management cluster network, workload cluster network	TCP:6443*	Allow Jumpbox to create management cluster and manage clusters.
dhcp	any	NSX: any / no NSX: DHCP IP	DHCP	Allows hosts to get DHCP addresses.
mc-pods-internal	management cluster pod network	.1 address within `SERVICE_CIDR` and `CLUSTER_CIDR`	TCP:	Allows internal traffic from pods on management cluster to Kubernetes API server and pods on workload clusters
to-harbor	management cluster network, workload cluster network, jumpbox IP	Harbor IP	HTTPS	Allows components to retrieve container images
to-vcenter	management cluster network, workload cluster network, jumpbox IP	vCenter IP	HTTPS	Allows components to access vSphere to create VMs and Storage Volumes
dns-ntp-outbound	management cluster network, workload cluster network, jumpbox IP	DNS, NTP servers	DNS, NTP	Core services
ssh-to-jumpbox	any	jumpbox IP	SSH	Access from outside to the jumpbox
For External Identity Provider
allow-pinniped	workload cluster network	management cluster network	TCP:31234	Allow Pinniped concierge on workload cluster to access Pinniped supervisor on management cluster, which may be running behind a `NodePort` or `LoadBalancer` service
bootstrap-allow-pinniped	Bootstrap machine**	management cluster network	TCP:31234	Allow the bootstrap machine to access Pinniped supervisor on management cluster, which may be running behind a `NodePort` or `LoadBalancer` service
For NSX ALB**
avi-nodeport	Avi SE	any workload cluster	TCP:30000-32767	Allow NSX ALB SE (service engine) traffic to pods, for clusters in `NodePort` mode (default) for both L4 and L7 Virtual Services
avi-nodeportlocal	Avi SE	any workload cluster	TCP:61000-62000	Allow NSX ALB SE traffic to pods, for clusters in `NodePortLocal` mode for both L4 and L7 Virtual Services
ako-to-avi	management cluster network, workload cluster network	Avi Controller**	TCP:443	Allow Avi Kubernetes Operator (AKO) and AKO Operator (AKOO) access to Avi Controller
avi-to-nsxt	Avi Controller	VMware NSX (formerly NSX-T)	TCP:443	Allow Avi controller to access VMware NSX, if Avi uses the NSX-T Cloud connector
Default deny-all rule
deny-all	any	any	all	deny by default

*You can override the port 6443 setting with the CLUSTER_API_SERVER_PORT or for environments with NSX Advanced Load Balancer, VSPHERE_CONTROL_PLANE_ENDPOINT_PORT cluster configuration variable.

** Bootstrap machine is where the Tanzu CLI commands are run. It can be a jumpbox (a remote machine to which you establish a connection through SSH), your local machine, or a CI/CD host.

***For additional firewall rules required for NSX Advanced Load Balancer, formerly known as Avi Vantage, see Protocol Ports Used by Avi Vantage for Management Communication in the Avi Networks documentation.

CIS Benchmarking for Clusters

To assess cluster security, you can run Center for Internet Security (CIS) Kubernetes benchmark tests on clusters deployed by Tanzu Kubernetes Grid.

For Tanzu Kubernetes Grid clusters that do not pass all sections of the tests, see explanations and possible remediations in the table below.

Expected Test Failures for CIS Benchmark Inspection on Kubernetes Clusters Provisioned by Tanzu Kubernetes Grid v1.5.1

Section	Test Description	Explanation
1.1.12	Ensure that the `etcd` data directory ownership is set to `etcd:etcd` (Scored)	The data directory (`/var/lib/etcd`) is owned by `root:root`. To provision clusters, Tanzu Kubernetes Grid uses Cluster API which, in turn, uses the `kubeadm` tool to provision Kubernetes. `kubeadm` makes `etcd` run containerized as a static pod, therefore the directory does not need to be set to a particular user. `kubeadm` configures the directory to not be readable by non-root users.
1.2.6	Ensure that the `–kubelet-certificate-authority` argument is set as appropriate (Scored)	This flag is not set by default.
1.2.16	Ensure that the admission control plugin `PodSecurityPolicy` is set (Automated)	This flag is not set by default. Remediation: Using VMware Tanzu Mission Control, you can make the deployments to your clusters more secure by implementing constraints that govern what deployed pods can do. Security policies, implemented using OPA Gatekeeper, allow you to restrict certain aspects of pod execution in your clusters, such as privilege escalation, Linux capabilities, and allowed volume types. See Pod Security Management in the Tanzu Mission Control documentation Alternatively, this admission controller flag can be enabled natively in Kubernetes. For more information, please refer to Pod Security Policies in the VMware Tanzu Developer Center
4.2.6	Ensure that the `–protect-kernel-defaults` argument is set to `true` (Scored)	This flag is set `false` by default. The CIS is concerned that without kernel default protection set, a pod might be run in the cluster that is a mismatch for the security posture of the cluster as a whole. This is a low-likelihood occurrence.
1.2.21	Ensure that the `–profiling` argument is set to `false` (Automated)	This flag is not set by default.
1.3.2	Ensure that the `–profiling` argument is set to `false` (Automated)	This flag is not set by default.
1.4.1	Ensure that the `–profiling` argument is set to `false` (Automated)	This flag is not set by default.

For help remediating these this items, engage with VMware Tanzu support.

For benchmarking clusters in Tanzu Mission Control, see the Expected Test Failures for CIS Benchmark Inspection on Provisioned Tanzu Kubernetes Clusters table under About the CIS Benchmark Inspection and Provisioned Tanzu Kubernetes Clusters, in the Tanzu Mission Control documentation.