For Tanzu Kubernetes Grid (TKG) with a standalone management cluster, this topic lists resources for securing infrastructure and complying with network security policies.
For TKG with a Supervisor, see vSphere with Tanzu Security in the vSphere 8 documentation.
Networking ports and protocols used by Tanzu Kubernetes Grid are listed in the VMware Ports and Protocols tool.
For each internal communication path, VMware Ports and Protocols lists:
You can use this information to configure firewall rules.
| Name | Source | Destination | Service(:Port) | Purpose |
|---|---|---|---|---|
| workload-to-mgmt | workload cluster network | management cluster network | TCP:6443* | Allow workload cluster to register with management cluster |
| mgmt-to-workload | management cluster network | workload cluster network | TCP:6443*, 5556 | Allow management network to configure workload cluster |
| allow-mgmt-subnet | management cluster network | management cluster network | all | Allow all internal cluster communication |
| allow-wl-subnet | workload cluster network | workload cluster network | all | Allow all internal cluster communication |
| jumpbox-to-k8s | jumpbox IP | management cluster network, workload cluster network | TCP:6443* | Allow Jumpbox to create management cluster and manage clusters. |
| dhcp | any | NSX: any / no NSX: DHCP IP | DHCP | Allows hosts to get DHCP addresses. |
| mc-pods-internal | management cluster pod network | .1 address within SERVICE_CIDR and CLUSTER_CIDR |
TCP: | Allows internal traffic from pods on management cluster to Kubernetes API server and pods on workload clusters |
| to-harbor | management cluster network, workload cluster network, jumpbox IP | Harbor IP | HTTPS | Allows components to retrieve container images |
| to-vcenter | management cluster network, workload cluster network, jumpbox IP | vCenter IP | HTTPS | Allows components to access vSphere to create VMs and Storage Volumes |
| dns-ntp-outbound | management cluster network, workload cluster network, jumpbox IP | DNS, NTP servers | DNS, NTP | Core services |
| ssh-to-jumpbox | any | jumpbox IP | SSH | Access from outside to the jumpbox |
| For External Identity Provider | ||||
| allow-pinniped | workload cluster network | management cluster network | TCP:31234 | Allow Pinniped concierge on workload cluster to access Pinniped supervisor on management cluster, which may be running behind a NodePort or LoadBalancer service |
| bootstrap-allow-pinniped | Bootstrap machine** | management cluster network | TCP:31234 | Allow the bootstrap machine to access Pinniped supervisor on management cluster, which may be running behind a NodePort or LoadBalancer service |
| For NSX ALB** | ||||
| avi-nodeport | Avi SE | any workload cluster | TCP:30000-32767 | Allow NSX ALB SE (service engine) traffic to pods, for clusters in NodePort mode (default) for both L4 and L7 Virtual Services |
| avi-nodeportlocal | Avi SE | any workload cluster | TCP:61000-62000 | Allow NSX ALB SE traffic to pods, for clusters in NodePortLocal mode for both L4 and L7 Virtual Services |
| ako-to-avi | management cluster network, workload cluster network | Avi Controller** | TCP:443 | Allow Avi Kubernetes Operator (AKO) and AKO Operator (AKOO) access to Avi Controller |
| avi-to-nsxt | Avi Controller | VMware NSX (formerly NSX-T) | TCP:443 | Allow Avi controller to access VMware NSX, if Avi uses the NSX-T Cloud connector |
| Default deny-all rule | ||||
| deny-all | any | any | all | deny by default |
*You can override the port 6443 setting with the CLUSTER_API_SERVER_PORT or for environments with NSX Advanced Load Balancer, VSPHERE_CONTROL_PLANE_ENDPOINT_PORT cluster configuration variable.
** Bootstrap machine is where the Tanzu CLI commands are run. It can be a jumpbox (a remote machine to which you establish a connection through SSH), your local machine, or a CI/CD host.
***For additional firewall rules required for NSX Advanced Load Balancer, formerly known as Avi Vantage, see Protocol Ports Used by Avi Vantage for Management Communication in the Avi Networks documentation.
To assess cluster security, you can run Center for Internet Security (CIS) Kubernetes benchmark tests on clusters deployed by Tanzu Kubernetes Grid.
For benchmarking clusters in Tanzu Mission Control, see the Expected Test Failures for CIS Benchmark Inspection on Provisioned Tanzu Kubernetes Clusters table under About the CIS Benchmark Inspection and Provisioned Tanzu Kubernetes Clusters, in the Tanzu Mission Control documentation.
