Using VMware Tanzu Mission Control, you can run preconfigured cluster inspections using Sonobuoy, an open source community standard.

Note: The cluster inspection features of Tanzu Mission Control are not available in Tanzu Mission Control Essentials. Also, the CIS Benchmark inspection type is only available in Tanzu Mission Control Advanced.
The following cluster inspections are available from the Overview and Inspection tabs of the cluster detail page in the Tanzu Mission Control console.
  • The Conformance inspection validates the binaries running on your cluster and ensures that your cluster is properly installed, configured, and working. You can view the generated report from within Tanzu Mission Control to assess and address any issues that arise. For more information, see the Kubernetes Conformance documentation at https://github.com/cncf/k8s-conformance/tree/master/docs.
  • The CIS benchmark inspection evaluates your cluster against the CIS Benchmark for Kubernetes published by the Center for Internet Security. This inspection type is available in the advanced version of Tanzu Mission Control.
  • The Lite inspection is a node conformance test that validates whether nodes meet requirements for Kubernetes. For more information, see Validate node setup in the Kubernetes documentation.

Because the cluster inspections provide a point-in-time report of the condition of the cluster, you might want to run them periodically (to avoid drifting out of conformance) and any time you make significant alterations, such as after you patch or upgrade a cluster.

From the Inspections page in the Tanzu Mission Control console, you can view a list of the most recent inspections that have been run against all the clusters in your organization, along with the results of those inspections. This page also allows you to start a new inspection.

About Lite and Conformance Inspections and Worker Node Taints

If you attempt to run a Lite or Conformance inspection on a cluster with more than one worker node taint, the inspection fails. This is because Sonobuoy's end-to-end test framework checks the cluster's status prior to running tests. If any worker node in the cluster has more than one taint, this pre-check determines that the inspection cannot be scheduled. For more information, see this FAQ in the Sonobuoy documentation.

About the CIS Benchmark Inspection and Provisioned Tanzu Kubernetes Clusters

The CIS Benchmark for Kubernetes is a set of opinionated and generalized tests that assess vulnerabilities in a Kubernetes implementation. In many implementations, including Tanzu Kubernetes clusters provisioned through Tanzu Mission Control, the vulnerabilities tested for in the benchmark can be mitigated in ways that are not detected by the inspection and result in a failure of some tests. For this reason, you can expect to see a failed status for a particular set of tests when you run the CIS benchmark inspection on a provisioned cluster. The following table shows the CIS benchmark test failures that occur in clusters provisioned through Tanzu Mission Control.

Table 1. Expected Test Failures for CIS Benchmark Inspection on Provisioned Tanzu Kubernetes Clusters
Section Test Description Explanation
1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)

The data directory (/var/lib/etcd) is owned by root:root.

This is not an issue on clusters provisioned through Tanzu Mission Control.

To provision clusters, Tanzu Mission Control uses Cluster API which, in turn, uses the kubeadm tool to provision Kubernetes.

kubeadm makes etcd run containerized as a static pod, therefore the directory does not need to be set to a particular user.

kubeadm configures the directory to not be readable by non-root users.

1.2.6 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)

This flag is not set.

The kubelet server certificate is used only when running kubectl exec or kubectl logs or gathering metrics. Tanzu Mission Control configures worker nodes to exist in private subnets in AWS, and therefore "man in the middle" attacks require host networking to be compromised.

Though unlikely, VMware views this as a potential security vulnerability and is tracking the issue in the Cluster API community.

1.2.33 Ensure that the --encryption-provider-config argument is set as appropriate (Scored) This flag is not set.

Tanzu Mission Control provides isolation for each cluster using AWS VPCs and security groups. This level of isolation significantly reduces the likelihood of a security breach involving access to an unencrypted volume.

1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored) This argument is set to true in clusters running Kubernetes version 1.12 and later. To avoid this error, upgrade your cluster to a newer version of Kubernetes.
4.1.3 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)

In clusters provisioned through Tanzu Mission Control, this issue is mitigated by running the kube-proxy as a daemonset, which does not use an on-disk kubeconfig.

Therefore, the audit cannot be performed, as the file is mounted as a config map in the kube-proxy pod and is not accessible to the CIS Inspection pod.

4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)

In clusters provisioned through Tanzu Mission Control, this issue is mitigated by running the kube-proxy as a daemonset, which does not use an on-disk kubeconfig.

Therefore, the audit cannot be performed, as the file is mounted as a config map in the kube-proxy pod and is not accessible to the CIS Inspection pod.

4.2.4 Ensure that the --read-only-port argument is set to 0 (Scored) In clusters provisioned through Tanzu Mission Control, this issue is mitigated by disabling the read-only port.
4.2.6 Ensure that the --protect-kernel-defaults argument is set to true (Scored) This flag is not set.

The CIS is concerned that without kernel default protection set, a pod might be run in the cluster that is a mismatch for the security posture of the cluster as a whole. This is a low-likelihood occurrence.

4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)

In clusters provisioned through Tanzu Mission Control, this issue is mitigated by disabling the read-only port.

By default, the TLS certificate and private key files are readable only by root.

4.2.12 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored) This argument is set to true in clusters running Kubernetes 1.12 and later. To avoid this error, upgrade your cluster to a newer version of Kubernetes.

About the CIS Benchmark Inspection and Attached Clusters

Some cloud providers do not allow access to the control plane node of clusters running in their platform (for example, GKE and AKS). Because of this limitation, the CIS benchmark inspection cannot run all tests on clusters running in these environments. Therefore there are fewer results from running a CIS benchmark inspection on an attached cluster running in such an environment.