Pod Security

This topic explains how to configure Pod Security Admission (PSA) controllers to secure workload clusters deployed by Tanzu Kubernetes Grid (TKG).

For clusters or namespaces within clusters running Kubernetes v1.23 and above, TKG supports applying pod security policies of type privileged, baseline, or restricted via the Pod Security Admission (PSA) controller.

Background: PSA Controllers

PSA controllers in Kubernetes let you configure workload security policies along two axes:

Profiles designate permission levels that workload pod can run at: privileged, baseline, and restricted, from least to most restrictive.
Modes specify actions triggered when an attempt to create the pod violates a profile: warn outputs a warning, audit adds an audit log entry, and enforce blocks creation of the pod.

You can configure PSA controllers to operate over specific namespaces, or cluster-wide. For example, you can configure a cluster to trigger an audit log when a baseline pod violates a security policy, and block a restricted pod if it violates the policy.

For more information, see , Pod Security Standards in the Kubernetes documentation.

Migration from Pod Security Policies

The PSA system replaces Pod Security Policies (PSPs), which were deprecated in Kubernetes v1.21 and removed in v1.25, as described in the Kubernetes v1.25 release notes.

PSPs for nodes were deprecated in TKG v2.1, to reflect their deprecation in Kubernetes; for how to migrate pods from PSPs to the PSA controller, see Migrate from PodSecurityPolicy to the Built-In PodSecurity Admission Controller.

PSA Configuration in TKG

Default PSA settings for a TKG workload clusters depend on the Kubernetes version that it is running:

Kubernetes v1.26 and v1.27, native to TKG v2.3 and v2.4:
- warn and audit modes: restricted
- enforce mode: restricted
Kubernetes v1.25 and v1.24, native to TKG v2.2 and v2.1:
- warn and audit modes: baseline
- enforce mode: no setting

The default PSA settings for clusters running the native Kubernetes versions in TKG 2.4, v2.3, v2.2 and v2.1 ensure that pods continue to run during migration from PSPs to PSA, even if they generate warnings about violating policy:

Beginning with TKG v2.2, all internal TKG components comply to the PSA restricted profile.

Configuring a Cluster-Wide PSA

You can configure a new cluster with cluster-wide PSA or change the PSA of an existing cluster.

Cluster-Wide PSA for a New Cluster

To configure a cluster-wide PSA for a new workload cluster, set POD_SECURITY_STANDARD_DEACTIVATED in its cluster configuration file to false and set the other POD_SECURITY_STANDARD_* variables as described in Pod Security Standards table in the Configuration File Variable Reference.

POD_SECURITY_STANDARD_DEACTIVATED
POD_SECURITY_STANDARD_AUDIT
POD_SECURITY_STANDARD_WARN
POD_SECURITY_STANDARD_ENFORCE

You can also generate the cluster manifest as step 1 of the two-step process described in Create a Class-Based Cluster, and then edit the podSecurityStandard variable values in the Cluster spec as in Change PSA for an Existing Class-Based Cluster below before you create the cluster in step 2.

Change PSA for an Existing Class-Based Cluster

To change the PSA configuration of an existing class-based cluster:

Open the Cluster object spec in an editor:

kubectl edit -namespace NAMESPACE CLUSTER

Edit the values in the podSecurityStandard block under spec.topology.variables:
```
- name: podSecurityStandard

  value:
    deactivated: DEACTIVATED
    audit: AUDIT-PROFILE
    enforce: ENFORCE-PROFILE
    warn: WARN-PROFILE
    auditVersion: AUDIT-VERSION
    enforceVersion: ENFORCE-VERSION
    warnVersion: WARN-VERSION
    exemptions:
      namespaces: [EXEMPT-NS]
```
Where:
- DEACTIVATED is false to apply a cluster-wide PSA and true otherwise
- AUDIT-PROFILE, ENFORCE-PROFILE, and WARN-PROFILE are the PSA profiles for each mode, which can be "privileged", "baseline", or "restricted". Default is "privileged".
- The VERSION values are the Kubernetes version for the three modes, for example "v1.26".
- EXEMPT-NS is a comma-separated list of namespaces to exclude from the PSA control.
Note: The system namespaces kube-system and tkg-system will always get excluded from the PSA control.
After you apply the Cluster object spec, the changes should roll out to the cluster’s control plane nodes.

Change PSA for an Existing Legacy Cluster

To change the PSA configuration of an existing legacy cluster:

Retrieve the current AdmissionConfiguraton spec file from the KubeadmControlPlane object and save it locally, for example:

kubectl get kcp my-cluster-control-plane --template='{{ range $v := .spec.kubeadmConfigSpec.files }}{{ if eq $v.path "/etc/kubernetes/admission-control-config.yaml" }}{{ $v.content }}{{ end }}{{ end }}' > admission-config.yaml

Edit the admission-config.yaml file to the new configuration. For example, the following sets enforce to baseline and keeps audit and warn modes at restricted:

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
configuration:
  apiVersion: pod-security.admission.config.k8s.io/v1beta1
  kind: PodSecurityConfiguration
  defaults:
    enforce: "baseline"
    enforce-version: "v1.24"
    audit: "restricted"
    audit-version: "v1.24"
    warn: "restricted"
    warn-version: "v1.24"
  exemptions:
    usernames: []
    runtimeClasses: []
    namespaces: ["kube-system", "tkg-system"]

Edit the KubeadmControlPlane object spec and replace its array entry at spec.kubeadmConfigSpec.files with our new configuration file.
After you apply the KubeadmControlPlane object spec, the changes should roll out to the cluster’s control plane nodes.

Check Compliance Before Adding or Changing a PSA Configuration

Before you add or change a PSA, prevent unexpected downtime by making sure all existing workloads in the PSA’s cluster or namespace will comply with the new policy settings.

You can assess if pods running in an existing namespace comply with a given policy by running kubectl label --overwrite --dry-run=server:

kubectl label --overwrite --dry-run=server namespace <namespace> pod-security.kubernetes.io/enforce=<policy>

The same information gets written to the audit logs, when configuring the audit mode of Pod Security Admission, or is provided when applying resources to a cluster and configuring the warn mode.

Up-to-date information about the different profiles for pod security is available at the Kubernetes documentation.