This topic describes how to use a Kubernetes Network Policy to secure access to the AWS instance metadata service from Kubernetes clusters created with VMware Tanzu Kubernetes Grid Integrated Edition (TKGI).

Overview

For Pods on TKGI clusters deployed on AWS, you can manage access to the AWS instance metadata service using a Kubernetes Network Policy:

• You can block app access to the instance metadata service for apps in either all cluster namespaces or specific namespaces.
• You can grant app access to the instance metadata service for apps in one or more labeled Pods.

To manage access to the AWS instance metadata service, complete one of the following:

• Secure Access to AWS Instance Metadata for a Namespace
• Secure Access for All Namespaces Using an Antrea Cluster-Wide Network Policy

For information on why you should secure access to AWS instance metadata, see Instance metadata and user data in the AWS documentation.

Secure Access to AWS Instance Metadata for a Namespace

You can use Kubernetes Network Policies to deny access to the AWS instance metadata service by default from all apps in a namespace and if desired, grant access to the service from specific Pods:

• Deny Access from a Specific Namespace
• Grant Access to Specific Apps in a Namespace

Deny Access from a Specific Namespace

To use a Kubernetes Network Policy to deny access to AWS instance metadata by default from a specific namespace:

1. To create a deny Network Policy:

   1. Create a YAML configuration file named np.yml.

   2. Populate the YAML file with one of the following deny Network Policy configurations:

      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: POLICY-NAME
        namespace: NAMESPACE
        annotations:  
          kubernetes.io/ingress.class: "nsx"
      spec:
        podSelector: {}
        policyTypes:
        - Egress
        egress:
        - to:
          - ipBlock:
              cidr: 0.0.0.0/0
              except:
              - 169.254.169.254/32
      

      Where:

      • POLICY-NAME is the name for this Network Policy. For example deny-metadata-access.
      • NAMESPACE is the name of the namespace to apply the Network Policy to. For example, default to manage access from Pods in the default namespace.
   3. Save the YAML configuration file.

2. To apply the deny Network Policy to your cluster:

   kubectl apply -f np.yml
   

   For example:

   
   kubectl apply -f np.yml
   networkpolicy.networking.k8s.io/deny-metadata-access created
   
   

3. Verify the Network Policy has been applied:

   kubectl get networkpolicy
   

   For example:

   
   kubectl get networkpolicy
   NAME                    POD-SELECTOR   AGE deny-metadata-access    <none>         8s 

4. Verify that all apps in Pods in the namespace are blocked from accessing the AWS instance metadata service.

Grant Access to Specific Apps in a Namespace

To configure a Kubernetes Network Policy to grant access to the AWS instance metadata service for apps in a specific Pod:

1. To create an allow Network Policy for apps in a Pod with a specific Pod label:

   1. Create a YAML configuration file named np-allow.yml.

   2. Populate the YAML file with one of the following allow Network Policy configurations:

      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: POLICY-NAME
        namespace: NAMESPACE
        annotations:  
          kubernetes.io/ingress.class: "nsx"
      spec:
        podSelector:
          matchLabels:
            POD-LABEL
        policyTypes:
        - Egress
        egress:
        - to:
          - ipBlock:
              cidr: 169.254.169.254/32        
      

      Where:

      • POLICY-NAME is the name for this Network Policy. For example allow-metadata-access.
      • NAMESPACE is the name of the namespace to apply the Network Policy to. For example, default to manage access from Pods in the default namespace.
      • POD-LABEL is the Pod label for the Pod to grant access to. Only the Pods tagged with the POD-LABEL label are affected by this configuration. For example, app: nginx.
   3. Save the YAML configuration file.

2. To apply the allow Network Policy to your cluster:

   kubectl apply -f np-allow.yml
   

   For example:

   
   kubectl apply -f np-allow.yml
   networkpolicy.networking.k8s.io/allow-metadata-access created 

3. Verify the Network Policy has been applied:

   kubectl get networkpolicy
   

   For example:

   
   kubectl get networkpolicy
   NAME                    POD-SELECTOR   AGE allow-metadata-access   app=nginx      3s 

4. Verify that only the apps in Pod(s) with the specified label have access to AWS instance metadata.

Secure Access for All Namespaces Using an Antrea Cluster-Wide Network Policy

You can use an Antrea cluster-wide Kubernetes Network Policy to manage Pod access to AWS instance metadata.

The benefit of using an Antrea cluster-wide Network Policy is that a single configuration applies to all namespaces, avoiding the need to create a standard Network Policy for each namespace you want to manage.

To manage app access to AWS instance metadata using an Antrea cluster-wide Kubernetes Network Policy:

• Deny Access to All Namespaces Using Antrea
• Allow Access to a Specific Application Using Antrea

For more information on the benefits of using an Antrea Network Policy configuration, see Antrea Network Policy CRDs in the Antrea GitHub repository.

Deny Access to All Namespaces Using Antrea

To deny app access to AWS instance metadata from all cluster namespaces using an Antrea cluster-wide Kubernetes Network Policy:

1. To create a deny Network Policy:

   1. Create a YAML configuration file named np-cluster-deny.yml.

   2. Populate the YAML file with the following deny Network Policy configuration:

      apiVersion: crd.antrea.io/v1alpha1
      kind: ClusterNetworkPolicy
      metadata:
        name: POLICY-NAME
      spec:
        priority: 3          ##### ====> deny access should have lower priority than allow access , or use 'tier' to determine what is taking effect first
        appliedTo:
         - podSelector: {}
        egress:
         - action: Drop
           to:
           - ipBlock:
              cidr: 169.254.169.254/32
      

      Where POLICY-NAME is the name for this Network Policy. For example deny-metadata-access.

   3. Save the YAML configuration file.

2. To apply the deny Network Policy to your cluster:

   kubectl apply -f np-cluster-deny.yml
   

   For example:

   
   kubectl apply -f np-cluster-deny.yml
   clusternetworkpolicy.crd.antrea.io/deny-metadata-access created 

3. Verify the Network Policy has been applied:

   kubectl get clusternetworkpolicies.crd.antrea.io -owide
   

   For example:

   
   kubectl get clusternetworkpolicies.crd.antrea.io -owide
   NAME                    TIER          PRIORITY   DESIRED NODES   CURRENT NODES   AGE deny-metadata-access    application   3          3               3               37s 

4. Verify that apps in all namespaces are blocked from accessing AWS instance metadata.

Allow Access to a Specific App Using Antrea

To configure an Antrea Kubernetes Network Policy to grant access to AWS instance metadata for apps in a specific Pod:

1. To create an Antrea allow Network Policy:

   1. Create a YAML configuration file named np-cluster-allow.yml.

   2. Populate the YAML file with the following allow Network Policy configuration:

      apiVersion: crd.antrea.io/v1alpha1
      kind: ClusterNetworkPolicy
      metadata:
        name: POLICY-NAME
      spec:
        priority: 2
        appliedTo:
          - podSelector:
              matchLabels:
                POD-LABEL
        egress:
         - action: Allow
           to:
             - ipBlock:
                 cidr: 169.254.169.254/32       
      

      Where:

      • POLICY-NAME is the name for this Network Policy. For example allow-metadata-access.
      • POD-LABEL is the Pod label for the Pod to grant access to. Only the Pods tagged with the POD-LABEL label are affected by this configuration. For example, app: nginx.
   3. Save the YAML configuration file.

2. To apply the allow Network Policy to your cluster:

   kubectl apply -f np-cluster-allow.yml
   

   For example:

   
   kubectl apply -f np-cluster-allow.yml
   clusternetworkpolicy.crd.antrea.io/allow-metadata-access created 

3. Verify the Network Policy has been applied:

   kubectl get clusternetworkpolicies.crd.antrea.io -owide
   

   For example:

   
   kubectl get clusternetworkpolicies.crd.antrea.io -owide
   NAME                    TIER          PRIORITY   DESIRED NODES   CURRENT NODES   AGE allow-metadata-access   application   2          1               1               33s 

4. Verify that only the apps in Pod(s) with the specified label have access to AWS instance metadata.