When deploying NCP, it is important to take steps to secure both the Kubernetes and the NSX-T environments.

Restrict NCP to Run Only on Designated Nodes

NCP has access to the NSX-T management plane and should be restricted to run only on designated infrastructure nodes. You can identify these nodes with an appropriate label. A nodeSelector for this label should then be applied to the NCP ReplicationController specification/ For example,

  nodeSelector:
      nsx-infra: True

You can also use other mechanisms, such as affinity, to assign pods to nodes. For more information, see https://kubernetes.io/docs/concepts/configuration/assign-pod-node.

Ensure that the Docker Engine is Up To Date

Docker periodically releases security updates. An automated procedure should be implemented to apply these updates.

Disallow NET_ADMIN and NET_RAW Capabilities of Untrusted Containers

Linux capabilities NET_ADMIN and NET_RAW can be exploited by attackers to compromise the pod network. You should disable these two capabilities of untrusted containers. By default, NET_ADMIN capability is not granted to a non-privileged container. Be wary if a pod specification explicitly enables it or sets the container to be in a privileged mode. In addition, for untrusted containers, disable NET_RAW by specifying NET_RAW in the list of dropped capabilities in the SecurityContext configuration of the container's specification. For example,

    securityContext:
       capabilities:
          drop:
            - NET_RAW
            - ...

Role-Based Access Control

Kubernetes uses Role-Based Access Control (RBAC) APIs to drive authorization decisions, allowing administrators to dynamically configure policies. For more information, see https://kubernetes.io/docs/admin/authorization/rbac.

Typically, the cluster administrator is the only user with privileged access and roles. For user and service accounts, the principle of least privilege must be followed when granting access.

The following guidelines are recommended:

  • Restrict access to Kubernetes API tokens to pods which need them.

  • Restrict access to NCP ConfigMap and NSX API client certificate's TLS secrets to the NCP pod.

  • Block access to Kubernetes networking API from pods that do not require such access.

  • Add a Kubernetes RBAC policy to specify which pods can have access to the Kubernetes API.

Recommended RBAC Policy for the NCP Pod

Create the NCP pod under a ServiceAccount and give this account a minimum set of privileges. Additionally, do not allow other pods or ReplicationControllers to access the ConfigMap and TLS Secrets that are mounted as volumes for the NCP ReplicationController and NSX node agent.

The following example shows how to specify roles and role bindings for NCP:

  Cluster wide role to read, watch and get resources
  ----
  kind: ClusterRole
  # Set the apiVersion to v1 if running with OpenShift
  apiVersion: rbac.authorization.k8s.io/v1beta1
  metadata:
    name: ncp-cluster-role
  rules:
    - apiGroups:
      - ""
      - extensions
      resources:
        - deployments
        - endpoints
        - pods
        - namespaces
        - networkpolicies
        - nodes
        - replicationcontrollers
        - services
      verbs:
        - get
        - watch
        - list
  ----
  Cluster wide role to read, watch, get and modify ingresses
  ----
  kind: ClusterRole
  # Set the apiVersion to v1 if running with OpenShift
  apiVersion: rbac.authorization.k8s.io/v1beta1
  metadata:
    name: ncp-ingress-role
  rules:
    - apiGroups:
      - ""
      - extensions
      resources:
      - ingresses
      verbs:
        - get
        - watch
        - list
        - update
        - patch
    - apiGroups:
      - extensions
      resources:
        - ingresses/status
      verbs:
        - replace
        - update
        - patch
  ----
  Bind roles to ServiceAccount belonging to NCP
  ----
  # Set the apiVersion to v1 if running with OpenShift
  apiVersion: rbac.authorization.k8s.io/v1beta1
  kind: ClusterRoleBinding
  metadata:
    name: ncp-cluster-role-binding
  roleRef:
    # Comment out the apiGroup if running with OpenShift
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: ncp-cluster-role
  subjects:
    - kind: ServiceAccount
      name: ncp-svc-account
      namespace: ncp-deployed-ns-name
  ----
  ----
  # Set the apiVersion to v1 if running with OpenShift
  apiVersion: rbac.authorization.k8s.io/v1beta1
  kind: ClusterRoleBinding
  metadata:
    name: ncp-ingress-role-binding
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: ncp-ingress-role
  subjects:
    - kind: ServiceAccount
      name: ncp-svc-account
      namespace: ncp-deployed-ns-name
Note:

The TLS Secret that is created using the Kubernetes API for the NSX-T client certificate and the private key pair are accessible to any pod that has access to the Kubernetes API server. Similarly, when a pod is created with no service account, it is automatically assigned the default service account in the same namespace which auto mounts the token to access Kubernetes API. Therefore, access to these tokens must be restricted to pods which need them.