Set up and run NCP and NSX node agent.

Procedure

  1. Edit roles/ncp/defaults/main.yaml and specify the OpenShift API server IP, NSX manager IP, and URL'sfor downloading NCP ReplicationController yaml and nsx-node-agent DaemonSet yaml.
  2. From the openshift-ansible-nsx directory, run the ncp role:
        ansible-playbook -i /PATH/TO/HOSTS/hosts ncp.yaml

Results

The ncp role performs the following steps:

  • Check if nsx-system project exists, and create one if it does not.

        oc new-project nsx-system
  • Download the ncp-rbac yaml file and change the apiVersion to v1.

  • Create the service account for the NCP pod, create a cluster role that specifies resources that NCP can access and bind the cluster role with the NCP service account.

  • Create the service account for the nsx-node-agent pod, create a cluster role that specifies the resources that the node agent can access and bind the cluster role with the node agent service account.

        oc apply -f /tmp/ncp-rbac.yml
  • Obtain the token associated with the above service accounts, and store it under /etc/nsx-ujo/<service_account>_token:

        secret=`kubectl get serviceaccount ncp-svc-account -o yaml | grep -A1 secrets | tail -n1 | awk {'print $3'}`
        kubectl get secret $secret -o yaml | grep 'token:' | awk {'print $2'} | base64 -d > /etc/nsx-ujo/ncp_token
        secret=`kubectl get serviceaccount nsx-node-agent-svc-account -o yaml | grep -A1 secrets | tail -n1 | awk {'print $3'}`
        kubectl get secret $secret -o yaml | grep 'token:' | awk {'print $2'} | base64 -d > /etc/nsx-ujo/node_agent_token
  • Download the SecurityContextConstraint (SCC) yaml file ncp-os-scc.yml for NCP and create the SCC based on the yaml.

        oc apply -f ncp-os-scc.yml

    The SCC yaml file specifies SELinux type as spc_t to ensure that NCP/nsx-node-agent has access permissions under SELinux. That is,

        seLinuxContext:
          seLinuxOptions:
          type: spc_t

    In the SCC yaml file, under seLinuxContext's seLinuxOptions, the SELinux label-based access control level is set to s0:c0:c1023 to allow ncp/node-agent access to targets from different file categories.

  • Add the SCC to the user who creates the NCP and NSX node agent pods. For example, to add the SCC to the default user in the current project:

        oc adm policy add-scc-to-user ncp-scc -z default
  • Add the SCC to the NCP and NSX node agent service accounts:

        oc adm policy add-scc-to-user ncp-scc -z ncp-svc-account
        oc adm policy add-scc-to-user ncp-scc -z nsx-node-agent-svc-account
  • Download the yaml files for NCP ReplicationController (RC) and nsx-node-agent DaemonSet (DS) and update the ConfigMap.

  • Download and load the NCP image (nsx-node-agent uses the same image).

  • Configure the service account and set up the required SecurityContextConstraint for NCP and nsx_node_agent.

  • Create the NCP ReplicationController and nsx-node-agent DaemonSet.

Note:

NCP opens persistent HTTP connections to the Kubernetes API server to watch for life cycle events of Kubernetes resources. If an API server failure or a network failure causes NCP's TCP connections to become stale, you must restart NCP so that it can re-establish connections to the API server. Otherwise, NCP will miss the new events.