Before setting up OpenShift and NCP, take note of the following information.
The names of OpenShift Namespace, Pod and Network Policy resource, OpenShift Cluster (as specified in ncp.ini) and Node, must not be longer than 40 characters.
Label key and value must not be longer than 20 and 40 characters, respectively.
A pod must have no more than 11 labels and a namespace must have no more than 12 labels.
Labels added for OpenShift internal usage, for example, a label with prefix openshift.io in its key, will be disregarded by NCP and thus user won't see the corresponding tags created on the related NSX resources. Here is a list of label prefixes used by OpenShift, and you should avoid using a label key starting with any of the following:
The nodes will need access to the pods, for example, for Kubelet health-checks. Make sure the host management interface is able to access the pod network.
Linux capabilities NET_ADMIN and NET_RAW can be exploited by attackers to compromise the pod network. You should disable these two capabilities of untrusted containers. By default, with restricted and anyuid SCC, NET_ADMIN is not granted. Be wary of any SCC that enables NET_ADMIN explicitly, or enables the pod to run in privileged mode. In addition, for untrusted containers, create a separate SCC based on, for example, anyuid SCC, with NET_RAW capability removed. This can be done by adding NET_RAW to `requiredDropCapabilities` list in the SCC definition.
Allow root access in PODs/Containers (only for testing). Commands below will require root access in all PODs of the oc project you are currently logged in to.
oc new-project test-project oc project test-project oadm policy add-scc-to-user anyuid -z default
Configure (add) the OpenShift Registry.
oc login -u system:admin -n default oadm registry --service-account=registry --config=/etc/origin/master/admin.kubeconfig
Delete the OpenShift Registry
oc login -u system:admin -n default oc delete svc/docker-registry dc/docker-registry
There's a missing IPtables firewall rule to allow DNS requests from the Docker default bridge containers to the dnsmasq process on the Node. It needs to be opened manually. Edit /etc/sysconfig/iptables and add the following Rules at the bottom of the file before COMMIT:
-A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 53 -j ACCEPT -A OS_FIREWALL_ALLOW -p udp -m state --state NEW -m udp --dport 53 -j ACCEPT COMMIT
Restart iptables, docker and origin-node (restarts kube-proxy and kubelet).
systemctl restart iptables systemctl restart docker systemctl restart origin-node
The internal docker registry of OpenShift needs to be allowed to use non-TLS for OpenShift to work. Normally this should be added automatically by the OpenShift Ansible installer, but it seems that this is currently not working. Edit /etc/sysconfig/docker and add:
systemctl restart docker
Configure (add) the OpenShift routers (HA-Proxy N/S LBs).
oc login -u system:admin -n default oadm router router --replicas=2 --service-account=router
Delete the created routers.
oc login -u system:admin -n default oc delete svc/router dc/router
Create a sample Ruby based 2 tier apps.
oc login -u system:admin -n default oc oc new-project nsx oc process -n openshift mysql-ephemeral -v DATABASE_SERVICE_NAME=database | oc create -f - oc new-app centos/ruby-22-centos7~https://github.com/openshift/ruby-hello-world.git oc expose service ruby-hello-world oc env dc database --list | oc env dc ruby-hello-world -e -