Installing VMware Container Networking with Antrea in networkPolicyOnly Mode

In networkPolicyOnly mode, Antrea runs as a secondary CNI and enforces Kubernetes network policies, Antrea network policies, and Antrea Cluster network policies. Pod IP management and network connectivity are provided by the primary CNI.

If your Kubernetes cluster is set up with other CNI plugins, you can install Antrea in networkPolicyOnly mode. If the Kubernetes cluster has no CNI plugins deployed, it is recommended that you install Antrea as the primary CNI.

For more information, see Running Antrea in networkPolicyOnly Mode.

Planning the Installation

If you have an existing Kubernetes cluster with a routed CNI other than Antrea, and want to use Antrea's NetworkPolicy features, you can deploy Antrea in networkPolicyOnly mode. Antrea networkPolicyOnly mode is compatible with CNIs with routed traffic model.

Note that uninstalling VMware Container Networking with Antrea in networkPolicyOnly mode is not supported.

Installation Steps

From your browser, go to the VMware Antrea download page. Download an Antrea commercial release of version 1.8.0 or later. The networkPolicyOnly all-in-one yaml file can be found in Debian Image and Deployment Manifests in the downloaded files.
Deploy the Antrea networkPolicyOnly all-in-one yaml in the Kubernetes cluster. For example, run the following command (note that the yaml file name contains "nponly"):
```
kubectl apply -f antrea-advanced-nponly-v1.13.0+vmware.1.yml
```
To verify that Antrea is deployed successfully, run the following command and make sure that the Antrea controller and agent Pods are in the Running state.
```
kubectl get po -A -owide | grep antrea
```
Restart all Pods (except hostNetwork Pods). The existing Pods require a restart to be managed by Antrea in networkPolicyOnly mode. The following sample script will delete all Pods and recreate them. The non-persistent data in the old Pods will not be kept, and the new Pods may get different IP addresses. The workload is unavailable during the recreation process.
```
for it in `kubectl get pods -A -o json | jq -c '.items[] | select(.spec.hostNetwork!=true and .status.phase!="Succeeded") | [.metadata.namespace, .metadata.name]'`; do
    namespace=`echo $it | jq -r '.[0]'`
    name=`echo $it | jq -r '.[1]'`
    kubectl get pod $name -n $namespace -o yaml | kubectl replace --force -f -
done
```
We recommend deploying Antrea in networkPolicyOnly mode before deploying any production workloads. If Antrea in networkPolicyOnly mode is deployed on a production cluster, we recommend performing a rolling update on existing workloads to minimize the impact on availability.
(Optional) Deploy Antrea-NSX adapter. For more information, see the NSX documentation Integration of Kubernetes Clusters with Antrea CNI.

After the installation, you can check network connectivity with the following steps:

Create a busybox Pod and access TTY. Run the following command:

kubectl run -n default busybox --image=busybox --image-pull-policy=IfNotPresent --restart=Never -it

Check the IP configuration with the command ip a.
Check network connectivity with the command ping <PodIP/ExternalIP>.
Exit and delete the busybox Pod.

Troubleshooting

After applying the all-in-one yaml, if there are problems, you can troubleshoot with the following steps:

Log in to to the worker node.
Go to the CNI folder with the command cd /etc/cni/net.d.
List the installed CNI conflist with the command ls -l and make sure that 05-antrea.conflist exists and appears at the top of the list.
View the content of 05-antrea.conflist. Make sure that the previous CNIs are in this file and Antrea is appended at the end.

Integration of Antrea with NSX

See Integration of Antrea Container Clusters.

The supported Antrea-NSX features are:

Register/de-register cluster to NSX
Manage Antrea Cluster Network Policies from NSX
Send stats/alarms/events to NSX
Traceflow
Support bundle

Limitation:

Traceflow to an external IP address may produce unexpected result.