This site will be decommissioned on December 31st 2024. After that date content will be available at techdocs.broadcom.com.

 

VMware Container Networking with Antrea. December 21, 2021 | 19065941

Check for additions and updates to these release notes.

What's New

  • Antrea to NSX-T integration
    • K8s Objects like Namespaces, Pods, Services & K8s and Antrea Network policies are collected and reported to NSX-T inventory
    • Antrea Cluster Network Policies can be created in the NSX-T Distributed Firewall UI and deployed into Antrea enabled K8s clusters as Antrea Cluster Network policies
    • Antrea Traceflow and log collection can be triggered from the NSX-T UI
    • Antrea cluster health status is reported to NSX-T for customers' attentions.
    • Details about this new integration including prerequisites, version requirements, and deployment steps can be found in the NSX-T 3.2.0 documentation
    • The integration deploys NSX-T adapters as Pod to the managed K8s cluster. The container image for the NSX-T adapters can be pulled from VMware distribution Harbor. To use the image from VMware distribution Harbor, edit the deployment manifest and replace all container images with one of the following URLs.
      • projects.registry.vmware.com/antreainterworking/interworking-debian:0.2.0
      • projects.registry.vmware.com/antreainterworking/interworking-ubuntu:0.2.0
      • projects.registry.vmware.com/antreainterworking/interworking-photon:0.2.0
  • Antrea OpenShift support and Antrea K8s Operator
    • This release adds the support for OpenShift through the Antrea K8s Operator. For supported versions, please review the Compatibility Testing Matrix bellow
    • The Operator image is available on VMware distribution Harbor. Customer can pull the image from below URL.
      • projects.registry.vmware.com/antreainterworking/antrea-operator:v1.2.3_vmware
    • The Antrea images can be pulled from the below URLs.
      • projects.registry.vmware.com/antreainterworking/antrea-standard-debian:v1.2.3_vmware.3
      • projects.registry.vmware.com/antreainterworking/antrea-advanced-debian:v1.2.3_vmware.3
      • projects.registry.vmware.com/antreainterworking/antrea-ubi:v1.2.3_vmware.3
  • Checksum offloading can now be globally disabled through a config parameter. This can be used if deployments experience issues caused e.g. by faulty NIC drivers. Refer to known issues section for details.

Compatibility Testing Matrix

K8S Distribution K8S Versions OS Encapsulation
K8s  1.18, 1.19, 1.20, 1.21 Ubuntu 18.04, PhotonOS 3, Debian10, Photon OS3 Geneve,  NoEncap, Hybrid,
AWS EKS 1.17 Amazon Linux 2 Policy Only Mode
Azure AKS, AKS Engine 1.18 Ubuntu 18.04 Policy Only Mode
GKE(Google Kubernetes Engine) 1.18 Ubuntu 18.04, Windows

NoEncap, Policy Only Mode
OpenShift OCP 4.7 & 4.8 RHCOS and RHEL  Geneve,  NoEncap, Hybrid

Change Logs 1.2.3- 2021-09-24

Includes all the changes from 1.2.0 and 1.2.1

Includes all the changes from 1.1.

Includes all the changes from  1.0.

Changed

  • Support returning partial supportbundle results when some Nodes fail to respond. (#2788, @hangyan)
  • Remove restriction that only GRE tunnels can be used when enabling IPsec: VXLAN can also be used, and so can Geneve (if the Linux kernel version for the Nodes is recent enough). (#2764, @luolanzone)
  • Reduce memory usage of antctl when collecting supportbundle. (#2821, @tnqn)

Fixed

  • Fix nil pointer error when collecting a supportbundle on a Node for which the antrea-agent container image does not include "iproute2"; this does not affect the standard antrea/antrea-ubuntu container image. (#2789, @liu4480)
  • When creating an IPsec OVS tunnel port to a remote Node, handle the case where the port already exists but with a stale config graciously: delete the existing port first, then recreate it. (#2765, @luolanzone)
  • Fix panic in the Antrea Controller when it processes ClusterGroups that are used by multiple ClusterNetworkPolicies. (#2768, @tnqn)
  • Fix nil pointer error when antrea-agent updates OpenFlow priorities of Antrea-native policies without Service ports. (#2758, @wenyingd)
  • Fix Pod-to-Service access on Windows when the Endpoints are not non-hostNetwork Pods (e.g. the kubernetes Service). (#2702, @wenyingd) [Windows]
  • Fix container network interface MTU configuration error when using containerd as the runtime on Windows. (#2773, @wenyingd) [Windows]

Known Issues

The known issues are grouped as follows.

    Known Issues

    • Container Network Traffic Throughput Drops to Zero on Buggy Physical NIC

      Antrea enables Geneve tunnel checksum offload by default. However, sometimes the container networking traffic throughput drop to nearly zero. In packet capture we see that TCP 3-way handshake is successful but the first data packet in MTU size gets wrong checksum and it's dropped in the receiver side. This can happen when the K8s node VMs are running on overlay network and the underlay network cannot correctly process checksum offloading in double encapsulation scenario, or the physical NIC has bug in checksum offloading.

      We introduced the following ConfigMap antrea-agent-tweaker-g56hc6fh8t in antrea.yml to allow disabling tunnel checksum offloading.

      apiVersion: v1
data:
  antrea-agent-tweaker.conf: |-
    # Enable disableUdpTunnelOffload will disable udp tunnel offloading feature on kubernetes node's default interface.
    # By default, no actions will be taken.
    disableUdpTunnelOffload: false
kind: ConfigMap
metadata:
  labels:
    app: antrea
  name: antrea-agent-tweaker-g56hc6fh8t
  namespace: kube-system

      This is only for Linux. You can use kubectl to edit the live ConfigMap on K8s API to disable tunnel checksum offload, then restart all Antrea agents (usually run the command kubectl delete pod -l component=antrea-agent -n kube-system) to make the option effective. You can also edit this ConfigMap in antrea.yml before deploying Antrea.
      We suggest not to set it to true only if you hit tunnel checksum offloading issue. Disabling tunnel checksum offloading drops container networking throughput by about 50%.

    • Realization of Network Policies takes longer after upgrade to 1.3.0-1.2.2 with large number (10,000+) of Antrea-Network Policy Custom Resources.

      When Antrea is upgraded to 1.3.0-1.2.2 from an older release (1.2.0-0.13.x or older), Antrea controller automatically mirrors Antrea Network Policy to new API group 'antrea.io'. So when a large number of ANP CRs are present, the Antrea Network policies are enforced slowly as the ANP CR is being mirrored to the new API. This is observed with more than 10,000+ ANPs.

      Workaround: Manually mirror old API group to new API group before upgrade

      1. Create the new CRDs of tiers.crd.antrea.io, networkpolicies.crd.antrea.io, clusternetworkpolicies.crd.antrea.io, clustergroups.crd.antrea.io. The CRDs can be fetched from the manifest yaml that you will use for upgrade.
      2. Dump the resources of the deprecated CRDs (with *.antrea.tanzu.vmware.com suffix), create the same resources with new CRDs (with *.antrea.io suffix):
      kubectl get tiers.security.antrea.tanzu.vmware.com -o yaml -A | sed "s/security.antrea.tanzu.vmware.com/crd.antrea.io/g" | kubectl apply -f -
kubectl get networkpolicies.security.antrea.tanzu.vmware.com -o yaml -A | sed "s/security.antrea.tanzu.vmware.com/crd.antrea.io/g" | kubectl apply -f -
kubectl get clusternetworkpolicies.security.antrea.tanzu.vmware.com -o yaml -A | sed "s/security.antrea.tanzu.vmware.com/crd.antrea.io/g" | kubectl apply -f -
kubectl get clustergroups.core.antrea.tanzu.vmware.com -o yaml -A | sed "s/core.antrea.tanzu.vmware.com/crd.antrea.io/g" | kubectl apply -f -

      Note, if there are no resources for a CRD, the command will get an error output "error: no objects passed to apply", which can be ignored.

    check-circle-line exclamation-circle-line close-line
    Scroll to top icon