VMware Container Networking with Antrea 6th April 2021 | 17808638 Check for additions and updates to these release notes.

What's New

• FIPS 140-2 support for control plane traffic.
• Support for opt-in to the VMware Customer Experience Improvement (CEIP) program.
• RedHat OpenShift Operator support for Antrea.
• Add containerd runtime support for Windows nodes.
• Add NodePort Local feature to improve integration with external load balancers (Alpha).
• Introduce the ClusterGroup CRD to logically group different network endpoints and reference them together in Antrea-native policies.
• Add support for BGP using KubeRouter for Baremetal nodes.
• Add IPv6 support for TraceFlow feature.
• The AntreaProxy feature is graduated from Alpha to Beta and is therefore enabled by default.
• Antrea-native policies (ANP, ACNP) with Tiers & ClusterGroups are graduated from Alpha to Beta.
• The Traceflow feature is graduated from Alpha to Beta and is therefore enabled by default.
• Support for Prometheus metrics is graduated from Alpha to Beta and Antrea metrics are therefore exposed by default.
• Support for IPv6 and dual-stack clusters is graduated from Alpha to Beta.
• Support for audit logging for Antrea-native policy rules.

Compatibility Testing Matrix

K8S Distribution	K8S Versions	OS	Encapsulation
K8s	1.17, 1.18, 1.19, 1.20	Ubuntu 18.04, PhotonOS 3, Debian10, Photon OS3	Geneve,  NoEncap, Hybrid,
AWS EKS	1.17	Amazon Linux 2	Policy Only Mode
Azure AKS, AKS Engine	1.18	Ubuntu 18.04	Policy Only Mode
GKE(Google Kubernetes Engine)	1.18	Ubuntu 18.04	NoEncap, Policy Only Mode

Change Logs 0.13.0- 2021-02-11

Includes all the changes from 0.12.1.

Includes all the changes from 0.11.1.

Added

• Add NodePortLocal feature to improve integration with external load-balancers. (#1459 #1743 #1758, @monotosh-avi@chauhanshubham @hemantavi) [Alpha - Feature Gate: NodePortLocal]
  • Services can be annotated with "nodeportlocal.antrea.io/enabled" to indicate that NodePortLocal should be enabled for this Service's Pod Endpoints.
  • For each container port exposed by such a Pod, the Antrea Agent will allocate a local Node port value and traffic sent to this Node port will be forwarded to the container port using DNAT.
  • The mapping from allocated Node ports to container ports is stored in a new Pod annotation, "nodeportlocal.antrea.io", e.g., to be consumed by external load balancers.
• Introduce the ClusterGroup CRD to logically group different network endpoints and reference them together in Antrea-native policies. (#1782, @abhiraut @Dyanngg)
  • The extra level of indirection enables separation between workload selection and policy definition.
  • ClusterGroups can be referenced in Antrea ClusterNetworkPolicies, either in the AppliedTo or as peers in policy rules (#1750#1734)
  • In addition to the Pod / Namespace selectors and ipBlocks, ClusterGroups can reference a Service by name directly, and all Pod Endpoints for this Service will be included in the ClusterGroup (#1797)
  • ClusterGroups can also select ExternalEntitites, which are used to represent labelled non-Pod endpoints (#1828)
  • The ClusterGroup CRD includes a Status subresource used to indicate whether the Antrea Controller has already computed the membership list for the group (#1778)
  • New APIs are defined in "controlplane.antrea.tanzu.vmware.com/v1beta2": "/clustergroupmembers" retrieves the list of members of a group and "/groupassociations" retrieves the list of groups that a given endpoint (Pod or ExternalEntity) belongs to (#1688)
• Add support for containerd runtime on Windows Nodes. (#1781 #1832, @ruicao93) [Windows]
• Add EndpointSlice support to AntreaProxy. (#1703, @hongliangl) [Alpha - Feature Gate: EndpointSlice]
  • EndpointSlice needs to be enabled in the K8s cluster.
  • Only the "discovery.k8s.io/v1beta1" EndpointSlice API is supported.
• Add support for arm/v7 and arm64 by providing Antrea Docker images for these architectures. (#1771, @antoninbas)
  • Refer to the documentation for instructions on how to use the image.
• Support IPv6 packets in Traceflow. (#1579, @gran-vmv)
• Add the following Prometheus metrics to the the AntreaProxy implementation: "antrea_proxy_sync_proxy_rules_duration_seconds", "antrea_proxy_total_endpoints_installed", "antrea_proxy_total_endpoints_updates", "antrea_proxy_total_services_installed", "antrea_proxy_total_services_updates". (#1704, @weiqiangt)
• Add the following Prometheus metrics to count Status updates for Antrea-native policies: "antrea_controller_acnp_status_updates", "antrea_controller_anp_status_updates". (#1801, @antoninbas)
• Add support for TLS between the Antrea Agent FlowExporter and the FlowAggregator, using self-signed certificates. (#1649, @zyiou)
• New Antrea Agent configuration option, "kubeAPIServerOverride", which can be used to explicitly provide an address for the K8s apiserver when the Agent is running as Pod; by default, the Agent uses the ClusterIP for the kubernetes Service. (#1735, @anfernee)
• Provide ability to configure TLS cipher suites supported by the Antrea apiservers (Agent and Controller). (#1784, @lzhecheng)
• Add liveness probe to Antrea Controller to ensure it is automatically restarted after a while by kubelet if it stops being responsive. (#1839, @antoninbas)
• Document workaround to install OVS and Antrea on Windows Nodes for which the CPU does not have the required virtualization capabilities, as may be the case for cloud VMs. (#1744, @ruicao93) [Windows]
• Improve documentation for "noEncap" and "hybrid" traffic modes, and add information about how to use Kube-router to advertise Pod CIDRs to the fabric with BGP. (#1798, @jianjuns)
• Add new NetworkPolicy testsuite based on auto-generated test cases. (#1765, @mattfenwick)

Changed

• Change permissions for the "/var/run/antrea" directory created by the Antrea Agent on each Node to prevent non-root users from accessing it; among other things, it includes the socket file used to send CNI commands to the Agent. (#1770, @jianjuns)
• Add multi-table support to the "antctl get ovsflows" command, to dump flows from multiple tables at once. (#1708, @weiqiangt)
• Change the sanity check performed by the Antrea Agent to validate that the Hyper-V dependency is satisfied. (#1741, @ruicao93)
• Periodically verify that the static iptables rules required by Antrea are present and install missing rules if any. (#1751, @siddhant94)
• Update Mellanox/sriovnet dependency to version v1.0.2 to support OVS hardware offload to Mellanox devices with Kernel versions 5.8 and above. (#1845, @Mmduh-483)
• Remove dependency on juju libraries, which are distributed under an LGPL v3 license. (#1796, @antoninbas)

Fixed

• Ensure that NodePort traffic does not bypass NetworkPolicies. (#1816, @tnqn)
  • NodePort traffic for which ExternalTrafficPolicy is set to Cluster goes through SNAT before NetworkPolicies are enforced; after SNAT the source IP is the IP of the local gateway interface (antrea-gw0)
  • Users will need to define the appropriate NetworkPolicies to allow ingress access to isolated Pods for NodePort traffic
  • This new behavior only applies to Linux Nodes using the OVS system datapath (default)
• When clearing the flow-restore-wait config for the OVS bridge after re-installing flows, ensure that the operation happened successfully and retry if anything unexpected happens; if flow-restore-wait is not cleared, the bridge will not forward packets correctly. (#1730, @tnqn)
• Stop mounting the host's kmod binary to the Antrea initContainer as it may depend on shared libraries not available in the container. (#1777, @antoninbas)
• Fix crashes in the FlowAggregator, along with numerous spurious warnings, by updating the version of the go-ipfix library. (#1817, @zyiou @srikartati)
• Fix issues with reference logstash configuration and improve reference Kibana dashboards for flow visualization with the FlowExporter feature. (#1727, @zyiou)