VMware NSX Container Plugin 4.1.0.3 Release Notes

VMware NSX Container Plugin 4.1.0.3 | 21 SEP2023 | Build 22377648 Check for additions and updates to these release notes.

What's New

NSX Container Plugin 4.1.0.3 is an update release that resolves issues found in earlier releases. For other details about this release, see NSX Container Plugin 4.1.0 Release Notes.

Resolved Issues

• Issue 3256562: NCP updates NSX IPSet with empty IP address after restart or leader election

  When NCP becomes the master instance after a restart or leader election, NCP will do initialization with Kubernetes to update NSX objects. In Manager mode, NCP uses NSX IPSet as source or destination in firewall rules. Occasionally, NCP may handle NetworkPolicy before the runtime Pod store is ready, and NCP will update IPSet with an empty IP address. This will cause Pod traffic to stop working.

  You can see that there is an empty IP address by running the following CLI command on the node where the NCP master instance is running.

  $ /var/vcap/jobs/ncp/bin/nsxcli -c "get ncp-store ip_set_store"

  Workaround: Restarting NCP may fix this problem. To avoid NCP re-sync with Kubernetes when leader election happens, increase the HA master_timeout value in the NCP ConfigMap.

  [ha]
  enable = True
  master_timeout = 60

• Issue 3242478: nsx-node-agent cannot enter its own network namespace to establish hyperbus channel

  Occasionally, when nsx-node-agent starts, it cannot enter its own network namespace to establish hyperbus channel. The nsx-node-agent CLI and the hyperbus CLI on ESX will report that the hyperbus channel status is down. New pods cannot run on the Kubernetes node. The nsx-node-agent log will have messages such as the following:

  2023-06-29T11:48:04.645Z cba56c49-2eed-4cf7-af2f-f34cf619f00e NSX 5506 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="WARNING"] oslo.privsep.daemon privsep log: File "/usr/local/lib/python3.8/dist-packages/nsx_ujo/agent/nsxrpc_client.py", line 49, in accept 
  2023-06-29T11:48:04.645Z cba56c49-2eed-4cf7-af2f-f34cf619f00e NSX 5506 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="WARNING"] oslo.privsep.daemon privsep log: netns.setns(agent_ns, os.O_EXCL) 
  2023-06-29T11:48:04.645Z cba56c49-2eed-4cf7-af2f-f34cf619f00e NSX 5506 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="WARNING"] oslo.privsep.daemon privsep log: File "/usr/local/lib/python3.8/dist-packages/pyroute2/netns/__init__.py", line 338, in setns 
  2023-06-29T11:48:04.645Z cba56c49-2eed-4cf7-af2f-f34cf619f00e NSX 5506 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="WARNING"] oslo.privsep.daemon privsep log: raise OSError(ctypes.get_errno(), 'failed to open netns', netns) 
  2023-06-29T11:48:04.645Z cba56c49-2eed-4cf7-af2f-f34cf619f00e NSX 5506 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="WARNING"] oslo.privsep.daemon privsep log: OSError: [Errno 22] failed to open netns: '/var/run/netns/nsx-node-agent’ Workaround: restart nsx-node-agent

  Workaround: Restart nsx-node-agent.