Release Date: JUN 30, 2020

Build Details

Download Filename:	ESXi650-202006001.zip
Build:	16389870
Download Size:	343.4 MB
md5sum:	d9f464e5e4e7a2358ef99a27ec8024b0
sha1checksum:	53764a2e5740479d3ec3ad3a3cddd0853e95827e
Host Reboot Required:	Yes
Virtual Machine Migration or Shutdown Required:	Yes

Bulletins

Bulletin ID	Category	Severity
ESXi650-202006401-BG	Bugfix	Important
ESXi650-202006402-BG	Bugfix	Important

Rollup Bulletin

This rollup bulletin contains the latest VIBs with all the fixes since the initial release of ESXi 6.5.

Bulletin ID	Category	Severity
ESXi650-202006001	Bugfix	Important

Image Profiles

VMware patch and update releases contain general and critical image profiles. Application of the general release image profile applies to new bug fixes.

Image Profile Name

ESXi-6.5.0-20200604001-standard

ESXi-6.5.0-20200604001-no-tools

For more information about the individual bulletins, see the Download Patches page and the Resolved Issues section.

Patch Download and Installation

The typical way to apply patches to ESXi hosts is through the VMware vSphere Update Manager. For details, see the About Installing and Administering VMware vSphere Update Manager.

ESXi hosts can be updated by manually downloading the patch ZIP file from the VMware download page and installing the VIB by using the esxcli software vib command. Additionally, the system can be updated using the image profile and the esxcli software profile command.

For more information, see the vSphere Command-Line Interface Concepts and Examples and the vSphere Upgrade Guide.

Resolved Issues

The resolved issues are grouped as follows.

• ESXi650-202006401-BG
• ESXi650-202006402-BG
• ESXi-6.5.0-20200604001-standard
• ESXi-6.5.0-20200604001-no-tools

ESXi650-202006401-BG

Patch Category	Bugfix
Patch Severity	Important
Host Reboot Required	Yes
Virtual Machine Migration or Shutdown Required	Yes
Affected Hardware	N/A
Affected Software	N/A
VIBs Included	VMware_bootbank_vsanhealth_6.5.0-3.129.16389873 VMware_bootbank_esx-base_6.5.0-3.129.16389870 VMware_bootbank_vsan_6.5.0-3.129.16389871 VMware_bootbank_esx-tboot_6.5.0-3.129.16389870
PRs Fixed	N/A
Related CVE numbers	N/A

This patch updates the esx-base, vsan, esx-tboot and vsanhealth VIBs.

ESXi650-202006402-BG

Patch Category	Bugfix
Patch Severity	Important
Host Reboot Required	Yes
Virtual Machine Migration or Shutdown Required	Yes
Affected Hardware	N/A
Affected Software	N/A
VIBs Included	VMW_bootbank_nvme_1.2.2.28-2vmw.650.3.129.16389870
PRs Fixed	2520145
Related CVE numbers	N/A

This patch updates the nvme VIB to update the following issue:

• PR 2520145: ESXi servers lose access to vSAN disk groups and virtual machines become inaccessible

  Due to a rare buffer underrun in the NVMe driver, some vSAN disk groups might go offline. In the vmkernel.log, you can see messages for the underrun such as:

  2020-04-22T12:44:20.887Z cpu98:2110681)nvme:scsiIoCompleteCommand:459:vmkCmd 0x45a5e27f3580[2Ah I:0x4307ac4e3200 SN:0x30abf72a] UNDERRUN 40960/65536.
2020-04-22T12:44:20.887Z cpu76:2098413)WARNING: ScsiCore: 1816: Invalid sense buffer: error=0x0, valid=0x0
2020-04-22T12:44:20.887Z cpu76:2098413)LSOMCommon: SSDLOG_SetUnrecoverableError:1099: Throttled: Invoking registered permanent device error callback for caching tier device.

  The issue only happens when the buffer underrun in the NVMe driver comes as result of multiple all queues are full events in the vmkernel.log. For example:

  2020-02-28T16:32:39.183Z cpu23:66160)nvme:NvmeIo_SubmitIo:992:Failed to submit vmkCmd 0x439962f8e700, all queues are full, ns: 1, vmhba0
2020-02-28T16:32:39.183Z cpu23:66160)nvme:NvmeIo_SubmitIo:992:Failed to submit vmkCmd 0x4399722f8cc0, all queues are full, ns: 1, vmhba0

  When the issue occurs, load on the remaining nodes on the cluster increases and this might lead to other vSAN disk groups going offline. As a result, ESXi servers lose access to the vSAN disk groups and virtual machines become inaccessible.

  This issue is resolved in this release. In the vmkernel.log, you can still see all queues are full events, but they do not lead to a buffer underrun.

ESXi-6.5.0-20200604001-standard

Profile Name	ESXi-6.5.0-20200604001-standard
Build	For build information, see the top of the page.
Vendor	VMware, Inc.
Release Date	June 30, 2020
Acceptance Level	PartnerSupported
Affected Hardware	N/A
Affected Software	N/A
Affected VIBs	VMware_bootbank_vsanhealth_6.5.0-3.126.15965596 VMware_bootbank_esx-tboot_6.5.0-3.126.16207673 VMware_bootbank_esx-base_6.5.0-3.126.16207673 VMware_bootbank_vsan_6.5.0-3.126.15965595 VMW_bootbank_nvme_1.2.2.28-2vmw.650.3.129.16389870
PRs Fixed	2520145
Related CVE numbers	N/A

• This patch updates the following issue:

  Due to a rare buffer underrun in the NVMe driver, some vSAN disk groups might go offline. In the vmkernel.log, you can see messages for the underrun such as:

  2020-04-22T12:44:20.887Z cpu98:2110681)nvme:scsiIoCompleteCommand:459:vmkCmd 0x45a5e27f3580[2Ah I:0x4307ac4e3200 SN:0x30abf72a] UNDERRUN 40960/65536.
2020-04-22T12:44:20.887Z cpu76:2098413)WARNING: ScsiCore: 1816: Invalid sense buffer: error=0x0, valid=0x0
2020-04-22T12:44:20.887Z cpu76:2098413)LSOMCommon: SSDLOG_SetUnrecoverableError:1099: Throttled: Invoking registered permanent device error callback for caching tier device.

  The issue only happens when the buffer underrun in the NVMe driver comes as result of multiple all queues are full events in the vmkernel.log. For example:

  2020-02-28T16:32:39.183Z cpu23:66160)nvme:NvmeIo_SubmitIo:992:Failed to submit vmkCmd 0x439962f8e700, all queues are full, ns: 1, vmhba0
2020-02-28T16:32:39.183Z cpu23:66160)nvme:NvmeIo_SubmitIo:992:Failed to submit vmkCmd 0x4399722f8cc0, all queues are full, ns: 1, vmhba0

  When the issue occurs, load on the remaining nodes on the cluster increases and this might lead to other vSAN disk groups going offline. As a result, ESXi servers lose access to the vSAN disk groups and virtual machines become inaccessible.

ESXi-6.5.0-20200604001-no-tools

Profile Name	ESXi-6.5.0-20200604001-no-tools
Build	For build information, see the top of the page.
Vendor	VMware, Inc.
Release Date	June 30, 2020
Acceptance Level	PartnerSupported
Affected Hardware	N/A
Affected Software	N/A
Affected VIBs	VMware_bootbank_vsanhealth_6.5.0-3.129.16389873 VMware_bootbank_esx-base_6.5.0-3.129.16389870 VMware_bootbank_vsan_6.5.0-3.129.16389871 VMware_bootbank_esx-tboot_6.5.0-3.129.16389870 VMW_bootbank_nvme_1.2.2.28-2vmw.650.3.129.16389870
PRs Fixed	2520145
Related CVE numbers	N/A

• This patch updates the following issue:

  Due to a rare buffer underrun in the NVMe driver, some vSAN disk groups might go offline. In the vmkernel.log, you can see messages for the underrun such as:

  2020-04-22T12:44:20.887Z cpu98:2110681)nvme:scsiIoCompleteCommand:459:vmkCmd 0x45a5e27f3580[2Ah I:0x4307ac4e3200 SN:0x30abf72a] UNDERRUN 40960/65536.
2020-04-22T12:44:20.887Z cpu76:2098413)WARNING: ScsiCore: 1816: Invalid sense buffer: error=0x0, valid=0x0
2020-04-22T12:44:20.887Z cpu76:2098413)LSOMCommon: SSDLOG_SetUnrecoverableError:1099: Throttled: Invoking registered permanent device error callback for caching tier device.

  The issue only happens when the buffer underrun in the NVMe driver comes as result of multiple all queues are full events in the vmkernel.log. For example:

  2020-02-28T16:32:39.183Z cpu23:66160)nvme:NvmeIo_SubmitIo:992:Failed to submit vmkCmd 0x439962f8e700, all queues are full, ns: 1, vmhba0
2020-02-28T16:32:39.183Z cpu23:66160)nvme:NvmeIo_SubmitIo:992:Failed to submit vmkCmd 0x4399722f8cc0, all queues are full, ns: 1, vmhba0

  When the issue occurs, load on the remaining nodes on the cluster increases and this might lead to other vSAN disk groups going offline. As a result, ESXi servers lose access to the vSAN disk groups and virtual machines become inaccessible.