VMware Cloud Foundation 4.5.2 | 17 AUG 2023 | Build 22223457 Check for additions and updates to these release notes. |
VMware Cloud Foundation 4.5.2 | 17 AUG 2023 | Build 22223457 Check for additions and updates to these release notes. |
The VMware Cloud Foundation (VCF) 4.5.2 release includes the following:
Keyed to keyless license conversion: The option to convert the licensing mode of a workload domain from a keyed license (VCF-S or VCF perpetual license) to a keyless license (VMware Cloud Foundation+) model is a now available.
Support for mixed license deployment: A combination of keyed and keyless licenses can now be used within the same VCF instance. The licensing within a given workload domain needs to be homogeneous (no mixing of keyed and keyless licensing within a workload domain).
BOM deviation precheck: Running an upgrade precheck now determines if the Async Patch Tool was used in the environment to patch components.
BOM updates: Updated Bill of Materials with new product versions.
NSX-T Data Center 3.2.3.1, which includes new features and enhancements as part of NSX 3.2.3.1 and critical bug fixes. See https://docs.vmware.com/en/VMware-NSX/3.2.3.1/rn/vmware-nsxt-data-center-3231-release-notes/index.html for more details.
VMware vCenter Server 7.0 Update 3m, which contains critical bug fixes. See https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-vcenter-server-70u3m-release-notes/index.html.
VMware ESXi 7.0 Update 3n, which contains critical bug fixes. See https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-esxi-70u3n-release-notes.html for more details.
The VMware Imaging Appliance (VIA), included with the VMware Cloud Builder appliance to image ESXi servers, is deprecated and removed.
The VMware Cloud Foundation software product is comprised of the following software Bill-of-Materials (BOM). The components in the BOM are interoperable and compatible.
Software Component |
Version |
Date |
Build Number |
---|---|---|---|
Cloud Builder VM |
4.5.2 |
17 AUG 2023 |
22223457 |
SDDC Manager |
4.5.2 |
17 AUG 2023 |
22223457 |
VMware vCenter Server Appliance |
7.0 Update 3m |
22 JUN 2023 |
21784236 |
VMware ESXi |
7.0 Update 3n |
06 JUL 2023 |
21930508 |
7.0 Update 3l |
30 MAR 2023 |
21424296 |
|
VMware NSX-T |
3.2.3.1 |
27 JUL 2023 |
22104592 |
VMware vRealize Suite Lifecycle Manager |
8.10 |
19 JUN 2023 |
21950667 |
VMware vSAN is included in the VMware ESXi bundle.
You can use vRealize Suite Lifecycle Manager to deploy vRealize Automation, vRealize Operations Manager, vRealize Log Insight, and Workspace ONE Access. vRealize Suite Lifecycle Manager determines which versions of these products are compatible and only allows you to install/upgrade to supported versions.
vRealize Log Insight content packs are installed when you deploy vRealize Log Insight.
The vRealize Operations Manager management pack is installed when you deploy vRealize Operations Manager.
You can access the latest versions of the content packs for vRealize Log Insight from the VMware Solution Exchange and the vRealize Log Insight in-product marketplace store.
The SDDC Manager software is licensed under the VMware Cloud Foundation license. As part of this product, the SDDC Manager software deploys specific VMware software products.
The following VMware software components deployed by SDDC Manager are licensed under the VMware Cloud Foundation license:
VMware ESXi
VMware vSAN
VMware NSX-T Data Center
The following VMware software components deployed by SDDC Manager are licensed separately:
VMware vCenter Server
NOTE Only one vCenter Server license is required for all vCenter Servers deployed in a VMware Cloud Foundation system.
For details about the specific VMware software editions that are licensed under the licenses you have purchased, see the VMware Cloud Foundation Bill of Materials (BOM).
For general information about the product, see VMware Cloud Foundation.
For details on supported configurations, see the VMware Compatibility Guide (VCG) and the Hardware Requirements section on the Prerequisite Checklist tab in the Planning and Preparation Workbook.
To access the VMware Cloud Foundation and VMware Cloud Foundation+ documentation, go to the VMware Cloud Foundation product documentation.
To access the documentation for VMware software products that SDDC Manager can deploy, see the product documentation and use the drop-down menus on the page to choose the appropriate version:
VMware vSphere product documentation, also has documentation about ESXi and vCenter Server
The VMware Cloud Foundation web-based interface supports the latest two versions of the following web browsers:
Google Chrome 89 or later
Mozilla Firefox 80 or later
Microsoft Edge 90 or later
For the Web-based user interfaces, the supported standard resolution is 1024 by 768 pixels. For best results, use a screen resolution within these tested resolutions:
1024 by 768 pixels (standard)
1366 by 768 pixels
1280 by 1024 pixels
1680 by 1050 pixels
Resolutions below 1024 by 768, such as 640 by 960 or 480 by 800, are not supported.
If your VCF 4.5.0/4.5.1 environment is subscribed to VMware Cloud Foundation+, then you cannot transition to mixed licensing mode in VCF 4.5.2 post-upgrade. Only customers whose VCF source environments are in perpetual licensing mode (in any previous VCF version) can transition to mixed licensing mode in VCF 4.5.2.
VMware Cloud Foundation 4.5.2 cannot upgrade to VMware Cloud Foundation 5.0 or 5.0.0.1. See KB article 94195 for more information.
You can install VMware Cloud Foundation 4.5.2 as a new release or perform a sequential or skip-level upgrade to VMware Cloud Foundation 4.5.2.
Installing as a New Release
The new installation process has three phases:
Phase One: Prepare the Environment
The Planning and Preparation Workbook provides detailed information about the software, tools, and external services that are required to implement a Software-Defined Data Center (SDDC) with VMware Cloud Foundation, using a standard architecture model.
Phase Two: Image all servers with ESXi
Image all servers with the ESXi version mentioned in the Cloud Foundation Bill of Materials (BOM) section. See the VMware Cloud Foundation Deployment Guide for information on installing ESXi.
Phase Three: Install Cloud Foundation 4.5.2
See the VMware Cloud Foundation Deployment Guide for information on deploying Cloud Foundation.
Upgrading to Cloud Foundation 4.5.2
You can perform a sequential or skip-level upgrade to VMware Cloud Foundation 4.5.2 from VMware Cloud Foundation 4.2.1 or later. If your environment is at a version earlier than 4.2.1, you must upgrade the management domain and all VI workload domains to VMware Cloud Foundation 4.2.1 and then upgrade to VMware Cloud Foundation 4.5. For more information see VMware Cloud Foundation Lifecycle Management.
If your VMware Cloud Foundation instance includes vRealize Suite Lifecycle Manager, you may need to install a Product Support Pack to support VMware Cloud Foundation 4.5.2. Check the release notes to see what Product Support Pack is required for your current version of vRealize Suite Lifecycle Manager:
VMware vRealize Suite Lifecycle Manager Product Support Pack Release Notes
VMware Aria Suite Lifecycle Product Support Pack Release Notes
See KB article 88829 for more information about vRealize Suite/Aria upgrade paths.
Before you upgrade a vCenter Server, take a file-based backup. See Manually Back Up vCenter Server.
VMware Cloud Foundation 4.4 and later deactivate the SSH service on ESXi hosts by default, following the vSphere security configuration guide recommendation. Scripts that rely on SSH being active on ESXi hosts will not work after upgrading to VMware Cloud Foundation 4.4 or later. Update your scripts to account for this new behavior. See KB 86230 for information about enabling and disabling the SSH service on ESXi hosts.
The following issues are resolved in this release.
Calling the /v1/nsxt-clusters
API fails with a GET_VLCM_CLUSTER_STATUS_FAILED
error when vCenter Server for the VI workload domain is down.
NFS datastore is not mounted on newly added ESXi hosts
In the Developer Center > API Explorer section of the SDDC Manager UI, reference to the Resource Locks API is not available.
Although the update of a component has completed with a failure, its update status in the update history still appears as in-progress.
vRealize Operations Manager admin account appears as disconnected.
Trying to log in to SDDC Manager fails with a no permission to display this page
error.
VCF 5.0 is not a supported upgrade path if you are running VCF 4.5.2 version.
During host reboot, NSX Edge goes down and ESX host shows as "Partial Success." under Fabric -> Host -> Node.
If the ESX host is taken out of Maintenance Mode in the partial success state and the VMs are vMotioned to this host, then those VMs will lose L3 connectivity.
Workaround: Reboot the host.
If this issue reoccurs after rebooting, see KB article 93882.
Cosmetic Issue: There are naming inconsistencies in the UI when referencing the new keyless and key-based licenses.
If you are using mixed licensing, the naming has changed from "perpetual" to "key-based" and "subscription" to "keyless." There are a few instances in the UI that still displays the previous terminology.
Workaround: None.
This issue is only cosmetic and does not impact functionality.
Password operation fails when NSX Edge node hostname is in uppercase
If the NSX Edge node hostname is in uppercase, the password rotation fails.
Workaround: Login to NSX Edge node as root user
and update the VM hostname in lowercase.
Workload Management does not support NSX-T Data Center Federation
You cannot deploy Workload Management (vSphere with Tanzu) to a workload domain when that workload domain's NSX-T Data Center instance is participating in an NSX-T Data Center Federation.
None.
NSX-T Guest Introspection (GI) and NSX-T Service Insertion (SI) are not supported on stretched clusters
There is no support for stretching clusters where NSX-T Guest Introspection (GI) or NSX-T Service Insertion (SI) are enabled. VMware Cloud Foundation detaches Transport Node Profiles from AZ2 hosts to allow AZ-specific network configurations. NSX-T GI and NSX-T SI require that the same Transport Node Profile be attached to all hosts in the cluster.
None.
Stretched clusters and Workload Management
You cannot stretch a cluster on which Workload Management is deployed.
None.
Configuration drift upgrade proceeds even when a workload domain is in unhealthy state and the upgrade fails
VMware Cloud Foundation does not check the health of workload domains before attempting to apply a configuration drift upgrade. If a workload domain is in an unhealthy state, the upgrade fails with the message: The domain <DOMAIN_NAME> is in unhealthy state and cannot proceed with config drift
.
Workaround: Resolve all the issues with the unhealthy workload domain and retry the upgrade.
Viewing the bundle details for the SDDC Manager upgrade bundle shows the incorrect "Required Version"
The "Required Version" displayed on the SDDC Manager bundle details page shows 4.5, even though skip-level upgrade from versions ealier than 4.5 is supported.
Workaround: None. This is a UI-only issue. You can still preform a skip-level upgrade from an earlier version of VMware Cloud Foundation.
Update history for a workload domain may show "IN-PROGRESS" for a failed NSX Manager upgrade
When you view the update status for a failed NSX Manager upgrade in SDDC Manager's Update History, it may show that the task is still in progress.
Workaround: None. This is a cosmetic issue. The upgrade can be re-triggered and does not impact future upgrades.
NSX-T Data Center upgrade fails
Upgrading NSX-T data Center may fail or time out under certain conditions. Reviewing the upgrade in the NSX Manager UI shows the upgrade status as Paused on the Hosts step and also reports post check issues.
Workaround: See KB 92313.
SDDC Manager upgrade fails at "Setup Common Appliance Platform"
If a virtual machine reconfiguration task (for example, removing a snapshot or running a backup) is taking place in the management domain at the same time you are upgrading SDDC Manager, the upgrade may fail.
Workaround: Schedule SDDC Manager upgrades for a time when no virtual machine reconfiguration tasks are happening in the management domain. If you encounter this issue, wait for the other tasks to complete and then retry the upgrade.
Parallel upgrades of vCenter Server are not supported
If you attempt to upgrade vCenter Server for multiple VI workload domains at the same time, the upgrade may fail while changing the permissions for the vpostgres configuration directory in the appliance. The message chown -R vpostgres:vpgmongrp /storage/archive/vpostgres
appears in the PatchRunner.log file on the vCenter Server Appliance.
Workaround: Each vCenter Server instance must be upgraded separately.
When you upgrade VMware Cloud Foundation, one of the vSphere Cluster Services (vCLS) agent VMs gets placed on local storage
vSphere Cluster Services (vCLS) ensures that cluster services remain available, even when the vCenter Server is unavailable. vCLS deploys three vCLS agent virtual machines to maintain cluster services health. When you upgrade VMware Cloud Foundation, one of the vCLS VMs may get placed on local storage instead of shared storage. This could cause issues if you delete the ESXi host on which the VM is stored.
Workaround: Deactivate and reactivate vCLS on the cluster to deploy all the vCLS agent VMs to shared storage.
Check the placement of the vCLS agent VMs for each cluster in your environment.
In the vSphere Client, select Menu > VMs and Templates.
Expand the vCLS folder.
Select the first vCLS agent VM and click the Summary tab.
In the Related Objects section, check the datastore listed for Storage. It should be the vSAN datastore. If a vCLS agent VM is on local storage, you need to deactivate vCLS for the cluster and then re-enable it.
Repeat these steps for all vCLS agent VMs.
Deactivate vCLS for clusters that have vCLS agent VMs on local storage.
In the vSphere Client, click Menu > Hosts and Clusters.
Select a cluster that has a vCLS agent VM on local storage.
In the web browser address bar, note the moref id for the cluster.
For example, if the URL displays as https://vcenter-1.vrack.vsphere.local/ui/app/cluster;nav=h/urn:vmomi:ClusterComputeResource:domain-c8:503a0d38-442a-446f-b283-d3611bf035fb/summary, then the moref id is domain-c8.
Select the vCenter Server containing the cluster.
Click Configure > Advanced Settings.
Click Edit Settings.
Change the value for config.vcls.clusters.<moref id>.enabled
to false
and click Save.
If the config.vcls.clusters.<moref id>.enabled
setting does not appear for your moref id, then enter its Name and false
for the Value and click Add.
Wait a couple of minutes for the vCLS agent VMs to be powered off and deleted. You can monitor progress in the Recent Tasks pane.
Enable vCLS for the cluster to place the vCLS agent VMs on shared storage.
Select the vCenter Server containing the cluster and click Configure > Advanced Settings.
Click Edit Settings.
Change the value for config.vcls.clusters.<moref id>.enabled
to true
and click Save.
Wait a couple of minutes for the vCLS agent VMs to be deployed and powered on. You can monitor progress in the Recent Tasks pane.
Check the placement of the vCLS agent VMs to make sure they are all on shared storage
Using the /v1/upgrades API to trigger parallel cluster upgrades across workload domains in a single API call does not upgrade the clusters in parallel
When using the VMware Cloud Foundation API to upgrade multiple workload domains in parallel, including multiple resource upgrade specifications (resourceUpgradeSpec
) in a single domain upgrade API (/v1/upgrades
) call does not work as expected.
Workaround: To get the best performance when upgrading multiple workload domains in parallel using the VMware Cloud Foundation API, do not include multiple resource upgrade specifications (resourceUpgradeSpec
) in a single domain upgrade call. Instead, invoke the domain upgrade multiple times with a single resourceUpgradeSpec
for each workload domain.
You can also use the SDDC Manager UI to trigger multiple parallel upgrades across workload domains.
SDDC Manager UI shows older VMware Cloud Foundation version after upgrading to 4.5.x
If you had vRealize Suite Lifecycle Manager deployed prior to upgrading to VMware Cloud Foundation 4.5.x, the SDDC Manager UI will not display the version as 4.5.x until you upgrade vRealize Suite Lifecycle Manager. For example:
The VMware Cloud Foundation 4.5.x BOM requires vRealize Suite Lifecycle Manager 8.8.2 or higher.
Workaround: Upgrade vRealize Suite Lifecycle Manager to version 8.8.2 or higher. See Upgrade vRealize Suite Lifecycle Manager for VMware Cloud Foundation.
NSX-T Data Center upgrade fails at "NSX T PERFORM BACKUP"
If you did not change the destination of NSX Manager backups to an external SFTP server, upgrades may fail due to an out-of-date SSH fingerprint for SDDC Manager.
Workaround:
Log in to the NSX Manager UI.
Click System > Backup & Restore.
Click Edit for the SFTP Server.
Remove the existing SSH fingerprint and click Save.
Click Add to add the server provided fingerprint.
Click Save.
Retry the NSX-T Data Center upgrade from the SDDC Manager UI.
Cluster-level ESXi upgrade fails
Cluster-level selection during upgrade does not consider the health status of the clusters and may show a cluster's status as Available, even for a faulty cluster. If you select a faulty cluster, the upgrade fails.
Always perform an update precheck to validate the health status of the clusters. Resolve any issues before upgrading.
You are unable to update NSX-T Data Center in the management domain or in a workload domain with vSAN principal storage because of an error during the NSX-T transport node precheck stage
In SDDC Manager, when you run the upgrade precheck before updating NSX-T Data Center, the NSX-T transport node validation results with the following error.
No coredump target has been configured. Host core dumps cannot be saved.:System logs on host sfo01-m01-esx04.sfo.rainpole.io are stored on non-persistent storage. Consult product documentation to configure a syslog server or a scratch partition.
Because the upgrade precheck results with an error, you cannot proceed with updating the NSX-T Data Center instance in the domain. VMware Validated Design supports vSAN as the principal storage in the management domain. However, vSAN datastores do no support scratch partitions. See VMware Knowledge Base article 2074026.
Disable the update precheck validation for the subsequent NSX-T Data Center update.
Log in to SDDC Manager as vcf using a Secure Shell (SSH) client.
Open the application-prod.properties
file for editing: vi /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties
Add the following property and save the file: lcm.nsxt.suppress.prechecks=true
Restart the life cycle management service: systemctl restart lcm
Log in to the SDDC Manager user interface and proceed with the update of NSX-T Data Center.
ESXi upgrade fails with the error "Incompatible patch or upgrade files. Please verify that the patch file is compatible with the host. Refer LCM and VUM log file."
This error occurs if any of the ESXi hosts that you are upgrading have detached storage devices.
Workaround: Attach all storage devices to the ESXi hosts being upgraded, reboot the hosts, and retry the upgrade.
Update precheck fails with the error "Password has expired"
If the vCenter Single Sign-On password policy specifies a maximum lifetime of zero (never expires), the precheck fails.
Workaround: Set the maximum lifetime password policy to something other than zero and retry the precheck.
Bring-up Network Configuration Validation fails with "Gateway IP Address for Management is not contactable"
The following failure "Gateway IP Address for MANAGEMENT is not contactable" is reported as fatal error in Cloud Buider UI and bring-up cannot continue. In some cases the validation fails to validate connectivity because it uses as set of predefined ports, however, ping is working.
See KB 89990 for more information.
The Cloud Foundation Builder VM remains locked after more than 15 minutes.
The VMware Imaging Appliance (VIA) locks out the admin user after three unsuccessful login attempts. Normally, the lockout is reset after fifteen minutes but the underlying Cloud Foundation Builder VM does not automatically reset.
Log in to the VM console of the Cloud Foundation Builder VM as the root
user. Unlock the account by resetting the password of the admin user with the following command:
pam_tally2 --user=<user> --reset
SDDC Manager UI redirects to the Subscription page
After you commit your VMware Cloud Foundation instance to subscription mode (keyless licensing), the SDDC Manager UI may redirect you to the Subscription page when you are trying to navigate to a different section of the UI (for example, Inventory > Workload Domains).
Workaround: Refresh your web browser page to reload the SDDC Manager UI.
Disabling CEIP on SDDC Manager does not disable CEIP on vRealize Automation and vRealize Suite Lifecycle Manager
When you disable CEIP on the SDDC Manager Dashboard, data collection is not disabled on vRealize Automation and vRealize Suite Lifecycle Manager. This is because of API deprecation in vRealize Suite 8.x.
Workaround: Manually disable CEIP in vRealize Automation and vRealize Suite Lifecycle Manager. For more information, see VMware vRealize Automation Documentation and VMware vRealize Suite Lifecycle Manager Documentation.
SDDC Manager UI shows a file size of 0 (zero) for vSphere Lifecycle Manager images
After you import a vSphere Lifecycle Manager image to SDDC Manager, the UI displays a file size of 0 for the image.
Workaround: None. This is a cosmetic issue only.
Updating DNS/NTP server does not apply the update to all NSX Managers
If you update the NTP or DNS server information for a VMware Cloud Foundation instance that includes more than one NSX Manager, only one of the NSX Managers gets updated with the new information.
NOTE: This issue only occurs when VMware Cloud Foundation was upgraded from an earlier version. It does not impact new VMware Cloud Foundation 4.5.1 deployments.
Workaround: Use the NSX Manager API or CLI to manually update the DNS/NTP server information for the remaining NSX Manager(s).
vRealize Operations Manager admin account appears as disconnected
SDDC Manager incorrectly shows the vRealize Operations Manager admin account as disconnected due to an expired password. The admin account password used for logging into the vRealize Operations Manager UI never expires, but SDDC Manager is actually checking the virtual appliance (Photon OS) admin account password.
Workaround: To clear the expired password/disconnected alert in SDDC Manager:
Log in to the affected vRealize Operations Manager node and update the virtual appliance admin password.
In SDDC Manager, remediate (or rotate or update) the password for the expired account. Or, use the VMware Cloud Foundation API to run POST /v1/credentials/expirations
.
vRealize Suite Lifecycle Manager deployment fails due to domain manager application.properties file permissions
In certain circumstances, the file permissions of /etc/vmware/vcf/domainmanager/application-prod.properties can get changed to read-only. If this happens, the domainmanager service logs (/var/log/vmware/vcf/domainmanager/) report an error-- /etc/vmware/vcf/domainmanager/application-prod.properties: Permission denied
-- and vRealize Suite Lifecycle Manager deployment fails.
Workaround: Contact VMware Support.
Deployment of vRealize Suite products fails after a cluster is renamed in SDDC Manager
If you rename a cluster in the SDDC Manager UI, deploying a vRealize Suite product may fail with the error: "No cluster found with provided details. Ensure the provided cluster is present in vCenter or retry giving the right cluster details. Invalid cluster passed for the request."
Workaround:Use the vRealize Suite Lifecycle Manager UI to refresh the vCenter data collection from the vRealize Suite Lifecycle Manager settings page and then retry the deployment.
SDDC Manager UI issues when using Google Chrome
Some versions of Google Chrome may have issues properly rendering the SDDC Manager UI screens.
Workaround: Use a different web browser.
Name resolution fails when configuring the NTP server
Under certain conditions, name resolution may fail when you configure an NTP server.
Workaround: Run the following command using the FQDN of the failed resource(s) to ensure name resolution is successful and then retry the NTP server configuration.
nslookup <FQDN>
Generate CSR task for a component hangs
When you generate a CSR, the task may fail to complete due to issues with the component's resources. For example, when you generate a CSR for NSX Manager, the task may fail to complete due to issues with an NSX Manager node. You cannot retry the task once the resource is up and running again.
Log in to the UI for the component to troubleshoot and resolve any issues.
Using SSH, log in to the SDDC Manager VM with the user name vcf
.
Type su to switch to the root account.
Run the following command: systemctl restart operationsmanager
Retry generating the CSR.
SoS utility options for health check are missing information
Due to limitations of the ESXi service account, some information is unavailable in the following health check options:
--hardware-compatibility-report:
No Devices and Driver
information for ESXi hosts.
--storage-health:
No vSAN Health Status
or Total no. of disks
information for ESXi hosts.
None.
Adding host fails when host is on a different VLAN
A host add operation can sometimes fail if the host is on a different VLAN.
Before adding the host, add a new portgroup to the VDS for that cluster.
Tag the new portgroup with the VLAN ID of the host to be added.
Add the Host. This workflow fails at the "Migrate host vmknics to dvs" operation.
Locate the failed host in vCenter, and migrate the vmk0 of the host to the new portgroup you created in step 1. For more information, see Migrate VMkernel Adapters to a vSphere Distributed Switch in the vSphere product documentation.
Retry the Add Host operation.
NOTE: If you later remove this host in the future, you must manually remove the portgroup as well if it is not being used by any other host.
NFS datastore is not mounted on newly added ESXi hosts
If you have a workload domain that uses NFS as principal storage and you add hosts to a cluster in that workload domain, the NFS datastore does not get mounted on the new hosts.
Workaround:
In the vSphere Client, navigate to the datastore.
Right-click the datastore to mount and select Mount Datastore on Additional Hosts.
Select the hosts that should access the datastore and click OK.
Deploying partner services on an NSX-T workload domain displays an error
Deploying partner services, such as McAfee or Trend, on a workload domain enabled for vSphere Update Manager (VUM), displays the “Configure NSX at cluster level to deploy Service VM” error.
Attach the Transport node profile to the cluster and try deploying the partner service. After the service is deployed, detach the transport node profile from the cluster.
If the witness ESXi version does not match with the host ESXi version in the cluster, vSAN cluster partition may occur
vSAN stretch cluster workflow does not check the ESXi version of the witness host. If the witness ESXi version does not match the host version in the cluster, then vSAN cluster partition may happen.
Upgrade the witness host manually with the matching ESXi version using the vCenter VUM functionality.
Replace or deploy the witness appliance matching with the ESXi version.
Adding a host to a vLCM-enabled workload domain configured with the Dell Hardware Support Manager (OMIVV) fails
When you try to add a host to a vSphere cluster for a workload domain enabled with vSphere Lifecycle Manager (vLCM), the task fails and the domain manager log reports "The host (host-name) is currently not managed by OMIVV." The domain manager logs are located at /var/log/vmware/vcf/domainmanager on the SDDC Manager VM.
Update the hosts inventory in OMIVV and retry the add host task in the SDDC Manager UI. See the Dell documentation for information about updating the hosts inventory in OMIVV.
Adding a vSphere cluster or adding a host to a workload domain fails
Under certain circumstances, adding a host or vSphere cluster to a workload domain fails at the Configure NSX-T Transport Node or Create Transport Node Collection subtask.
Enable SSH for the NSX Manager VMs.
SSH into the NSX Manager VMs as admin and then log in as root.
Run the following command on each NSX Manager VM: sysctl -w net.ipv4.tcp_en=0
Login to NSX Manager UI for the workload domain.
Navigate to System > Fabric > Nodes > Host Transport Nodes.
Select the vCenter server for the workload domain from the Managed by drop-down menu.
Expand the vSphere cluster and navigate to the transport nodes that are in a partial success state.
Select the check box next to a partial success node, click Configure NSX.
Click Next and then click Apply.
Repeat steps 7-9 for each partial success node.
When all host issues are resolved, transport node creation starts for the failed nodes. When all hosts are successfully created as transport nodes, retry the failed add vSphere cluster or add host task from the SDDC Manager UI.
The vSAN Performance Service is not enabled for vSAN clusters when CEIP is not enabled
If you do not enable the VMware Customer Experience Improvement Program (CEIP) in SDDC Manager, when you create a workload domain or add a vSphere cluster to a workload domain, the vSAN Performance Service is not enabled for vSAN clusters. When CEIP is enabled, data from the vSAN Performance Service is provided to VMware and this data is used to aid VMware Support with troubleshooting and for products such as VMware Skyline, a proactive cloud monitoring service. See Customer Experience Improvement Program for more information on the data collected by CEIP.
Enable CEIP in SDDC Manager. See the VMware Cloud Foundation Documentation. After CEIP is enabled, a scheduled task that enables the vSAN Performance Service on existing clusters in workload domains runs every three hours. The service is also enabled for new workload domains and clusters. To enable the vSAN Performance Service immediately, see the VMware vSphere Documentation.
Creation or expansion of a vSAN cluster with more than 32 hosts fails
By default, a vSAN cluster can grow up to 32 hosts. With large cluster support enabled, a vSAN cluster can grow up to a maximum of 64 hosts. However, even with large cluster support enabled, a creation or expansion task can fail on the sub-task Enable vSAN on vSphere Cluster.
Enable Large Cluster Support for the vSAN cluster in the vSphere Client. If it is already enabled skip to step 2.
Select the vSAN cluster in the vSphere Client.
Select Configure > vSAN > Advanced Options.
Enable Large Cluster Support.
Click Apply.
Click Yes.
Run a vSAN health check to see which hosts require rebooting.
Put the hosts into Maintenance Mode and reboot the hosts.
For more information about large cluster support, see https://kb.vmware.com/kb/2110081.
Removing a host from a cluster, deleting a cluster from a workload domain, or deleting a workload domain fails if Service VMs (SVMs) are present
If you deployed an endpoint protection service (such as guest introspection) to a cluster through NSX-T Data Center, then removing a host from the cluster, deleting the cluster, or deleting the workload domain containing the cluster will fail on the subtask Enter Maintenance Mode on ESXi Hosts.
For host removal: Delete the Service VM from the host and retry the operation.
For cluster deletion: Delete the service deployment for the cluster and retry the operation.
For workload domain deletion: Delete the service deployment for all clusters in the workload domain and retry the operation.
vCenter Server overwrites the NFS datastore name when adding a cluster to a VI workload domain
If you add an NFS datastore with the same NFS server IP address, but a different NFS datastore name, as an NFS datastore that already exists in the workload domain, then vCenter Server applies the existing datastore name to the new datastore.
If you want to add an NFS datastore with a different datastore name, then it must use a different NFS server IP address.
Terminology used to show the licensing mode of a domain is different between the UI and the API.
UI uses the terminology Keyless and Key-based, whereas the API uses Subscription and Perpetual.
Workaround: None
Stretch cluster operation fails
If the cluster that you are stretching does not include a powered-on VM with an operating system installed, the operation fails at the "Validate Cluster for Zero VMs" task.
Make sure the cluster has a powered-on VM with an operating system installed before stretching the cluster.