VMware Cloud Foundation 5.0 | 01 JUN 2023 | Build 21822418

Check for additions and updates to these release notes.

Updated Upgrade Notice

Note: VCF+/VCF-S customers are now able to upgrade to VCF 5.0.

What's New

The VMware Cloud Foundation (VCF) 5.0 release includes the following:

  • In-place upgrade: VCF 4.3.x, VCF 4.4.x, VCF 4.5, and VCF 4.5.1 can upgrade to VCF 5.0 from SDDC Manager UI (with internet access) or using the Bundle Transfer Utility (without internet access).

  • Isolated Workload Domains: Create and manage workload domains that can each join the management domain's vCenter Single Sign-On domain or a new vCenter Single Sign-On domain that is not used by any other workload domain.

  • BOM component interoperability data checks: Auto-checks of BOM component interoperability within SDDC Manager.

  • BOM product re-licensing: Ability to perform relicensing of VCF BOM components from SDDC Manager UI.

  • Licensing update: SDDC Manager is now licensed as part of VCF. Therefore, a separate SDDC Manager license key is no longer required.

  • Improvements to upgrade prechecks: Upgrade prechecks now provides granular control over version-specific and individual component prechecks.

  • Improvements to Configuration Update workflow: Configuration Updates can be applied between upgrade applications or can be saved to be applied at a later time.

  • New upgrade option for ESXi Upgrade: If there are any powered-off virtual machines in the chosen cluster to be upgraded, they will only be evacuated during the upgrade process if this option is enabled. Otherwise, they will not be migrated specifically for the purpose of the upgrade.

  • Scaled support: Ability to upgrade 10 clusters (hosts) across 5 workload domains (5*10) simultaneously in ESXi, allowing for parallel upgrades to occur.

  • In-product feedback: SDDC Manager can prompt customers at random intervals to provide feedback related to VMware Cloud Foundation.

  • VMware NSX-T Data Center rebranding: Starting with version 4.0, VMware NSX-T Data Center is known as VMware NSX.

  • VMware Validated Solutions: All VMware Validated Solutions are updated to support VMware Cloud Foundation 5.0. Visit VMware Validated Solutions for the updated guides.

  • VMware Cloud Foundation with Skyline Health Diagnostics: New content available for Proactive Diagnostics of VMware Cloud Foundation with Skyline Health Diagnostics.

  • BOM updates: Updated Bill of Materials with new product versions.

Deprecation Notices

  • The VMware Imaging Appliance (VIA), included with the VMware Cloud Builder appliance to image ESXi servers, is deprecated in VMware Cloud Foundation 5.0 and will be removed in a future release.

  • In a future release, the "Connect Workload Domains" option from the vRealize Operations card located in SDDC Manager > Administration > vRealize Suite section will be removed. Additionally, related VCF Public API options will also be deprecated.

    Starting with vRealize Operations 8.10, an improved functionality for connecting VCF Workload Domains to vRealize Operations is available directly from the vRealize Operations UI itself. Users are encouraged to use this method within the vRealize Operations UI for connecting VCF Workload Domains.

  • VMware Update Manager as the default option for management domain cluster will be deprecated in a subsequent release and vSphere Lifecycle Manager will become the default. This is to align with the vSphere deprecation of VMware Update Manager in future release as noted here: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-vcenter-server-703-release-notes.html

VMware Cloud Foundation Bill of Materials (BOM)

The VMware Cloud Foundation software product is comprised of the following software Bill-of-Materials (BOM). The components in the BOM are interoperable and compatible.

Software Component



Build Number

Cloud Builder VM


01 JUN 2023


SDDC Manager


01 JUN 2023


VMware vCenter Server Appliance

8.0 Update 1a

01 JUN 2023


VMware ESXi

8.0 Update 1a

01 JUN 2023


VMware vSAN Witness Appliance

8.0 Update 1

14 APR 2023


VMware NSX

01 JUN 2023


VMware vRealize Suite Lifecycle Manager

8.10 Patch 1

21 FEB 2023


  • VMware vSAN is included in the VMware ESXi bundle.

  • You can use vRealize Suite Lifecycle Manager to deploy vRealize Automation, vRealize Operations Manager, vRealize Log Insight, and Workspace ONE Access. vRealize Suite Lifecycle Manager determines which versions of these products are compatible and only allows you to install/upgrade to supported versions.

  • vRealize Log Insight content packs are installed when you deploy vRealize Log Insight.

  • The vRealize Operations Manager management pack is installed when you deploy vRealize Operations Manager.

  • You can access the latest versions of the content packs for vRealize Log Insight from the VMware Solution Exchange and the vRealize Log Insight in-product marketplace store.

VMware Software Edition License Information

The SDDC Manager software is licensed under the VMware Cloud Foundation license. As part of this product, the SDDC Manager software deploys specific VMware software products.

The following VMware software components deployed by SDDC Manager are licensed under the VMware Cloud Foundation license:

  • VMware ESXi

  • VMware vSAN

  • VMware NSX

The following VMware software components deployed by SDDC Manager are licensed separately:

  • VMware vCenter Server

NOTE Only one vCenter Server license is required for all vCenter Servers deployed in a VMware Cloud Foundation system.

For details about the specific VMware software editions that are licensed under the licenses you have purchased, see the VMware Cloud Foundation Bill of Materials (BOM).

For general information about the product, see VMware Cloud Foundation.

Supported Hardware

For details on supported configurations, see the VMware Compatibility Guide (VCG) and the Hardware Requirements section on the Prerequisite Checklist tab in the Planning and Preparation Workbook.


To access the Cloud Foundation documentation, go to the VMware Cloud Foundation product documentation.

To access the documentation for VMware software products that SDDC Manager can deploy, see the product documentation and use the drop-down menus on the page to choose the appropriate version:

Browser Compatibility and Screen Resolutions

The VMware Cloud Foundation web-based interface supports the latest two versions of the following web browsers:

  • Google Chrome

  • Mozilla Firefox

  • Microsoft Edge

For the Web-based user interfaces, the supported standard resolution is 1920 by 1080 pixels.

Installation and Upgrade Information

You can install VMware Cloud Foundation 5.0 as a new release or perform a sequential or skip-level upgrade to VMware Cloud Foundation 5.0.


If you are upgrading to VMware Cloud Foundation 5.0 or above, read the critical information in KB article 91751.

Installing as a New Release

The new installation process has three phases:

  • Phase One: Prepare the Environment: The Planning and Preparation Workbook provides detailed information about the software, tools, and external services that are required to implement a Software-Defined Data Center (SDDC) with VMware Cloud Foundation, using a standard architecture model.

  • Phase Two: Image all servers with ESXi: Image all servers with the ESXi version mentioned in the Cloud Foundation Bill of Materials (BOM) section. See the VMware Cloud Foundation Deployment Guide for information on installing ESXi.

  • Phase Three: Install Cloud Foundation 5.0: See the VMware Cloud Foundation Deployment Guide for information on deploying Cloud Foundation.

Upgrading to Cloud Foundation 5.0

You can perform a sequential or skip-level upgrade to VMware Cloud Foundation 5.0 from VMware Cloud Foundation 4.3.x or later. If your environment is at a version earlier than 4.3.x, you must upgrade the management domain and all VI workload domains to VMware Cloud Foundation 4.3.x or above and then upgrade to VMware Cloud Foundation 5.0. For more information see VMware Cloud Foundation Lifecycle Management.


Before you upgrade a vCenter Server, take a file-based backup. See Manually Back Up vCenter Server.


Since VMware Cloud Foundation 5.0 disables the SSH service by default, scripts that rely on SSH being enabled on ESXi hosts will not work after upgrading to VMware Cloud Foundation 5.0. Update your scripts to account for this new behavior. See KB 86230 for information about enabling and disabling the SSH service on ESXi hosts.

Known Issues

VMware Cloud Foundation Known Issues

  • Lifecycle Management Precheck does not throw an error when NSX Manager inventory is out of sync

    The Lifecycle Management Precheck displays a green status and does not generate any errors for NSX Manager inventory.

    Workaround: None

  • Missing license information error in UI

    A Missing domain licensing information error is shown after:

    1. A domain creation operation is triggered from a workload domain screen through public API (outside of the SDDC UI)

    2. then an Add Cluster workflow is triggered through the UI

    Workaround: Refresh the page and then trigger the "Add Cluster" workflow.

  • The upgrade screen displays the incorrect version when upgrading from VCF 4.3 to version 5.0.

    The VCF UI shows the incorrect source version when upgrading from version 4.3 to 5.0. However, the actual upgrade is still correctly performing a direct skip upgrade from the source version to target version.

    Workaround: None

  • Creating workload domains in parallel does not enforce maximum limit of compute managers per NSX Manager.

    Creating workload domains in parallel can lead to one or more workload domain creations to fail since the compute managers limit per NSX Manager will be exceeded.

    Workaround: Delete the failed workload domain

  • In VCF 5.0 Mixed BOM with vCenter Enhanced Linked Mode, you can see vCenter Server systems of version 8.0 from a 7.x vCenter Server instance

    If you have a vCenter Enhanced Linked Mode group that contains vCenter Server instances of versions 8.x and 7.x, when you log in to a 7.x vCenter Server instance, in the vSphere Client, you can see vCenter Server systems of version 8.0. Since vCenter Server 8.0 introduces new functionalities, you can run workflows specific to vSphere 8.0 on the 8.0 vCenter Server, but this might lead to unexpected results when run from the 7.x vSphere Client.

    Workaround: This issue is fixed in vSphere 7.0 P07.

  • Upgrade Pre-Check Scope dropdown may contain additional entries

    When performing Upgrade Prechecks through SDDC Manager UI and selecting a target VCF version, the Pre-Check Scope dropdown may contain more selectable entries than necessary. SDDC Manager may appear as an entry more than once. It also may be included as a selectable component for VI domains, although it's a component of the management domain.

    Workaround: None. The issue is visual with no functional impact.

  • Converting clusters from vSphere Lifecycle Manager baselines to vSphere Lifecycle Manager images is not supported.

    vSphere Lifecycle Manager baselines (previously known as vSphere Update Manager or VUM) are deprecated in vSphere 8.0, but continue to be supported. See KB article 89519 for more information.

    VMware Cloud Foundation 5.0 does not support converting clusters from vSphere Lifecycle Manager baselines to vSphere Lifecycle Manager images. This capability will be supported in a future release.

    Workaround: None

  • Workload Management does not support NSX Data Center Federation

    You cannot deploy Workload Management (vSphere with Tanzu) to a workload domain when that workload domain's NSX Data Center instance is participating in an NSX Data Center Federation.

    Workaround: None.

  • NSX Guest Introspection (GI) and NSX Network Introspection (NI) are not supported on stretched clusters

    There is no support for stretching clusters where NSX Guest Introspection (GI) or NSX Network Introspection (NI) are enabled. VMware Cloud Foundation detaches Transport Node Profiles from AZ2 hosts to allow AZ-specific network configurations. NSX GI and NSX NI require that the same Transport Node Profile be attached to all hosts in the cluster.


  • Stretched clusters and Workload Management

    You cannot stretch a cluster on which Workload Management is deployed.


Upgrade Known Issues

  • VCF 4.5.1 with the async patch for NSX 3.2.3 are unable to upgrade to VCF 5.0

    NSX 3.2.3 to upgrade is not supported. It supports only NSX 4.1.1 onwards.

    Workaround: None.

    This issue will be resolved in a future release.

  • VC upgrade fails during install

    When running the workload VC-upgrade from VCF, it fails at the VC install with target vc upgrade precheck stage failing.

    Workaround: Do not reuse the same temporary IP for sequential VC upgrades. Use a separate temporary IP for every VC undergoing upgrade.

  • NSX upgrade may fail if there are any active alarms in NSX Manager

    If there are any active alarms in NSX Manager, the NSX upgrade may fail.

    Workaround: Check the NSX Manager UI for active alarms prior to NSX upgrade and resolve them, if any. If the alarms are not resolved, the NSX upgrade will fail. The upgrade can be retried once the alarms are resolved.

  • Precheck for NSX has failed with ERROR message

    If the Upgrade Prerequisite Backup SDDC Manager, all vCenter Servers, and NSX Managers is ignored and the STFPlocation defined for the NSX Configuration Backup does not have enough disk space, then the ERROR Message Precheck for NSX has failed will display. Although the root cause of the error sftp server disk is full appears in the Operations Manager Log, it is not currently displayed in SDDC Manager.

    Workaround: Increase the amount of available disk space on the SFTP server and then complete the Upgrade Prerequisite before proceeding.

  • Lifecycle Management Precheck does not throw an error when NSX Manager inventory is out of sync

    Workaround None.

  • The SDDC Manager bundle details should show as the "Required Version" for the skip level upgrade

    During a skip-level upgrade from VMware Cloud Foundation to 5.0, there is an error in the SDDC Manager bundle details that incorrectly lists VMware Cloud Foundation 4.5.0 as the "Required Version."

    Workaround: None

  • VCF ESXi upgrade fails during post validation due to HA related cluster configuration issue

    The upgrade of ESXi Cluster fails with error that is similar to below error message:

    Cluster Configuration Issue: vSphere HA failover operation in progress in cluster <cluster-name> in datacenter <datacenter-name>: 0 VMs being restarted, 1 VMs waiting for a retry, 0 VMs waiting for resources, 0 inaccessible vSAN VMs

    Workaround: See KB article 90985.

  • SDDC Manager upgrade fails at "Setup Common Appliance Platform"

    If a virtual machine reconfiguration task (for example, removing a snapshot or running a backup) is taking place in the management domain at the same time you are upgrading SDDC Manager, the upgrade may fail.

    Workaround: Schedule SDDC Manager upgrades for a time when no virtual machine reconfiguration tasks are happening in the management domain. If you encounter this issue, wait for the other tasks to complete and then retry the upgrade.

  • Parallel upgrades of vCenter Server are not supported

    If you attempt to upgrade vCenter Server for multiple VI workload domains at the same time, the upgrade may fail while changing the permissions for the vpostgres configuration directory in the appliance. The message chown -R vpostgres:vpgmongrp /storage/archive/vpostgres appears in the PatchRunner.log file on the vCenter Server Appliance.

    Workaround: Each vCenter Server instance must be upgraded separately.

  • When you upgrade VMware Cloud Foundation, one of the vSphere Cluster Services (vCLS) agent VMs gets placed on local storage

    vSphere Cluster Services (vCLS) ensures that cluster services remain available, even when the vCenter Server is unavailable. vCLS deploys three vCLS agent virtual machines to maintain cluster services health. When you upgrade VMware Cloud Foundation, one of the vCLS VMs may get placed on local storage instead of shared storage. This could cause issues if you delete the ESXi host on which the VM is stored.

    Workaround: Deactivate and reactivate vCLS on the cluster to deploy all the vCLS agent VMs to shared storage.

    1. Check the placement of the vCLS agent VMs for each cluster in your environment.

      1. In the vSphere Client, select Menu > VMs and Templates.

      2. Expand the vCLS folder.

      3. Select the first vCLS agent VM and click the Summary tab.

      4. In the Related Objects section, check the datastore listed for Storage. It should be the vSAN datastore. If a vCLS agent VM is on local storage, you need to deactivate vCLS for the cluster and then re-enable it.

      5. Repeat these steps for all vCLS agent VMs.

    2. Deactivate vCLS for clusters that have vCLS agent VMs on local storage.

      1. In the vSphere Client, click Menu > Hosts and Clusters.

      2. Select a cluster that has a vCLS agent VM on local storage.

      3. In the web browser address bar, note the moref id for the cluster.

        For example, if the URL displays as https://vcenter-1.vrack.vsphere.local/ui/app/cluster;nav=h/urn:vmomi:ClusterComputeResource:domain-c8:503a0d38-442a-446f-b283-d3611bf035fb/summary, then the moref id is domain-c8.

      4. Select the vCenter Server containing the cluster.

      5. Click Configure > Advanced Settings.

      6. Click Edit Settings.

      7. Change the value for config.vcls.clusters.<moref id>.enabled to false and click Save.

        If the config.vcls.clusters.<moref id>.enabled setting does not appear for your moref id, then enter its Name and false for the Value and click Add.

      8. Wait a couple of minutes for the vCLS agent VMs to be powered off and deleted. You can monitor progress in the Recent Tasks pane.

    3. Enable vCLS for the cluster to place the vCLS agent VMs on shared storage.

      1. Select the vCenter Server containing the cluster and click Configure > Advanced Settings.

      2. Click Edit Settings.

      3. Change the value for config.vcls.clusters.<moref id>.enabled to true and click Save.

      4. Wait a couple of minutes for the vCLS agent VMs to be deployed and powered on. You can monitor progress in the Recent Tasks pane.

    4. Check the placement of the vCLS agent VMs to make sure they are all on shared storage

  • SDDC Manager UI issues when running multiple parallel upgrade prechecks

    If you initiate a precheck on more than one workload domain at the same time, the SDDC Manager UI may flicker and show incorrect information.

    Workaround: Do not run multiple parallel prechecks. If you do, wait until the prechecks are complete to evaluate the results.

  • Upgrade precheck results for ESXi display the error "TPM 2.0 device detected but a connection cannot be established."

    This issue can occur for ESXi hosts that have Trusted Platform Modules (TPM) chips partially configured.

    Workaround: Ensure that the TPM is configured in the ESXi host's BIOS to use the SHA-256 hashing algorithm and the TIS/FIFO (First-In, First-Out) interface and not CRB (Command Response Buffer). For information about setting these required BIOS options, refer to the vendor documentation.

  • Performing parallel prechecks on multiple workload domains that use vSphere Lifecycle Manager images may fail

    If you perform parallel prechecks on multiple workload domains that use vSphere Lifecycle Manager images at the same time as you are peforming parallel upgrades, the prechecks may fail.

    Workaround: Use the following guidance to plan your upgrades and prechecks for workload domains that use vSphere Lifecycle Manager images.

    • For parallel upgrades, VMware Cloud Foundation supports up to five workload domains with up to five clusters each.

    • For parallel prechecks, VMware Cloud Foundation supports up to three workload domains with up to four clusters each.

    • Do not run parallel upgrades and prechecks at the same time.

  • Using the /v1/upgrades API to trigger parallel cluster upgrades across workload domains in a single API call does not upgrade the clusters in parallel

    When using the VMware Cloud Foundation API to upgrade multiple workload domains in parallel, including multiple resource upgrade specifications (resourceUpgradeSpec) in a single domain upgrade API (/v1/upgrades) call does not work as expected.

    Workaround: To get the best performance when upgrading multiple workload domains in parallel using the VMware Cloud Foundation API, do not include multiple resource upgrade specifications (resourceUpgradeSpec) in a single domain upgrade call. Instead, invoke the domain upgrade multiple times with a single resourceUpgradeSpec for each workload domain.

    You can also use the SDDC Manager UI to trigger multiple parallel upgrades across workload domains.

  • NSX Data Center upgrade fails at "NSX T PERFORM BACKUP"

    If you did not change the destination of NSX Manager backups to an external SFTP server, upgrades may fail due to an out-of-date SSH fingerprint for SDDC Manager.


    1. Log in to the NSX Manager UI.

    2. Click System > Backup & Restore.

    3. Click Edit for the SFTP Server.

    4. Remove the existing SSH fingerprint and click Save.

    5. Click Add to add the server provided fingerprint.

    6. Click Save.

    7. Retry the NSX Data Center upgrade from the SDDC Manager UI.

  • Cluster-level ESXi upgrade fails

    Cluster-level selection during upgrade does not consider the health status of the clusters and may show a cluster's status as Available, even for a faulty cluster. If you select a faulty cluster, the upgrade fails.

    Always perform an update precheck to validate the health status of the clusters. Resolve any issues before upgrading.

  • You are unable to update NSX Data Center in the management domain or in a workload domain with vSAN principal storage because of an error during the NSX transport node precheck stage

    In SDDC Manager, when you run the upgrade precheck before updating NSX Data Center, the NSX transport node validation results with the following error.

    No coredump target has been configured. Host core dumps cannot be saved.:System logs on host sfo01-m01-esx04.sfo.rainpole.io are stored on non-persistent storage. Consult product documentation to configure a syslog server or a scratch partition.

    Because the upgrade precheck results with an error, you cannot proceed with updating the NSX Data Center instance in the domain. VMware Validated Design supports vSAN as the principal storage in the management domain. However, vSAN datastores do no support scratch partitions. See VMware Knowledge Base article 2074026.

    Disable the update precheck validation for the subsequent NSX Data Center update.

    1. Log in to SDDC Manager as vcf using a Secure Shell (SSH) client.

    2. Open the application-prod.properties file for editing: vi /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties

    3. Add the following property and save the file: lcm.nsxt.suppress.prechecks=true

    4. Restart the life cycle management service: systemctl restart lcm

    5. Log in to the SDDC Manager user interface and proceed with the update of NSX Data Center.

  • NSX-T upgrade may fail at the step NSX T TRANSPORT NODE POSTCHECK STAGE

    NSX-T upgrade may not proceed beyond the NSX T TRANSPORT NODE POSTCHECK STAGE.

    Contact VMware support.

  • ESXi upgrade fails with the error "Incompatible patch or upgrade files. Please verify that the patch file is compatible with the host. Refer LCM and VUM log file."

    This error occurs if any of the ESXi hosts that you are upgrading have detached storage devices.

    Workaround: Attach all storage devices to the ESXi hosts being upgraded, reboot the hosts, and retry the upgrade.

  • Update precheck fails with the error "Password has expired"

    If the vCenter Single Sign-On password policy specifies a maximum lifetime of zero (never expires), the precheck fails.

    Workaround: Set the maximum lifetime password policy to something other than zero and retry the precheck.

Bring-up Known Issues

  • Cannot log in to the VMware Cloud Builder appliance

    If you deploy the VMware Cloud Builder appliance with an 8-character admin or root password, you are not able to log in to the appliance and you see the error Invalid username or password.

    Workaround: Redeploy the VMware Cloud Builder appliance with passwords that include at least 12 characters.

  • Bring-up Network Configuration Validation fails with "Gateway IP Address for Management is not contactable"

    The following failure "Gateway IP Address for MANAGEMENT is not contactable" is reported as fatal error in Cloud Buider UI and bring-up cannot continue. In some cases the validation fails to validate connectivity because it uses as set of predefined ports, however, ping is working.

    Workaround: See KB 89990 for more information.

SDDC Manager Known Issues

  • New - SDDC Manager incorrectly renders Dark Mode due to inheriting OS settings in VCF 5.0

    SDDC Manager UI does not natively support Dark Mode. It attempts to render Dark Mode based on the OS settings and may cause issues with rendering some screens.

    Workaround: There are two possible ways to workaround this issue:

    • Turn off Dark Mode at the OS level and clear the browser cache, or

    • Execute the following script in the Developer console of the browser you are using (replacing the Domain field with the FQDN of your system):

    document.cookie = 'clarity-theme=Light; Max-Age=2147483647; path=/; Domain=sddc-manager.vrack.vsphere.local; SameSite=Lax'

  • SDDC Manager license key is not needed

    A SDDC manager license key is not needed, and any errors seen related to an existing SDDC manager license key can be ignored.

    Workaround: None.

  • vRealize Operations Manager admin account appears as disconnected

    SDDC Manager incorrectly shows the vRealize Operations Manager admin account as disconnected due to an expired password. The admin account password used for logging into the vRealize Operations Manager UI never expires, but SDDC Manager is actually checking the virtual appliance (Photon OS) admin account password.

    Workaround: To clear the expired password/disconnected alert in SDDC Manager:

    1. Log in to the affected vRealize Operations Manager node and update the virtual appliance admin password.

    2. In SDDC Manager, remediate (or rotate or update) the password for the expired account. Or, use the VMware Cloud Foundation API to run POST /v1/credentials/expirations.

  • SDDC Manager UI always shows Local OS as default identity source

    If you add Active Directory over LDAP or OpenLDAP as an identity source in SDDC Manager and use the vSphere Client to set that identity source as the default, the SDDC Manager UI (Administration > Single Sign On > Identity Provider) continues to show Local OS as the default identity source.

    Workaround: Use the vSphere Client to confirm the actual default identity source.

  • Name resolution fails when configuring the NTP server

    Under certain conditions, name resolution may fail when you configure an NTP server.

    Workaround: Run the following command using the FQDN of the failed resource(s) to ensure name resolution is successful and then retry the NTP server configuration.

    nslookup <FQDN>
  • Generate CSR task for a component hangs

    When you generate a CSR, the task may fail to complete due to issues with the component's resources. For example, when you generate a CSR for NSX Manager, the task may fail to complete due to issues with an NSX Manager node. You cannot retry the task once the resource is up and running again.

    1. Log in to the UI for the component to troubleshoot and resolve any issues.

    2. Using SSH, log in to the SDDC Manager VM with the user name vcf.

    3. Type su to switch to the root account.

    4. Run the following command: systemctl restart operationsmanager

    5. Retry generating the CSR.

  • SoS utility options for health check are missing information

    Due to limitations of the ESXi service account, some information is unavailable in the following health check options:

    • --hardware-compatibility-report: No Devices and Driver information for ESXi hosts.

    • --storage-health: No vSAN Health Status or Total no. of disks information for ESXi hosts.


Workload Domain Known Issues

  • SDDC Manager UI incorrectly shows "Owner" of Isolated Workload Doamin as the management domain SSO admin user.

    When creating an isolated workload domain, the "Workload Domains" view in the UI incorrectly shows the "Owner" as the management domain SSO admin user.

    Workaround: The domain info can be seen under "SSO Domain" as a column. To include the SSO Domain as a column, select it in the table configuration to show it as a column. The Owner column on the UI is not applicable for Isolated Workload Domains.

  • An unxpected replication agreement is still left after the successful decommission of the SSO node task

    The decommission of the SSO node was successful, but the replication partner remained.

    Workaround: Manually remove the replication agreement of the invalid SSO node, and restart the delete workload domain.

    The vCenter command /usr/lib/vmware-vmdir/bin/vdcrepadmin can be used to add/remove replication agreements.

  • Heterogeneous operations "Cluster Creation" and "VI Creation" are not supported to be ran in parallel when they are operating against same shared NSX instance.

    If there is a running VI Creation workflow operating on an NSX resource, then creating a cluster on domains that are sharing that NSX is not possible.

    Workaround: None. The VI Creation workflow should complete before the cluster creation workflow can be started.

  • SDDC Manager UI allows you to select an unsupported NSX Manager instance

    When you create a new VI workload domain, it cannot share an NSX Manager instance with a VI workload domain that is in different SSO domain. Although the SDDC Manager UI allows you to select an unsupported NSX Manager instance, the VI workload domain creation task will fail.

    Workaround: None

  • Adding host fails when host is on a different VLAN

    A host add operation can sometimes fail if the host is on a different VLAN.

    1. Before adding the host, add a new portgroup to the VDS for that cluster.

    2. Tag the new portgroup with the VLAN ID of the host to be added.

    3. Add the Host. This workflow fails at the "Migrate host vmknics to dvs" operation.

    4. Locate the failed host in vCenter, and migrate the vmk0 of the host to the new portgroup you created in step 1. For more information, see Migrate VMkernel Adapters to a vSphere Distributed Switch in the vSphere product documentation.

    5. Retry the Add Host operation.

    NOTE: If you later remove this host in the future, you must manually remove the portgroup as well if it is not being used by any other host.

  • Deploying partner services on an NSX workload domain displays an error

    Deploying partner services, such as McAfee or Trend, on a workload domain enabled for vSphere Update Manager (VUM), displays the “Configure NSX at cluster level to deploy Service VM” error.

    Attach the Transport node profile to the cluster and try deploying the partner service. After the service is deployed, detach the transport node profile from the cluster.

  • If the witness ESXi version does not match with the host ESXi version in the cluster, vSAN cluster partition may occur

    vSAN stretch cluster workflow does not check the ESXi version of the witness host. If the witness ESXi version does not match the host version in the cluster, then vSAN cluster partition may happen.

    1. Upgrade the witness host manually with the matching ESXi version using the vCenter VUM functionality.

    2. Replace or deploy the witness appliance matching with the ESXi version.

  • vSAN partition and critical alerts are generated when the witness MTU is not set to 9000

    If the MTU of the witness switch in the witness appliance is not set to 9000, the vSAN stretch cluster partition may occur.

    Set the MTU of the witness switch in the witness appliance to 9000 MTU.

  • Adding a host to a vLCM-enabled workload domain configured with the Dell Hardware Support Manager (OMIVV) fails

    When you try to add a host to a vSphere cluster for a workload domain enabled with vSphere Lifecycle Manager (vLCM), the task fails and the domain manager log reports "The host (host-name) is currently not managed by OMIVV." The domain manager logs are located at /var/log/vmware/vcf/domainmanager on the SDDC Manager VM.

    Update the hosts inventory in OMIVV and retry the add host task in the SDDC Manager UI. See the Dell documentation for information about updating the hosts inventory in OMIVV.

  • Adding a vSphere cluster or adding a host to a workload domain fails

    Under certain circumstances, adding a host or vSphere cluster to a workload domain fails at the Configure NSX Transport Node or Create Transport Node Collection subtask.

    1. Enable SSH for the NSX Manager VMs.

    2. SSH into the NSX Manager VMs as admin and then log in as root.

    3. Run the following command on each NSX Manager VM: sysctl -w net.ipv4.tcp_en=0

    4. Login to NSX Manager UI for the workload domain.

    5. Navigate to System > Fabric > Nodes > Host Transport Nodes.

    6. Select the vCenter server for the workload domain from the Managed by drop-down menu.

    7. Expand the vSphere cluster and navigate to the transport nodes that are in a partial success state.

    8. Select the check box next to a partial success node, click Configure NSX.

    9. Click Next and then click Apply.

    10. Repeat steps 7-9 for each partial success node.

    When all host issues are resolved, transport node creation starts for the failed nodes. When all hosts are successfully created as transport nodes, retry the failed add vSphere cluster or add host task from the SDDC Manager UI.

  • The vSAN Performance Service is not enabled for vSAN clusters when CEIP is not enabled

    If you do not enable the VMware Customer Experience Improvement Program (CEIP) in SDDC Manager, when you create a workload domain or add a vSphere cluster to a workload domain, the vSAN Performance Service is not enabled for vSAN clusters. When CEIP is enabled, data from the vSAN Performance Service is provided to VMware and this data is used to aid VMware Support with troubleshooting and for products such as VMware Skyline, a proactive cloud monitoring service. See Customer Experience Improvement Program for more information on the data collected by CEIP.

    Enable CEIP in SDDC Manager. See the VMware Cloud Foundation Documentation. After CEIP is enabled, a scheduled task that enables the vSAN Performance Service on existing clusters in workload domains runs every three hours. The service is also enabled for new workload domains and clusters. To enable the vSAN Performance Service immediately, see the VMware vSphere Documentation.

  • Creation or expansion of a vSAN cluster with more than 32 hosts fails

    By default, a vSAN cluster can grow up to 32 hosts. With large cluster support enabled, a vSAN cluster can grow up to a maximum of 64 hosts. However, even with large cluster support enabled, a creation or expansion task can fail on the sub-task Enable vSAN on vSphere Cluster.

    1. Enable Large Cluster Support for the vSAN cluster in the vSphere Client. If it is already enabled skip to step 2.

      1. Select the vSAN cluster in the vSphere Client.

      2. Select Configure > vSAN > Advanced Options.

      3. Enable Large Cluster Support.

      4. Click Apply.

      5. Click Yes.

    2. Run a vSAN health check to see which hosts require rebooting.

    3. Put the hosts into Maintenance Mode and reboot the hosts.

    For more information about large cluster support, see https://kb.vmware.com/kb/2110081.

  • Removing a host from a cluster, deleting a cluster from a workload domain, or deleting a workload domain fails if Service VMs (SVMs) are present

    If you deployed an endpoint protection service (such as guest introspection) to a cluster through NSX Data Center, then removing a host from the cluster, deleting the cluster, or deleting the workload domain containing the cluster will fail on the subtask Enter Maintenance Mode on ESXi Hosts.

    • For host removal: Delete the Service VM from the host and retry the operation.

    • For cluster deletion: Delete the service deployment for the cluster and retry the operation.

    • For workload domain deletion: Delete the service deployment for all clusters in the workload domain and retry the operation.

  • vCenter Server overwrites the NFS datastore name when adding a cluster to a VI workload domain

    If you add an NFS datastore with the same NFS server IP address, but a different NFS datastore name, as an NFS datastore that already exists in the workload domain, then vCenter Server applies the existing datastore name to the new datastore.

    If you want to add an NFS datastore with a different datastore name, then it must use a different NFS server IP address.

API Known Issues

  • LCM debug log (lcm-debug.log) spammed with error message "does not have a valid product api version string"

    This issue is caused by the release version format issue of the vRealize interoperability data. As a result, the logs will reset frequently.

    Workaround: None.

    This issue will be resolved in a future release.

  • An API for NSX Clusters is listed on VMware Cloud Foundation Developer Center in error

    Public API "2.36.6. Get the transport zones from the NSX cluster" is listed on VMware Cloud Foundation Developer Center in error.

    Workaround: None

  • Stretch cluster operation fails

    If the cluster that you are stretching does not include a powered-on VM with an operating system installed, the operation fails at the "Validate Cluster for Zero VMs" task.

    Make sure the cluster has a powered-on VM with an operating system installed before stretching the cluster.

check-circle-line exclamation-circle-line close-line
Scroll to top icon