VMware Cloud Director 10.4.2 Release Notes

VMware Cloud Director 10.4.2 \| 18 APR 2023 \| Build 21596272 (installed build 21595985) Check for additions and updates to these release notes.

VMware Cloud Director 10.4.2 | 18 APR 2023 | Build 21596272 (installed build 21595985)

Check for additions and updates to these release notes.

What's New

VMware Cloud Director version 10.4.2 includes the following:

IDP (Identity Provider) Proxy

You can now configure VMware Cloud Director as an identity provider proxy server. You can register an OAuth 2.0 OpenID Connect compliant Identity Provider with VMware Cloud Director, and relying parties can use VMware Cloud Director for tenant-aware authentication of users known to VMware Cloud Director. See Using VMware Cloud Director as an Identity Provider Proxy Server. For more information on the OpenID Connect standard, see OpenID Connect Core 1.0.
Trusted Platform Module support

VMware Cloud Director now supports VMs with Trusted Platform Module (TPM) devices. TPM devices provide enhanced security to the guest operating system and some operating systems, such as Windows 11, require them. A TPM device can be added to a new or existing VMs as long as the VM Guest OS and the underlying vCenter Server infrastructure meet certain pre-requisites: the Guest OS must be Windows Server 2008 and later, Windows 7 and later, or Linux; the VM boot firmware must be UEFI; the underlying vCenter Server instances must support VM encryption and have a Key Provider configured. See the service provider information in Understanding Trusted Platform Module Devices or the tenant information in Working with Virtual Machines.
User Management Enhancements
- To assist with remapping users between identity providers (IDPs), VMware Cloud Director 10.4.2 provides a UI-based bulk user edit option. System administrators can use this feature to remap users to a different IDP by updating their provider type and username, if different. See Remapping Users Between Identity Providers.
- In a future release, the email property will become mandatory. In VMware Cloud Director 10.4.2, empty or null email property for any user is deprecated. This change is a step towards assisting with user notification. You must enter an email for all new users and during system setup. System administrators must review and ensure that these values are present for all existing users.
  - For local users, email must be filled in.
  - For users imported from IDPs, you must ensure that the attributes are appropriately mapped so that VMware Cloud Director retrieves the email during the login process.
vApp Lease Expiry Timeout Defaults to 'Unlimited'

Prior to this release, the vApp lease expiration time defaulted to 7 days. You can modify the lease expiration at the tenant organization level. You must edit each tenant organization in which you want to modify the vApp lease expiration.

Starting with VMware Cloud Director 10.4.2, for newly created tenant organizations, the default lease expiration time setting is 'unlimited'.
vSAN HCI Mesh Placement Policy

VMware Cloud Director prevents VMs from spreading the VM and its disks across multiple datastores if one of the datastores is a remote datastore in HCI Mesh vSAN cluster. This restriction is necessary because vSAN HCI Mesh does not allow a VM to split between datastores if one of the datastores is a HCI Mesh remote datastore. See the Sharing Remote Datastores with HCI Mesh section in Managing the VM Storage Policies on a Provider Virtual Data Center.
Improved Provider Login Experience

In VMware Cloud Director 10.4.2, on the https://vcloud.example.com/ login page, when you enter the word system in the Organization name text box, VMware Cloud Director redirects you to the Service Provider Admin Portal login page.
Improved Tenant Login Experience When SAML or OIDC Are Configured

If an organization in VMware Cloud Director has SAML or OIDC configured, the UI displays only the Sign in with Single Sign-On option. To log in as a local user, navigate to https://vcloud.example.com/tenant/tenant_name/login or https://vcloud.example.com/provider/login.
APIs Under Accelerated Deprecation

VMware Cloud Director API 37.2 (VMware Cloud Director 10.4.2) contains APIs for vRealize Orchestrator 7.0 that are under accelerated deprecation and will be removed in future releases.

Product Support Notices

VMware Cloud Director 10.4.2 supports only vCenter Server 7.0 Update 2 and later releases.

Security Fixes

Photon OS 3.0 Security Updates

VMware Cloud Director appliance version 10.4.2 includes Photon OS 3.0 security updates for advisories up to and including PHSA-2023-3.0-0556. See the Photon OS 3.0 Security Advisories.

Documentation

To access the full set of product documentation, go to VMware Cloud Director Documentation.

Previous Releases of VMware Cloud Director 10.4.x

VMware Cloud Director 10.4.1.1 Release Notes

VMware Cloud Director 10.4.1 Release Notes

VMware Cloud Director 10.4 Release Notes

Resolved Issues

New - VMware Cloud Director does not apply yellow and red storage thresholds on individual datastores in a datastore cluster

If you set thresholds on a datastore cluster, VMware Cloud Director does not apply the thresholds on each individual datastore in the cluster. As a result, the placement engine keeps on placing virtual machines on the datastores even after the threshold is breached.

This is happening because VMware Cloud Director calculates the thresholds as an aggregation on the datastore cluster level.

After upgrading to VMware Cloud Director 10.4.2, for every pre-existing datastore cluster, you must divide the existing threshold to the number of datastores in that cluster.

If you configure the thresholds for new datastore clusters, VMware Cloud Director sets the threshold value on each individual datastore and not as an aggregation on the datastore cluster level.
New - You cannot convert an organization VDC from allocation pool allocation model to flex allocation model
When the allocation model has a maximum compute policy with a 0.0 memory and CPU reservation guarantee, attempting to convert an allocation pool model organization VDC to a flex organization VDC fails with the following error.
```
com.vmware.vcloud.api.presentation.service.BadRequestException: vDC cpu reservation, memory reservation or vCpu speed cannot exceed values defined in vDC maximum compute policy. Maximum Cpu reservation null, Maximum Memory reservation 0, Maximum vCpuSpeed null.
```
New - When you use the VMware Cloud Director UI to create a new VM with a placement policy, all virtual machines that are part of the VM group defined in the used placement policy might disappear

When you use the VMware Cloud Director UI to create a new VM that uses a certain placement policy, all virtual machines listed in the VM group that's defined in the used placement policy might disappear from the VM group.
New - VMware Cloud Director UI and tasks are slow to load and complete

The Artemis message bus communication is not working and when you trigger operations from the UI, they can take up to 5 minutes to complete or might time out. The performance issues can affect operations such as powering on VMs and vApps, provider VDC creation, vApp deployment, and so on.
New - Deleting a VM with an attached named disk results in a NullPointerException error message and the VM remains in the vCenter Server inventory

If you attempt to delete a VM that has an attached named disk, the operation fails with a NullPointerException error message. The VM continues to be shown in the vCenter Server inventory and you cannot detach its named disk.
VMware Cloud Director does not assign the custom configurations of an OVF for reservation, shares, and limits when using the OVF to deploy a VM in a flex organization VDC

If you configure custom values for the reservation, shares, and limits in an OVF and you deploy a new VM by using this OVF in a flex organization VDC, VMware Cloud Director does not honor the custom configurations and assigns the default organization VDC sizing policy to the VM.
Publishing the rights bundle of the defined entity type to tenants takes longer time to complete or times out when you have configured 1000 or more tenant organizations

If you configure 1000 or more tenant organizations, attempting to publish the rights bundle of the defined entity type to all tenants takes long time to complete or times out.

If the rights bundle is already published to 1000 or more tenants, publishing it to a new tenant times out.
Login to VMware Cloud Director as a SAML group user fails with a NullPointer Exception error message

If the list of roles for a SAML group contains an empty entry, SAML login fails with a NullPointer Exception error message.
The VMware Cloud Director UI freezes when you attempt to create or update a distributed firewall rule

When creating and updating a distributed firewall rule, operations such as filtering, sorting, and changing of a page in the Applications, Destination, Context, and Source grid views causes the VMware Cloud Director UI to freeze.
Running a custom workflow with external validation in vRO Workflow Execution UI plug-in fails with an Error performing external validation error message

When running a custom workflow with external validation through the vRO Workflow Execution UI plug-in, the process fails with an Error performing external validation error message. The issue occurs because vRealize Orchestrator does not perform validation on the inputs in the custom form in VMware Cloud Foundation.
Deleting an organization VDC fails with a NullPointerException error message

If the organization VDC page displays the value for the number of users associated with this organization as -1, attempting to delete this organization VDC fails with NullPointerException error message.
Using the VMware Cloud Director quick search to find VMs and update their virtual disks triggers misconfiguration of the VMs settings

When you use the VMware Cloud Director quick search to find a VM and to update its virtual disks, if you navigate to a different VM and update its virtual disk before the completion of the first update, vSphere misconfigures the VMs settings. The misconfiguration can include settings such as VM name, VM description, CPU, memory, networking, and guest OS.
Publishing a vRealize Orchestrator workflow to the VMware Cloud Director service library fails with an error message

When you attempt to publish a vRealize Orchestrator workflow, the operation fails with a 500 Server Error error message.

This happens because the API returns a large number of links for each individual tenant to which the workflow is published and causes an overflow in the HTTP headers.
Moving a vApp to a different organization VDC removes the vApp description

When you move a vApp from one organization VDC to a different organization VDC, the vApp description gets deleted.
Registering an NSX-T Cloud instance with VMware Cloud Director fails with a duplicateKeyException error message

If your environment has multiple NSX-T managers, registering an NSX-T Cloud with VMware Cloud Director fails with a duplicateKeyException error message.

This happens because each NSX-T manager contains a default transport zone named nsx-overlay-transportzone with the same ID, and while obtaining the transport zone, VMware Cloud Director uses this ID. As a result, VMware Cloud Director detects duplicate values.
An attempt to obtain the media records by running a VMware Cloud Director API request does not return the description for the media

When you run a VMware Cloud Director API request to obtain a media record, the response does not contain information about the media description.
Enabling the guest customization for an Ubuntu VM with IP mode set to DHCP fails

When creating a vApp from an Ubuntu template with IP mode set to DHCP, enabling the guest customization on the resulting vApp fails.

The /var/log/vmware-imc/toolsDeployPkg.log file displays an error message.

Customization command failed with stderr: 'dpkg: warning: version '^A.0.0' has bad syntax: version number does not start with digit'.
Publish a Provider VDC Kubernetes Policy to an Organization VDC fails with an error message

When you attempt to publish a provider VDC Kubernetes policy to an organization VDC, the VMware Cloud Director UI initially displays the operation as successful, but after a few seconds displays an error message.

Caused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(256)
Adding, changing, and removing a vApp owner results in a cannot change vApp ownership because VMs are attached to named disk error message

If you attach a named disk to VMs in a vApp, operations such as adding, changing, and removing the owner of the vApp results in an error message.

cannot change vApp ownership because VMs are attached to named disk
You cannot set the lease for a vApp template to Never Expires

When you attempt to set the lease for a vApp template to Never Expires, nothing happens and the lease expiry date remains unchanged.
If you use vRealize Orchestrator 8.x, hidden input parameters in workflows are not populated automatically in the VMware Cloud Director UI

If you use vRealize Orchestrator 8.x, when you attempt to run a workflow through the VMware Cloud Director UI, hidden input parameters are not populated automatically in the VMware Cloud Director UI.
Expanding the virtual disk of a VM fails with a Task VAPP_UPDATE_VM terminated abruptly error message

If a storage policy is backed by a datastore and datastore cluster, and you apply the same storage policy to a VM, expanding the virtual disk of the VM fails with an error message.

Task VAPP_UPDATE_VM terminated abruptly

This happens when the virtual disk cannot fit on the datastore and VMware Cloud Director must place the VM disk on the datastore cluster.
A VM with IP mode set to DHCP might not be able to connect to an external network

If a VM with IP mode set to DHCP is connected to a vApp network that uses port forwarding, the VM cannot connect to an external network. This happens because in NSX-backed organization VDCs, enabling IP masquerading for a vApp network does not create a corresponding SNAT rule on the vApp edge in NSX to allow outbound access for a VM without a static IP.
Updating the VM NIC from Static-Pool to Static-Manual IP mode does not apply the change

In a VM, if the VM NIC is connected to a vApp direct network and you attempt to update the IP mode from Static-Pool to Static-Manual while keeping the same IP address, the system does not save the change and the configuration of the IP mode remains Static-Pool.
Deployment to vCenter Server of an OVA that is exported from VMware Cloud Director fails with an Issues detected with selected template. Details: - 107:17:VALUE_ILLEGAL: Duplicate value ''1'' for element ''Address'' error message

If you export an OVA from VMware Cloud Director and you attempt to deploy the same OVA to vCenter Server, the operation fails with an error message.

Issues detected with selected template. Details: - 107:17:VALUE_ILLEGAL: Duplicate value ''1'' for element ''Address''

This happens because the OVF template contains a duplicate line for the Address element.
Publishing a Guided Tour unpublishes the global tenant roles from the organization

If a system administrator publishes global roles to all tenants in an organization, and then publishes a Guided Tour, the global roles get unpublished and are no longer active for the tenants.

Known Issues

New - When using the CloudAPI to create or update an organization, you cannot set to true the canPublish flag

When using the CloudAPI to create an organization or update an organization enabling it to publish catalogs, the canPublish field remains false, despite you setting the value to true. The legacy API is not affected.

Workaround: Use the VMware Cloud Director UI to activate or deactivate the option to Publish catalog externally for an organization.
New - VMware Cloud Director backup fails

If you use Ubuntu or Linux distributions that are based on Debian as NFS for the VMware Cloud Director appliance, the NFS server cannot be configured appropriately to support the creation of backups through the PostgreSQL user.
Workaround: Depending on the file that the appliance has, run the following commands from appliance's secure shell as the root user.
- If the appliance has the /opt/vmware/appliance/bin/create-db-backup file, run the following command.
sed -i '/PG_BACK_UP() {/,/}/ { /PG_BACK_UP() {/!{ /}/!d }}; /PG_BACK_UP() {/ a\su - postgres -c "$VMWARE_POSTGRES_BIN\/pg_dump -v -Fc \$DBNAME" > \$DB_DUMP_PATH 2>> \$LOG_FILE' /opt/vmware/appliance/bin/create-db-backup
- If the appliance has the /opt/vmware/appliance/bin/create-backup.sh file, run the following commands.
sed -i '/DB_BACKUP() {/,/}/ { /DB_BACKUP() {/!{ /}/!d }}; /DB_BACKUP() {/ a\su - postgres -c "$VMWARE_POSTGRES_BIN\/pg_dump -v -Fc \$DBNAME" > \$DB_DUMP_PATH 2>> \$LOG_FILE' /opt/vmware/appliance/bin/create-backup.sh

sed -i '/DB_USER_BACKUP() {/,/}/ { /DB_USER_BACKUP() {/!{ /}/!d }}; /DB_USER_BACKUP() {/ a\su - postgres -c "$VMWARE_POSTGRES_BIN\/pg_dumpall --roles-only | grep -e '\''CREATE ROLE vcloud;\\|ALTER ROLE vcloud WITH'\''" > \$BACKUP_DIR\/vcloud-user.sql' /opt/vmware/appliance/bin/create-backup.sh
New - After upgrading a VMware Cloud Director appliance, the management API and the management UI report an incorrect older version of the appliance

The problem occurs because the VMware Cloud Director appliance management API uses a different source of truth for obtaining the current version of the VMware Cloud Director appliance than the vamicli version --appliance command. This alternate source of truth is not always being updated during the appliance upgrade causing incorrect information to appear.

Workaround: Use the vamicli version --appliance command to verify the VMware Cloud Director appliance version.
New - Any update to a VM triggers a relocation even when the current location can still accommodate the VM

The issue occurs because of missing per-disk datastore requirements that also pin the disks.

Workaround: Deactivate Storage DRS.
New - When you deploy a VM from a template with a storage policy that includes a configured IOPS limit, after deployment, the VM disks do not have an IOPS limit configured or have a different IOPS limit

The problem occurs because the the I/O Operations Per Second (IOPS) limit set in the VM template overrides the storage policy's IOPS limit. For example, if the VM template does not have a configured IOPS value, after deployment, the VM disks do not have a configured IOPS limit.

Workaround: You can use vApp templates, or edit the VM after deployment.
New - VMware Cloud Director assigns the vGPU policy to a newly deployed virtual machine tagged with the general purpose policy

If you add a newly deployed virtual machine to a vApp and you tag the virtual machine with the general purpose policy, VMware Cloud Director assigns the vGPU policy instead.

Workaround: None.
New - When sharing vApps with users, you can navigate to nonexistent pages

When sharing vApps with users, the buttons to go to the previous or next page are available even though you are already on the respective first or last page. As a result, you can navigate to pages that do not exist and do not have content.

Workaround: None.
New - The VMware Cloud Director API does not return some provider VDCs as merge candidates

If you attempt to get merge candidates for a provider VDC and there are more provider VDCs in the system than the value specified in the page size query parameter, the merge candidate API only processes the first page size number of provider VDCs to check if they are merge candidates and ignores the other provider VDCs in the system.

Workaround: To ensure the VMware Cloud Director API processes all the provider VDCs, specify a page size greater than or equal to the number of provider VDCs in the system.
New - Synchronization of subscribed catalogs fails with a URISyntaxException error

If you create a VM template with spaces or special characters in its name in vCenter, when you subscribe to the vCenter content library from VMware Cloud Director, the synchronization fails.

Workaround: Remove any spaces or special characters from the VM template names.
New - Attempting to modify the port for the NSX Edge load balancer pool fails with an INTERNAL_SERVER_ERROR

After you delete a virtual service, trying to update the pool which was previously connected to the deleted virtual service fails with an INTERNAL_SERVER_ERROR. For example, changing the port for the pool fails.

Workaround: None.
New - Deleting an organization in VMware Cloud Director UI fails with a You must delete this Organization's Application Port Profiles before you can delete the organization error
If application port profiles are created on an edge gateway associated with an organization, attempting to delete the organization fails. The issue occurs because VMware Cloud Director deletes the edge gateways before deleting the port profiles, which causes the following error.
```
com.vmware.vcloud.api.presentation.service.InvalidStateException: You must delete this Organization's Application Port Profiles before you can delete the organization.
```
Workaround: Use the VMware Cloud Director API to force delete an organization and to delete the stranded application port profiles associated with it. See Delete Stranded Application Port Profiles from VMware Cloud Director.
New - You cannot access the Service Provider Admin Portal and the VMware Cloud Director Tenant Portal after rebooting the VMware Cloud Director VM

If you reboot the VMware Cloud Director VM by using a method other than using the vSphere Client, for example, by using vSphere High Availability or VMware Host Client, you cannot access the Service Provider Admin Portal and the VMware Cloud Director Tenant Portal. The problem occurs because after the reboot, the deployment OVF parameters are deleted from the ovfEnv.xml file, and the cell cannot be accessed.

Workaround: Power off and then power on the VMware Cloud Director VM by using the vSphere Client.
New - The VMware Cloud Director proxy does not work when the proxy is configured for a vCenter instance registered with a URL containing port 443

When a vCenter instance is registered with a URL containing port 443, for example, https://vcenter.com:443, and you configure a proxy for that vCenter instance, VMware Cloud Director does not use the proxy and the java.net.SocketTimeoutException: connect timed out error appears in the logs.

Workaround: Remove the 443 port from the vCenter URL.
New - Changing the primary IP address of an NSX edge gateway fails with a Cannot allocate multiple primary IP addresses for NSX-T Edge Gateway error
When an NSX edge gateway has multiple IPv4 and IPv6 subnets, changing the primary IP address of the edge gateway using the VMware Cloud Director UI fails with an error similar to the following.
```
Error: [ e4dc8a76-86ea-408b-b2aa-27ea42305ec2 ] Cannot allocate multiple primary IP addresses for NSX-T Edge Gateway edgw1(com.vmware.vcloud.entity.gateway:8d17e279-2e4a-4452-b0c7-4a410a304374).
```
Workaround: You can use the VMware Cloud Director API.

New - Provisioning of a new Tanzu Kubernetes Grid Service cluster might fail

When attempting to provision a Tanzu Kubernetes Grid Service cluster, the operation might fail. The corresponding user task shows a status error message similar to the following.

[ <some unique id> ] An operation in vSphere for Kubernetes failed, reason message: Bad Request - admission webhook "default.validating.tanzukubernetescluster.run.tanzu.vmware.com" denied the request: Spec.Topology.ControlPlane.TKR.Reference.Name unable to resolve that TKR due to could not resolve TKR/OSImage for controlPlane, machineDeployments: [workers], query: {controlPlane: {k8sVersionPrefix: 'v1.23.8+vmware.3-tkg.1.ubuntu', tkrSelector: '', osImageSelector: 'os-name=photon'}, machineDeployments: [{k8sVersionPrefix: 'v1.23.8+vmware.3-tkg.1.ubuntu', tkrSelector: '', osImageSelector: 'os-name=photon'}]}, result: {controlPlane: {k8sVersion: '', tkrName: '', osImagesByTKR: map[]}, machineDeployments: [{k8sVersion: '', tkrName: '', osImagesByTKR: map[]}]} - Bad Request

The problem is related to a vSphere Supervisor cluster API backward compatibility issue that breaks the integration with VMware Cloud Director. The compatibility is completely broken for with vSphere 8.x and later updates, and partially broken for vSphere 7.x updates.

Workaround: None.

New - VM does not receive the DNS Server IP addresses from the DHCP scope that is defined in the vApp network

When you connect a VM to a routed vApp network in DHCP IP mode, the VM does not receive the DNS addresses defined in the DHCP scope.

Workaround: Using NSX Manager, manually configure the DNS servers in the routed vApp network segment.
New - If you shut down the guest OS before deleting a VM, VMware Cloud Director cannot reuse the VM's IP address from the static pool of IP addresses
If you shut down the guest OS of a VM with a static pool IP allocation before deleting the VM, VMware Cloud Director does not release the IP address back into the IP pool, resulting in IP exhaustion. Creating a scale group or a VM with this IP address might fail with the following error.
```
No Static IP Pools or no free IP within any Static IP Pool to allocate to VM nic at index 0.
```
Workaround: Instead of shutting down the guest OS, power off the VM and then, delete it.
New - API clients throw Invalid mime type errors for responses from multisite VMware Cloud Director APIs
If the multisite field in the response header values specifies a list of organizations, the API client generates the following error.
```
org.springframework.util.InvalidMimeTypeException: Invalid mime type 
```
The issue occurs because the VMware Cloud Director API returns an illegal @ character in the MIME (Multipurpose Internet Mail Extensions) type headers of the response. You can ignore the error because VMware Cloud Director continues to function properly.
Workaround: None.
New - Creating custom application port profiles results in a Bad Request error message

When creating custom application port profiles on a VMware NSX edge gateway, if you configure more than 15 ports, the operation fails with an error message.

Bad Request: Field level validation errors: {service_entries[0].destination_ports has exceeded maximum size 15}, error code 255

The port profile is created and any subsequent update operation on it results in a Required operation parameter 'service_id' is missing error message.

Workaround: Do not configure more than 15 ports for custom application port profiles on VMware NSX edge gateways.
New - The VMware Cloud Director appliance database disk resize script might fail if the backing SCSI disk identifier changes

The database disk resize script runs successfully only if the backing database SCSI disk ID remains the same. If the ID changes for any reason, the script might appear to run successfully but fails. The /opt/vmware/var/log/vcd/db_diskresize.log shows that the script fails with a No such file or directory error.
Workaround:
1. Log in directly or by using an SSH client to the primary cell as root.
2. Run the lsblk --output NAME,FSTYPE,HCTL command.
3. In the output, find the disk containing the database_vg-vpostgres partition and make note of its ID. The ID is under the HCTL column and has the following sample format 2:0:3:0.
4. In the db_diskresize.sh script, modify the partition ID with the ID from Step 3. For example, if the ID is 2:0:3:0, in line
```
echo 1 > /sys/class/scsi_device/2\:0\:2\:0/device/rescan
```
  you must change the ID to 2:0:3:0.
```
echo 1 > /sys/class/scsi_device/2\:0\:3\:0/device/rescan
```
5. Аfter saving the changes, manually re-invoke the resize script or reboot the appliance.
New - Deleting auto-discovered VMs from VMware Cloud Director moves the existing VMs in vApps to the StrandedItems folder and renames them

When you delete the auto-discovered VMs from VMware Cloud Director, the system moves the existing VMs that reside in vApps to the StrandedItems folder in vCenter Server and renames the vCenter Server managed VMs with a suffix before the VMs UUID, similar to vcentervm-1 (vm-uuid).

Workaround: None.
New - Upgrading to VMware Cloud Director 10.4.1 or later fails with a Fix postgres user home directory error
When you try to upgrade to VMware Cloud Director 10.4.1 or later, the upgrade fails. The update-postures-db.log contains the following error.
```
2023-05-15 16:38:01 | update-postgres-db.sh | Fix postgres user home directory
usermod: user postgres is currently used by process 17236
```
Other processes that are logged in as the postgres user on the VMware Cloud Director appliance might block the script that upgrades the PostgreSQL major version from 10 to 14.
Workaround:
1. Before starting the VMware Cloud Director upgrade, find any processes that are logged in as the postgres user on the VMware Cloud Director appliance by running ps -u postgres on the appliance.
2. Stop any process that the command returns by running kill -9 <PID>, where PID is the unique process identifier.
New - Users cannot log in to some organizations after migration to or from the system organization LDAP configuration

If you migrate a user from the shared system organization LDAP configuration to another IDP source, and the reverse, that user cannot log in to any organization other than the one doing the migration. For example, in a deployment where the system organization manages TenantA and TenantB and all organizations import User1 from the shared system organization LDAP configuration, if TenantA sets up a SAML configuration and migrates User1 from LDAP to SAML, then, User1 can log in to TenantA through SAML, but they cannot log in to the system organization or TenantB.

Workaround: None.
New - Creating an organization VDC Kubernetes policy with provider gateways that uses IP spaces fails

If you configure an IP space backed provider gateway and you create a VDC and an edge gateway based on the same IP space, an attempt to create a Kubernetes policy for this VDC fails with an error message.

com.vmware.ssdc.util.LMException: Index 0 out of bounds for length 0

This happens because the IP space backed edge gateways are not associated with a primary IP address, which is required for the creation of SNAT by the Kubernetes policy.

Workaround: Create VDC and edge gateways with NSX network provider type and provider gateways that use legacy IP blocks.
You cannot create a disabled organization using the legacy VMware Cloud Director API
Attempting to use the legacy VMware Cloud Director API organization creation endpoint POST [vcd_public_endpoint]/api/admin/orgs to create a disabled organization results in a 400 BadRequestException containing the following snippet:
```
<Error ... stackTrace="com.vmware.vcloud.api.presentation.service.BadRequestException: Unexpected error.&#10;unexpected end of subtree
```
Workaround: Use the VMware Cloud Director OpenAPI endpoint to create a disabled organization. Alternatively, you can use the UI, OpenAPI, or legacy API to create an enabled organization and disable it after creation.
VMware Cloud Director shows an empty value for the IOPS limit for a VM disk with VC-IOPS enabled storage policy

If you apply a VC-IOPS enabled storage policy with custom reservation, limit, and shares, on a VM disk, VMware Cloud Director displays the values for IOPS reservations, but displays the IOPS limit as empty. This happens because vCenter Server 8U1 introduces a new mechanism for Storage I/O Control (SIOC) which no longer sets the IOPS limit as a VM disk property.

Workaround: None.
Changing the storage policy on a virtual disk of a VM fails with a The operation failed because no suitable resource was found error message

If the virtual disk of a VM resides on a remote vSAN datastore, changing the storage policy of the virtual disk results in an error message.

The operation failed because no suitable resource was found

Workaround: To move the VM to a different storage policy, change the virtual disk storage policy to VM default policy and then change the VM storage policy to the desired storage policy.
Creating an organization VDC template with NSX network provider type and provider gateways that uses IP spaces fails

When you attempt to create an organization VDC template with NSX network provider type and provider gateway that uses IP spaces, the operation fails with the following error. Error:Cannot support external Network that is utilizing IP Spaces. Only external networks with legacy IP blocks are supported.

Workaround: Create organization VDC templates with NSX network provider type and provider gateways that use legacy IP blocks.
When starting the VMware Cloud Director appliance, the message [FAILED] Failed to start Wait for Network to be Configured. See 'systemctl status systemd-networkd-wait-online.service' for details appears

The message appears incorrectly and does not indicate an actual problem with the network. You can disregard the message and continue to use the VMware Cloud Director appliance as usual.

Workaround: None.
If you try to restore the VMware Cloud Director appliance with the console proxy certificates, the restore fails

In the VMware Cloud Director appliance management UI, if you want to restore the appliance and select the Console Proxy check box under Select the certificates to be restored on to this node from the selected backup, the restore fails.

Workaround: Starting with version 10.4, the console proxy and REST API use a single certificate. In version 10.4.1 and later, the legacy console proxy implementation is not supported and selecting the check box is not necessary. Repeat the restore procedure without selecting the Console Proxy check box.
You cannot select Tanzu Kubernetes version 2.0 or later when creating a TKGs cluster

As a tenant, when attempting to create a TKGs cluster, you cannot select a Tanzu Kubernetes cluster version 2.0 and later.

Workaround: To offer and use Tanzu Kubernetes 2.0 and later, use VMware Cloud Director Container Service Extension 4.0.
Migrating VMs between organization VDCs might fail with an insufficient resource error

If VMware Cloud Director is running with vCenter Server 7.0 Update 3h or earlier, when relocating a VM to a different organization VDC, the VM migration might fail with an insufficient resource error even if the resources are available in the target organization VDC.

Workaround: Upgrade vCenter Server to version 7.0 Update 3i or later.
The VMware Cloud Director Tenant Portal UI does not display the IOPS limits and reservations for a vSAN storage policy

vSAN manages itself the IOPS limits on vSAN storage policies. As a result, the VMware Cloud Director Tenant Portal UI does not display the IOPS reservations and limits for a vSAN storage policy and you cannot modify their values.

Workaround: None.
VMware Cloud Director appliance upgrade fails with an invalid version error when FIPS mode is enabled

For VMware Cloud Director versions 10.3.x and later, when FIPS mode is enabled, VMware Cloud Director appliance upgrade fails with the following error.

Failure: Installation failed abnormally (program aborted), the current version may be invalid.
Workaround:
1. Before you upgrade the VMware Cloud Director appliance, deactivate FIPS Mode on the cells in the server group and the VMware Cloud Director appliance. See Activate or Deactivate FIPS Mode on the VMware Cloud Director Appliance.
2. Verify that the /etc/vmware/system_fips file does not exist on any appliance.
3. Upgrade the VMware Cloud Director appliance.
4. Enable FIPS mode again.
Restore from an appliance backup might fail with an Invalid command-line arguments. Missing argument for option: consoleproxy-cert error

If you run the clear-console-proxy-settingsCMT command before you take an appliance backup, then, if you choose to restore the console proxy certificate from the backup, the restore process fails with an Invalid command-line arguments. Missing argument for option: consoleproxy-cert error.

The issue occurs because the command to clear the console proxy settings removes the console proxy certificate, and the console proxy settings are missing for the backup. If the console proxy certificate is not in the backup, you cannot restore it.

If the console proxy settings were cleared, run the appliance restore without selecting to restore the console proxy certificate.
You can't view and edit the license type for your previously registered NSX Advanced Load Balancer Controller instances in the VMware Cloud Director API

You can't view and edit the license for your previously registered NSX Advanced Load Balancer Controller instances in the VMware Cloud Director API. This happens because in VMware Cloud Director 10.4, the Controller license type was replaced by a selection between a Standard and a Premium feature set at the Service Engine Group level to provide more flexibility.

Workaround: Use the supportedFeatureSet path for service engine groups and on edge gateways to activate and deactivate the available features.
You cannot create and use VMware Cloud Director VDC templates in VMware Cloud Director service environments that use VMware Cloud on AWS network pools

If you are using only a provider network pool that is backed by VMware Cloud on AWS for your provider VDC, you cannot create a VDC template and instantiate a VDC from a template. This happens because creating and instantiating VDC templates is supported only for provider VDCs backed by NSX-T Data Center and by NSX Data Center for vSphere. You can use VMware Cloud Director VDC templates with on-premises, Microsoft Azure VMware Solution, Oracle Cloud VMware Solution, or Google Cloud VMware Engine SDDCs.

Workaround: None.
Creating a new VM with encrypted vSAN storage policy fails with an Invalid storage policy for encryption operation error message

When creating a new VM, if you specify the storage policy of the VM as vSAN encrypted and the storage policy for the VM hard disk as both non-encrypted and non-vSAN, the operation fails with an error message.

Invalid storage policy for encryption operation
1. Specify the storage policies for the VM and the VM hard disk as vSAN encrypted.
2. After the VM deploys successfully, update the hard disk storage policy for the VM to non-encrypted and non-vSAN. For information, see Edit Virtual Machine Properties.
You cannot connect to VMware Cloud Director through VMware OVF Tool version 4.4.3 or earlier

When you attempt to connect to VMware Cloud Director through OVF Tool version 4.4.3 or earlier, this results in the following error. Error: No supported vCloud version was found. This happens because of an API behavior change in VMware Cloud Director 10.4 where the API does not return links to all the VDCs in an organization.

Workaround: Upgrade to OVF Tool 4.5.0. See VMware OVF Tool Release Notes.
You are unable to log in to VMware Cloud Director by using VMware PowerCLI 12.7.0 or earlier

When you attempt to log in to VMware Cloud Director by using VMware PowerCLI version 12.7.0 or earlier, this results in the following error. NOT_ACCEPTABLE: The request has invalid accept header: Invalid API version requested. This happens because VMware PowerCLI earlier than 13.0.0 do not support VMware Cloud Director API versions later than 33.0. See VMware Product Interoperability Matrix.

Workaround: Upgrade VMware PowerCLI to version 13.0.0.
VMware Cloud Director displays the old version for an upgraded vCenter Server instance

After you upgrade a vCenter Server instance to a newer version, in the list of vCenter Server instances, VMware Cloud Director still displays the old version for the upgraded instance.

Reset the connection between the vCenter Server instance and VMware Cloud Director. See Reconnect a vCenter Server Instance in VMware Cloud Director Service Provider Admin Portal Guide.
Refreshing the LDAP page in your browser does not take you back to the same page

In the Service Provider Admin Portal, refreshing the LDAP page in your browser takes you to the provider page instead of back to the LDAP page.

Workaround: None.
Mounting an NFS datastore from NetApp storage array fails with an error message during the initial VMware Cloud Director appliance configuration

During the initial VMware Cloud Director appliance configuration, if you configure an NFS datastore from NetApp storage array, the operation fails with an error message.

Backend validation of NFS failed with: is owned by an unknown user

Workaround: Configure the VMware Cloud Director appliance by using the VMware Cloud Director Appliance API.
The synchronization of a subscribed catalog times out while synchronizing large vApp templates

If an external catalog contains large vApp templates, synchronizing the subscribed catalog with the external catalog times out. The issue occurs when the timeout setting is set to its default value of five minutes.

Workaround: Using the manage-config subcommand of the cell management tool, update the timeout configuration setting.

./cell-management-tool manage-config -n transfer.endpoint.socket.timeout -v [timeout-value]
In an IP prefix list, configuring any as the Network value results in an error message

When creating an IP prefix list, if you want to deny or accept any route and you configure the Network value as any, the dialog box displays an error message.

"any" is not a valid CIDR notation. A valid CIDR is a valid IP address followed by a slash and a number between 0 and 32 or 64, depending on the IP version.

Workaround: Leave the Network text box blank.
The vpostgres process in a standby appliance fails to start

The vpostgres process in a standby appliance fails to start and the PostgreSQL log shows an error similar to the following. FATAL: hot standby is not possible because max_worker_processes = 8 is a lower setting than on the master server (its value was 16). This happens because PostgreSQL requires standby nodes to have the same max_worker_processes setting as the primary node. VMware Cloud Director automatically configures the max_worker_processes setting based on the number of vCPUs assigned to each appliance VM. If the standby appliance has fewer vCPUs than the primary appliance, this results in an error.

Workaround: Deploy the primary and standby appliances with the same number of vCPUs.
Upgrading from VMware Cloud Director 10.3.x to VMware Cloud Director 10.4.x results in an Connection to sfcbd lost error message

If you upgrade from VMware Cloud Director 10.3.x to VMware Cloud Director 10.4.x, the upgrade operation reports an error message.

Connection to sfcbd lost. Attempting to reconnect

Workaround: You can ignore the error message and continue with the upgrade.
When using FIPS mode, trying to upload OpenSSL-generated PKCS8 files fails with an error

OpenSSL cannot generate FIPS-complaint private keys. When VMware Cloud Director is in FIPS mode and you try to upload PKCS8 files generated using OpenSSL, the upload fails with a Bad request: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: ... not available: No such algorithm: ...error or salt must be at least 128 bits error.

Workaround: Deactivate the FIPS mode to upload the PKCS8 files.
Creation of Tanzu Kubernetes cluster by using the Kubernetes Container Clusters plug-in fails

When you create a Tanzu Kubernetes cluster by using the Kubernetes Container Clusters plug-in, you must select a Kubernetes version. Some of the versions in the drop-down menu are not compatible with the backing vSphere infrastructure. When you select an incompatible version, the cluster creation fails.

Workaround: Delete the failed cluster record and retry with a compatible Tanzu Kubernetes version. For information on the incompatibilities between Tanzu Kubernetes and vSphere, see Updating the vSphere with Tanzu Environment.
If you have any subscribed catalogs in your organization, when you upgrade VMware Cloud Director, the catalog synchronization fails

After upgrade, if you have subscribed catalogs in your organization, VMware Cloud Director does not trust the published endpoint certificates automatically. Without trusting the certificates, the content library fails to synchronize.

Workaround: Manually trust the certificates for each catalog subscription. When you edit the catalog subscription settings, a trust on first use (TOFU) dialog prompts you to trust the remote catalog certificate.

If you do not have the necessary rights to trust the certificate, contact your organization administrator.
After upgrading VMware Cloud Director and enabling the Tanzu Kubernetes cluster creation, no automatically generated policy is available and you cannot create or publish a policy

When you upgrade VMware Cloud Director to version 10.3.1 and vCenter Server to version 7.0.0d or later, and you create a provider VDC backed by a Supervisor Cluster, VMware Cloud Director displays a Kubernetes icon next to the VDC. However, there is no automatically generated Kubernetes policy in the new provider VDC. When you try to create or publish a Kubernetes policy to an organization VDC, no machine classes are available.

Workaround: Manually trust the corresponding Kubernetes endpoint certificates. See VMware knowledge base article 83583.
Entering a Kubernetes cluster name with non-Latin characters deactivates the Next button in the Create New Cluster wizard

The Kubernetes Container Clusters plug-in supports only Latin characters. If you enter non-Latin characters, the following error appears.

Name must start with a letter and only contain alphanumeric or hyphen (-) characters. (Max 128 characters).

Workaround: None.
NFS downtime can cause VMware Cloud Director appliance cluster functionalities to malfunction

If the NFS is unavailable due to the NFS share being full, becoming read only, and so on, can cause appliance cluster functionalities to malfunction. HTML5 UI is unresponsive while the NFS is down or cannot be reached. Other functionalities that might be affected are the fencing out of a failed primary cell, switchover, promoting a standby cell, and so on. For more information about setting up correctly the NFS shared storage, see Preparing the Transfer Server Storage for the VMware Cloud Director Appliance.
Workaround:
- Fix the NFS state so that it is not read-only.
- Clean up the NFS share if it is full.
Trying to encrypt named disks in vCenter Server version 6.5 or earlier fails with an error

For vCenter Server instances version 6.5 or earlier, if you try to associate new or existing named disks with an encryption enabled policy, the operation fails with a Named disk encryption is not supported in this version of vCenter Server. error.

Workaround: None.
A fast-provisioned virtual machine created on a VMware vSphere Storage APIs Array Integration (VAAI) enabled NFS array, or vSphere Virtual Volumes cannot be consolidated

In-place consolidation of a fast provisioned virtual machine is not supported when a native snapshot is used. Native snapshots are always used by VAAI-enabled datastores, as well as by vSphere Virtual Volumes. When a fast-provisioned virtual machine is deployed to one of these storage containers, that virtual machine cannot be consolidated .

Workaround: Do not enable fast provisioning for an organization VDC that uses VAAI-enabled NFS or vSphere Virtual Volumes. To consolidate a virtual machine with a snapshot on a VAAI or a vSphere Virtual Volumes datastore, relocate the virtual machine to a different storage container.
If you add an IPv6 NIC to a VM and then you add an IPv4 NIC to the same VM, the IPv4 north-south traffic breaks

Using the HTML5 UI, if you add an IPv6 NIC first or configure an IPv6 NIC as the primary NIC in a VM, and then you add an IPv4 NIC to the same VM, the IPv4 north-south communication breaks.

Workaround: First you must add the IPv4 NIC to the VM and then the IPv6 NIC.