VMware Cloud Director 10.3.3.4 | 22 JUN 2023 | Build 21923261 ( installed build 21922251)

Check for additions and updates to these release notes.

What's New

The VMware Cloud Director 10.3.3.4 release provides bug fixes, updates the VMware Cloud Director appliance base OS and the VMware Cloud Director open-source components.

System Requirements and Installation

For information about system requirements and installation instructions, see VMware Cloud Director 10.3 Release Notes.

Documentation

To access the full set of product documentation, go to VMware Cloud Director Documentation.

Resolved Issues

  • VMware Cloud Director ignores the NO_PROXY setting and applies the proxy rules for all outbound communication

    In VMware Cloud Director, when you define a proxy exclusion list for an outbound communication by configuring the NO_PROXY setting, the system ignores the configuration. As a result, the proxy rules are applied to all targets and might result in network connectivity issues.

  • You cannot set the DNS suffix for an external network by using the VMware Cloud Director API

    By using the VMware Cloud Director API, when you attempt to set the DNS information for an NSX-T segment backed external network, the suffix value is not applied.

  • The Move Virtual Machine wizard displays an incorrect list of virtual hard disks for a VM

    If you move VM-A to a different vApp and immediately you attempt to move VM-B to another vApp, the Move Virtual Machine wizard displays a list with virtual hard disks for VM-A along with the list with virtual hard disks for VM-B.

  • Creation of a new VM fails with a Reconfiguration of vmGroups: timed out for cluster error message

    If an organization VDC has only one VM sizing policy this policy is the default placement policy for new VMs, instead of applying this default placement policy during the VM and vApp creation, the system applies the VM sizing policy. This might cause the operation to fail with an error message.

    Reconfiguration of vmGroups: timed out for cluster

  • You cannot complete the wizard for creating a vApp from a vApp template without a configured network

    During the instantiation of a vApp from a vApp template, if you select None on the Configure Networking page, the wizard becomes unresponsive. After configuring the vApp settings, when you reach the Ready to complete page, a spinning wheel appears, and you cannot click the Finish button.

  • An attempt to obtain the media records by running a VMware Cloud Director API request does not return the description for the media

    When you run a VMware Cloud Director API request to obtain a media record, the response does not contain information about the media description.

  • VMware Cloud Director UI and tasks are slow to load and complete

    The Artemis message bus communication is not working and when you trigger operations from the UI, they can take up to 5 minutes to complete or might time out. The performance issues can affect operations such as powering on VMs and vApps, provider VDC creation, vApp deployment, and so on.

  • vCenter Server deployment of an OVA that is exported from VMware Cloud Director fails with an Issues detected with selected template. Details: - 107:17:VALUE_ILLEGAL: Duplicate value ''1'' for element ''Address'' error message

    If you export an OVA from VMware Cloud Director and you attempt to deploy the same OVA to vCenter Server, the operation fails with an error message.

    Issues detected with selected template. Details: - 107:17:VALUE_ILLEGAL: Duplicate value ''1'' for element ''Address''

    This happens because the OVF template contains a duplicate line for the Address element.

  • Organization administrators can't add an NSX edge gateway to a data center group without the Update Gateway right

    Users with the organization administrator role can't add an NSX edge gateway to a data center group unless they are assigned the Update Gateway right. The Update Gateway right provides tenants with additional rights which are not required for their role, such as Enable DFW. This issue is resolved in this release. To add an edge gateway to a data center group, a user now needs to be assigned the Configure VDC Group and View Gateway rights.

  • A VM with IP mode set to DHCP might not be able to connect to an external network

    If a VM with IP mode set to DHCP is connected to a vApp network that uses port forwarding, the VM cannot connect to an external network unless you add to the vApp network a second vApp with a static IP and an explicit DNAT rule that allows access to the external network. This happens because in NSX-backed organization VDCs, enabling IP masquerading for a vApp network does not create a corresponding SNAT rule on the vApp edge in NSX to allow outbound access for a VM without a static IP.

  • When you move a vApp to another organization VDC, the vApp description is lost

    When you move a vApp from one organization VDC to another, the description for the vApp is not preserved.

  • Publishing the rights bundle of the defined entity type to tenants takes longer time to complete or times out when you have configured 1000 or more tenant organizations

    If you configure 1000 or more tenant organizations, attempting to publish the rights bundle of the defined entity type to all tenants takes long time to complete or times out.

    If the rights bundle is already published to 1000 or more tenants, publishing it to a new tenant times out.

  • Runtime defined entity (RDE) modify event entries cause the Audit_trail database table to grow at an uncontrollable rate

    Runtime defined entities (RDE) modify event entries cause the Audit_trail database table to grow uncontrollably. This happens because the database backs up the complete RDE and not only the changes.

  • Using the VMware Cloud Director quick search to find VMs and update their virtual disks triggers misconfiguration of the VMs settings

    When you use the VMware Cloud Director quick search to find a VM and to update its virtual disks, if you navigate to a different VM and update its virtual disk before the completion of the first update, vSphere misconfigures the VMs settings. The misconfiguration can include settings such as VM name, VM description, CPU, memory, networking, and guest OS.

  • Viewing the load balancer pools for an NSX edge gateway in the VMware Cloud Director UI might fail with a duplicate key error

    Viewing the load balancer pools for an NSX edge gateways that uses NSX Advanced Load Balancing fails with Duplicate key error. This happens because of a failed automated creation of load balancing server pools with pool_backing_id set to null that are not used for any virtual services.

  • Guest OS customizations like hostname and network do not work for the AlmaLinux OS

    If you deploy an AlmaLinux template, VMware Cloud Director ignores the hostname and network configurations even when you force the guest customizations.

  • Tasks that require synchronization between cells are either very slow or time out

    vCenter Server updates VMware Cloud Director with changes to its inventory. VMware Cloud Director transmits these changes over the internal message bus. If these messages exceed a certain size, they are treated as large messages for the purposes of message delivery. If many large messages arrive from vCenter Server, the ability of VMware Cloud Director to deliver these messages reduces and potentially stops. Any task that requires communication between the VMware Cloud Director cells is either very slow or times out.

  • When you use the VMware Cloud Director UI to create a new VM with a placement policy, all virtual machines that are part of the VM group defined in the used placement policy might disappear

    When you use the VMware Cloud Director UI to create a new VM that uses a placement policy, all virtual machines that are part of the VM group defined in the used placement policy might dissapear from the VM group.

  • Attempting to create a new application port profile fails with a duplicate error message

    If two organization VDCs that are part of the same organization are backed by different NSX Manager instances, attempting to create an application port profile with the same name on both edge gateways might fail with the error message Application Port Profile already exists in Organization.

  • Migrating a VM that is connected to a vSphere-backed external network between resource pools fails

    If a VM is connected to an external network which is backed by multiple vSphere networks, and you attempt to migrate the VM between resource pools, the operation fails if the source and destination resource pools are backed by different host clusters and if the destination resource pool does not have access to the external network to which the VM was originally connected.

  • Moving a VM to a different provider VDC fails with an Internal Server Error message

    If two provider VDCs are backed by different vCenter Server instances and you configure different names for their storage profiles, moving a VM between the provider VDCs fails with the following error.

    Internal Server Error

  • You cannot convert an organization VDC from allocation pool allocation model to flex allocation model

    When the allocation model has a maximum compute policy with a 0.0 memory and CPU reservation guarantee, attempting to convert an allocation pool model organization VDC to a flex organization VDC fails with the following error.

    com.vmware.vcloud.api.presentation.service.BadRequestException: vDC cpu reservation, memory reservation or vCpu speed cannot exceed values defined in vDC maximum compute policy. Maximum Cpu reservation null, Maximum Memory reservation 0, Maximum vCpuSpeed null.

  • Running a custom workflow with external validation in vRO Workflow Execution UI plug-in fails with an Error performing external validation error message

    When running a custom workflow with external validation through the vRO Workflow Execution UI plug-in, the process fails with an Error performing external validation error message. The issue occurs because vRealize Orchestrator does not perform validation on the inputs in the custom form in VMware Cloud Foundation.

  • Active Directory users cannot use the cell management tool on the VMware Cloud Director appliance

    Active Directory users cannot run the cell management tool on the VMware Cloud Director appliance. The cell-management-tool.log file contains the following exception.

    Unable to connect to the cell: Invalid credentials. Exiting. | java.lang.SecurityException: Invalid credentials at com.vmware.vcloud.common.jmx.VCloudJMXAuthenticator.authenticate

  • The VMware Cloud Director dashboards for flex organization VDCs show incorrect CPU use

    For flex organization VDCs, if the Make Allocation pool Org VDCs elastic option is activated, the flex organization VDC dashboard displays incorrect information about the vCPU use. For example, in a flex organization VDC with default vCPU speed of 1 GHz where the sizing policy defines the vCPU speed to 2GHz, if you create a VM, the dashboard incorrectly shows the vCPU use as 1 GHz. In the flex allocation model, the VM compute resource allocation depends on the VM sizing policies, and the real vCPU speed is 2GHz. When the Make Allocation pool Org VDCs elastic option is deactivated, the metrics appear correctly.

  • VMware Cloud Director does not assign the custom configurations of an OVF for reservation, shares, and limits when using the OVF to deploy a VM in a flex organization VDC

    If you configure custom values for the reservation, shares, and limits in an OVF and you deploy a new VM by using this OVF in a flex organization VDC, VMware Cloud Director does not honor the custom configurations and assigns the default organization VDC sizing policy to the VM.

  • Enabling the guest customization for an Ubuntu VM with IP mode set to DHCP fails

    When creating a vApp from an Ubuntu template with IP mode set to DHCP, enabling the guest customization on the resulting vApp fails.

    The /var/log/vmware-imc/toolsDeployPkg.log file displays an error message.

    Customization command failed with stderr: 'dpkg: warning: version '^A.0.0' has bad syntax: version number does not start with digit'.

  • Login to VMware Cloud Director as a SAML group user fails with a NullPointer Exception error message

    If the list of roles for a SAML group contains an empty entry, SAML login fails with a NullPointer Exception error message.

  • VMware Cloud Director operations, such as powering a VM on and off takes longer time to complete

    VMware Cloud Director operations, such as powering a VM on or off takes longer time to complete. The task displays a Starting virtual machine status and nothing happens.

    The jms-expired-messages.logs log file displays an error.

    RELIABLE:LargeServerMessage & expiration=

  • If you use vRealize Orchestrator 8.x, hidden input parameters in workflows are not populated automatically in the VMware Cloud Director UI

    If you use vRealize Orchestrator 8.x, when you attempt to run a workflow through the VMware Cloud Director UI, hidden input parameters are not populated automatically in the VMware Cloud Director UI.

  • The Customer Experience Improvement Program (CEIP) status is Enabled even after deactivating it during the installation of VMware Cloud Director

    During the installation of VMware Cloud Director, if you deactivate the option to join the CEIP, after the installation completes, the CEIP status is active.

Known Issues

  • New - The VMware Cloud Director appliance database disk resize script might fail if the backing SCSI disk identifier changes

    The database disk resize script runs successfully only if the backing database SCSI disk ID remains the same. If the ID changes for any reason, the script might appear to run successfully but fails. The /opt/vmware/var/log/vcd/db_diskresize.log shows that the script fails with a No such file or directory error.

    Workaround:

    1. Log in directly or by using an SSH client to the primary cell as root.

    2. Run the lsblk --output NAME,FSTYPE,HCTL command.

    3. In the output, find the disk containing the database_vg-vpostgres partition and make note of its ID. The ID is under the HCTL column and has the following sample format 2:0:3:0.

    4. In the db_diskresize.sh script, modify the partition ID with the ID from Step 3. For example, if the ID is 2:0:3:0, in line

      echo 1 > /sys/class/scsi_device/2\:0\:2\:0/device/rescan

      you must change the ID to 2:0:3:0.

      echo 1 > /sys/class/scsi_device/2\:0\:3\:0/device/rescan

    5. Аfter saving the changes, manually re-invoke the resize script or reboot the appliance.

  • Publishing a vRealize Orchestrator workflow to the VMware Cloud Director service library fails with an error message

    When you attempt to publish a vRealize Orchestrator workflow, the operation fails with a 500 Server Error error message.

    This happens because the API returns a large number of links for each individual tenant to which the workflow is published and causes an overflow in the HTTP headers.

    Workaround: To publish the workflow, use CURL or POSTMAN to run an API request with increased HTTP header size limit.

  • Suspending a VM through the VMware Cloud Director UI results in a partially suspended state of the VM

    In the VMware Cloud Director Tenant Portal, when you suspend a VM, VMware Cloud Director does not undeploy the VM, and the VM becomes Partially Suspended instead of Suspended.

    Workaround: None.

  • Migrating VMs between organization VDCs might fail with an insufficient resource error

    If VMware Cloud Director is running with vCenter Server 7.0 Update 3h or earlier, when relocating a VM to a different organization VDC, the VM migration might fail with an insufficient resource error even if the resources are available in the target organization VDC.

    Workaround: Upgrade vCenter Server to version 7.0 Update 3i or later.

  • VMs become non-compliant after converting a reservation pool VDC into a flex organization VDC

    In an organization VDC with a reservation pool allocation model, if some of the VMs have nonzero reservation for CPU and Memory, non-unlimited configuration for CPU and Memory, or both, after converting into a flex organization VDC, these VMs become non-compliant. If you attempt to make the VMs compliant again, the system applies an incorrect policy for the reservation and limit and sets the CPU and Memory reservations to zero and the limits to Unlimited.

    Workaround:

    1. A system administrator must create a VM sizing policy with the correct configuration.

    2. A system administrator must publish the new VM sizing policy to the converted flex organization VDC.

    3. The tenants can use the VMware Cloud Director API or the VMware Cloud Director Tenant Portal to assign the VM sizing policy to the existing virtual machines in the flex organization VDC.

  • You cannot create VMware Cloud Director VDC templates in VMware Cloud Director service environments

    VMware Cloud Director service does not support Virtual Data Center (VDC) templates. You can use VDC templates on environments with provider VDCs with an NSX network provider type or an NSX Data Center for vSphere provider type. You cannot use VDC templates on VMware Cloud Director service environments because the provider VDCs have the VMC network provider type.

    Workaround: None.

  • Role name and description are localized in the VMware Cloud Director UI and can cause duplication of role names

    The problem occurs because the UI translation does not affect the back end and API. You might create roles with the same names as the translated names which results in perceived duplicate roles in the UI and conflicts with the API usage of role names when creating service accounts.

    Workaround: None.

  • When starting the VMware Cloud Director appliance, the message [FAILED] Failed to start Wait for Network to be Configured. See 'systemctl status systemd-networkd-wait-online.service' for details appears.

    The message appears incorrectly and does not indicate an actual problem with the network. You can disregard the message and continue to use the VMware Cloud Director appliance as usual.

    Workaround: None.

  • VMware Cloud Director appliance upgrade fails with an invalid version error when FIPS mode is enabled

    For VMware Cloud Director versions 10.3.x and later, when FIPS mode is enabled, VMware Cloud Director appliance upgrade fails with the following error.

    Failure: Installation failed abnormally (program aborted), the current version may be invalid.

    Workaround:

    1. Before you upgrade the VMware Cloud Director appliance, deactivate FIPS Mode on the cells in the server group and the VMware Cloud Director appliance. See Activate or Deactivate FIPS Mode on the VMware Cloud Director Appliance.

    2. Verify that the /etc/vmware/system_fips file does not exist on any appliance.

    3. Upgrade the VMware Cloud Director appliance.

    4. Enable FIPS mode again.

  • After upgrade, configuring an additional cell in the existing server group fails with a validation error

    When you upgrade to version 10.3.3.1 and later, VMware Cloud Director modifies the file permissions for the NFS mount directory to 770. The directory permission must be 750, and as a result, the deployment of new cells fails with the following error.

    Backend validation of NFS mount failed with: Unexpected ownership and/or permissions on provided NFS share. Expected: vcloud:vcloud with mode: 750. Found: vcloud:vcloud with mode 770

    Workaround: After you upgrade all cells, log in to any cell, and change the file permission for the mounted NFS folder to 750 using the following command.

    # chmod 750 /opt/vmware/vcloud-director/data/transfer

    See Preparing the Transfer Server Storage for the VMware Cloud Director Appliance.

    If you upgrade an appliance after changing the permission, you must change the permission to 750 again.

  • Refreshing the LDAP page in your browser does not take you back to the same page

    In the Service Provider Admin Portal, refreshing the LDAP page in your browser takes you to the provider page instead of back to the LDAP page.

    Workaround: None.

  • An attempt to migrate tenant storage fails with an Internal Server Error error message

    In the HTML5 UI, using the Migrate Tenant Storage option to migrate all the items stored on a datastore to other datastores in an SDRS Clusterfails to migrate the VMs with an errors message.

    Internal Server ErrorCaused by: java.lang.RuntimeException: The operation failed because no suitable resource was found. Out of x candidate hubs:x hubs eliminated because: No valid storage containers found for VirtualMachine "{vm-uuid}". All x available storage containers were filtered out as being invalid.

  • Mounting an NFS datastore from NetApp storage array fails with an error message during the initial VMware Cloud Director appliance configuration

    During the initial VMware Cloud Director appliance configuration, if you configure an NFS datastore from NetApp storage array, the operation fails with an error message.

    Backend validation of NFS failed with: is owned by an unknown user

    Workaround: Configure the VMware Cloud Director appliance by using the VMware Cloud Director Appliance API.

  • The synchronization of a subscribed catalog times out while synchronizing large vApp templates

    If an external catalog contains large vApp templates, synchronizing the subscribed catalog with the external catalog times out. This happens when the timeout setting is set to its default value of five minutes.

    Workaround: Using the manage-config subcommand of the cell management tool, update the timeout configuration setting.

    ./cell-management-tool manage-config -n transfer.endpoint.socket.timeout -v [timeout-value]

  • After upgrade to VMware Cloud Director 10.3.2a, opening the list of external networks results in a warning message

    When trying to open the list of external networks, the VMware Cloud Director UI displays a warning message.

    One or more external networks or T0 Gateways have been disconnected from its IP address data.

    This happens because the external network gets disconnected from the Classless Inter-Domain Routing (CIDR) configuration before the upgrade to VMware Cloud Director 10.3.2a.

    Workaround: Contact VMware Global Support Services (GSS) for assistance with the workaround for this issue.

  • In an IP prefix list, configuring any as the Network value results in an error message

    When creating an IP prefix list, if want to deny or accept any route and you configure the Network value as any, the dialog box displays an error message.

    "any" is not a valid CIDR notation. A valid CIDR is a valid IP address followed by a slash and a number between 0 and 32 or 64, depending on the IP version.

    Workaround: Leave the Network text box blank.

  • The vpostgres process in a standby appliance fails to start

    The vpostgres process in a standby appliance fails to start and the PostgreSQL log shows an error similar to the following. FATAL: hot standby is not possible because max_worker_processes = 8 is a lower setting than on the master server (its value was 16). This happens because PostgreSQL requires standby nodes to have the same max_worker_processes setting as the primary node. VMware Cloud Director automatically configures the max_worker_processes setting based on the number of vCPUs assigned to each appliance VM. If the standby appliance has fewer vCPUs than the primary appliance, this results in an error.

    Workaround: Deploy the primary and standby appliances with the same number of vCPUs.

  • VMware Cloud Director API calls to retrieve vCenter Server information return a URL instead of a UUID

    The issue occurs with vCenter Server instances that failed the initial registration with VMware Cloud Director version 10.2.1 and earlier. For those vCenter Server instances, when you make API calls to retrieve the vCenter Server information, the VMware Cloud Director API incorrectly returns a URL instead of the expected UUID.

    Workaround: Reconnect to the vCenter Server instance to VMware Cloud Director.

  • Upgrading from VMware Cloud Director 10.2.x to VMware Cloud Director 10.3.x results in an Connection to sfcbd lost error message

    If you upgrade from VMware Cloud Director 10.2.x to VMware Cloud Director 10.3, the upgrade operation reports an error message.

    Connection to sfcbd lost. Attempting to reconnect

    Workaround: You can ignore the error message and continue with the upgrade.

  • When using FIPS mode, trying to upload OpenSSL-generated PKCS8 files fails with an error

    OpenSSL cannot generate FIPS-complaint private keys. When VMware Cloud Director is in FIPS mode and you try to upload PKCS8 files generated using OpenSSL, the upload fails with a Bad request: org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: ... not available: No such algorithm: ...error or salt must be at least 128 bits error.

    Workaround: Deactivate the FIPS mode to upload the PKCS8 files.

  • Creation of Tanzu Kubernetes cluster by using the Kubernetes Container Clusters plug-in fails

    When you create a Tanzu Kubernetes cluster by using the Kubernetes Container Clusters plug-in, you must select a Kubernetes version. Some of the versions in the drop-down menu are not compatible with the backing vSphere infrastructure. When you select an incompatible version, the cluster creation fails.

    Workaround: Delete the failed cluster record and retry with a compatible Tanzu Kubernetes version. For information on the incompatibilities between Tanzu Kubernetes and vSphere, see Updating the vSphere with Tanzu Environment.

  • If you have any subscribed catalogs in your organization, when you upgrade VMware Cloud Director, the catalog synchronization fails

    After upgrade, if you have subscribed catalogs in your organization, VMware Cloud Director does not trust the published endpoint certificates automatically. Without trusting the certificates, the content library fails to synchronize.

    Workaround: Manually trust the certificates for each catalog subscription. When you edit the catalog subscription settings, a trust on first use (TOFU) dialog prompts you to trust the remote catalog certificate.

    If you do not have the necessary rights to trust the certificate, contact your organization administrator.

  • After upgrading VMware Cloud Director and enabling the Tanzu Kubernetes cluster creation, no automatically generated policy is available and you cannot create or publish a policy

    When you upgrade VMware Cloud Director to version 10.3.1 and vCenter Server to version 7.0.0d or later, and you create a provider VDC backed by a Supervisor Cluster, VMware Cloud Director displays a Kubernetes icon next to the VDC. However, there is no automatically generated Kubernetes policy in the new provider VDC. When you try to create or publish a Kubernetes policy to an organization VDC, no machine classes are available.

    Workaround: Manually trust the corresponding Kubernetes endpoint certificates. See VMware knowledge base article 83583.

  • Entering a Kubernetes cluster name with non-Latin characters deactivates the Next button in the Create New Cluster wizard

    The Kubernetes Container Clusters plug-in supports only Latin characters. If you enter non-Latin characters, the following error appears.

    Name must start with a letter and only contain alphanumeric or hyphen (-) characters. (Max 128 characters).

    Workaround: None.

  • NFS downtime can cause VMware Cloud Director appliance cluster functionalities to malfunction

    If the NFS is unavailable due to the NFS share being full, becoming read only, and so on, can cause appliance cluster functionalities to malfunction. HTML5 UI is unresponsive while the NFS is down or cannot be reached. Other functionalities that might be affected are the fencing out of a failed primary cell, switchover, promoting a standby cell, and so on. For more information about setting up correctly the NFS shared storage, see Preparing the Transfer Server Storage for the VMware Cloud Director Appliance.

    Workaround: 

    • Fix the NFS state so that it is not read-only.

    • Clean up the NFS share if it is full.

  • Trying to encrypt named disks in vCenter Server version 6.5 or earlier fails with an error

    For vCenter Server instances version 6.5 or earlier, if you try to associate new or existing named disks with an encryption enabled policy, the operation fails with a Named disk encryption is not supported in this version of vCenter Server. error.

    Workaround: None.

  • A fast-provisioned virtual machine created on a VMware vSphere Storage APIs Array Integration (VAAI) enabled NFS array, or vSphere Virtual Volumes (VVols) cannot be consolidated

    In-place consolidation of a fast provisioned virtual machine is not supported when a native snapshot is used. Native snapshots are always used by VAAI-enabled datastores, as well as by VVols. When a fast-provisioned virtual machine is deployed to one of these storage containers, that virtual machine cannot be consolidated .

    Workaround: Do not enable fast provisioning for an organization VDC that uses VAAI-enabled NFS or VVols. To consolidate a virtual machine with a snapshot on a VAAI or a VVol datastore, relocate the virtual machine to a different storage container.

  • If you add an IPv6 NIC to a VM and then you add an IPv4 NIC to the same VM, the IPv4 north-south traffic breaks

    Using the HTML5 UI, if you add an IPv6 NIC first or configure an IPv6 NIC as the primary NIC in a VM, and then you add an IPv4 NIC to the same VM, the IPv4 north-south communication breaks.

    Workaround: First you must add the IPv4 NIC to the VM and then the IPv6 NIC.

check-circle-line exclamation-circle-line close-line
Scroll to top icon