VMware Telco Cloud Infrastructure 3.0 | 31 AUG 2023

Check for additions and updates to these release notes.

What's New

VMware Telco Cloud Infrastructure – Cloud Director Edition Release 3.0 is a significant milestone in the evolving roadmap of the Telco Cloud Infrastructure platform. With the introduction of vSphere 8.0 U1 and NSX 4.1.1, this release optimizes the use of core CPU resources by utilizing the DPU-based acceleration, enhances the networking and security performance and scalability of vSphere and NSX, and provides extended support for Virtual Infrastructure Management (VIM) component such as VMware Cloud Director. In addition to the existing IPv6 support for the data plane, IPv6 is now supported in the NSX control and management planes. IPv6 support for Tunnel End Point (TEP) is supported for transport nodes. This release also delivers various Enhanced Data Path (EDP) enhancements such as MACVLAN support and Virtual Hyperthreading improvements.

SmartNICs in NSX help CSPs to run EDP functions, freeing up valuable core CPU resources to support other platforms and network functions. With vSphere Data Processing Units (DPUs), CSPs can offload NSX services (networking and security) to a DPU directly. In addition, the VMware Cloud Director Long Term Support (LTS) release allows CSPs to stay longer on a VMware Cloud Director release and to schedule the upgrades in a more flexible manner.

Carrier-Grade Resilient Networking and Security

VMware NSX 4.1.1 (formerly VMware NSX-T Data Center) introduces the following new features. This release also inherits features and enhancements from NSX 4.1.0.2 and 4.0.1.1.

  • IPv6 Tunnel End Point (TEP) support for Transport Nodes

  • Filter VLAN support on VLAN Transport Zone

  • Enhanced Data Path (EDP) enhancements:

    • Support of MAC and VLAN filtering, which allows a physical NIC driver to program (MAC, VLAN) pair to physical NIC Rx queue.  

    • Optimization of flow cache for Geneve overlay traffic reducing the impact of large numbers of flows on forwarding performance.

  • Support for NVIDIA BlueField-2 (100Gbps)

  • Intel 810 support for Bare Metal Edge

  • LTA support on ESXi ENS Fastpath

  • IPv6 support in the Control and Management Planes

  • Bidirectional Forwarding Detection (BFD) support for IPv6 Border Gateway Protocol (BGP) neighbors and IPv6 static routes

  • Ethernet VPN (EVPN) Route Server Mode Enhancements

  • Support for upgrading the Edge node Operating System to Ubuntu 20.04

  • Support for DPU-based acceleration through NVIDIA and AMD Pensando SmartNICs

  • Virtual Routing and Forwarding (VRF) enhancements

  • NVIDIA ConnectX-6 support for Bare Metal Edge

For more information, see the VMware NSX 4.1.1 Release Notes, VMware NSX 4.1.0.2 Release Notes, and VMware NSX 4.0.1.1 Release Notes.

Multi-Tenancy Management, UI, and Networking Enhancements

VMware Cloud Director 10.4.2 introduces the following new features. This release also inherits features and enhancements from VMware Cloud Director 10.4.1 and 10.4:

  • VMware Cloud Director can be configured as an Identity Provider (IDP) Proxy, which enables relying parties to use Cloud Director for tenant-aware authentication of users known to Cloud Director. See Using VMware Cloud Director as an Identity Provider Proxy Server. For more information about the OpenID Connect standard, see OpenID Connect Core 1.0.

  • Support for VMs with Trusted Platform Module (TPM) devices that provide additional security to the guest OS. See the service provider information in Understanding Trusted Platform Module Devices or the tenant information in Working with Virtual Machines.

  • User Management Enhancements

    • Support for UI-based bulk user edit option

    • Deprecation of empty or null email property for users. This change is a step towards assisting with user notification.

    • Improved Provider and Tenant login experience

    • Improved Tenant login experience When SAML or OIDC are configured

  • Support for vSphere 8.0 u1

  • Support for NSX 4.1.1

  • Support for VMware NSX Advanced Load Balancer (formerly Avi Networks) capabilities including:

    • Feature set configuration at the service engine group level: Allows users to restrict a tenant's feature set to Standard (Basic) at the service engine group level.

    • Standard feature set improvements for tenants: Allows users to configure a single virtual service to use more than one port protocol and other virtual services including IPv6 for virtual IP addresses and for load balancer server pools.

  • Support for Photon OS 3: Provides improved security and upgraded OS packages.

  • Support for Solution Add-On Management: Enables VMware Cloud Director to create, deploy, and manage extensions using a new extensibility framework.

  • Support for IP Spaces, an enhanced and more structured P address management service.

  • New Branding and Theming Experience (UI and API): Enable Providers to create, manage, and assign themes for their tenants.

  • Security Technical Implementation Guides (STIG) released as part of VMware Cloud Director 10.4: Provides technical, standards-based hardening guidance for United States Department of Defense (DoD). For more information, see VMware Cloud Director STIG Readiness Guide.

  • Other key features:

    • Enhanced trust management integration with vSphere with SSL connectivity to all vSphere infrastructure components

    • VMware Cloud Director appliance certificate backup and restore

    • Catalogue enhancements including a detailed view of the currently running catalog synchronization task step and the progress percentage of that step.

    • Persistence of the sync from the previous transfer session, in case a failure of catalog synchronization during the file transfer phase due to VMware Cloud Director crash or restart

    • Security STIG Compliance and Hardening

For more information, see the VMware Cloud Director 10.4.2 Release Notes, VMware Cloud Director 10.4.1 Release Notes, and VMware Cloud Director 10.4 Release Notes.

Workload Management, Storage, and Reliability Enhancements

VMware ESXi 8.0 Update 1a addresses TPM-related security enhancements that are necessary for the upcoming CPU support in newer Lenovo and HPE platforms. This patch release introduces the following new features and also inherits features and enhancements from ESXi 8.0b:

  • vSphere Configuration Profiles:

    • Helps manage ESXi cluster configurations by specifying a host configuration at the cluster level

    • Helps automate the scanning of ESXi hosts for compliance with the specified configuration and remediate any host that is not compliant

  • Support for heterogenous virtual Graphics Processing Unit (vGPU) profiles on the same GPU hardware

  • HTTP/JSON-based wire protocol as an alternative to SOAP/XML

  • Distributed Services Engine (DSE) Support – DPU Offload:

    • Using a vSphere Distributed Switch version 8.0 and NSX, network services can be offloaded to the DPU, leading to improved network performance.

  • Life cycle management enhancements:

    • Automatically remediate the ESXi installation on a DPU in lock-step with the host ESXi version

    • Support for staging update/upgrade payloads to speed up remediation

    • Remediate multiple hosts in parallel

    • Manage standalone hosts using vSphere Lifecycle Manager API.

  • Configuration management at scale

  • Resource management enhancement

    • vSphere Green Metrics introduces new power consumption metrics for hosts and VMs.

  • vSAN enhancements

    • Performance enhancements

    • Resource and space efficiency

    • Availability and serviceability enhancements

  • vSphere scalability enhancements

    • Increased vGPU resources

    • 96 hosts per vSphere Lifecycle Manager cluster

    • 10K VMs per cluster

  • vSphere Distributed Services Engine supports the following:

    • NVIDIA BlueField-2 DPUs to server designs from Lenovo (Lenovo ThinkSystem SR650 V2)

    • 100G NVIDIA BlueField-2 DPUs to server designs from Dell

    • UPTv2 for NVIDIA BlueField-2 DPUs

    • AMD Genoa CPU based server designs from Dell

  • Supports up to 8 GPUs with NVSwitch connections between them on a single ESXi host

For more information about all these features, see the VMware ESXi 8.0U1 Release Notes and VMware ESXi 8.0b Release Notes.

vCenter Server 8.0U1 includes new features and improvements. For more information, see the VMware vCenter Server 8.0U1 Release Notes.

VMware vSAN 8.0U1 introduces the following features and enhancements:

  • Disaggregated Storage:

    • Disaggregation with vSAN Express Storage Architecture (ESA)

    • Disaggregation for vSAN stretched clusters (vSAN OSA)

    • Disaggregation across clusters using multiple vCenter Servers (vSAN OSA)

  • Optimized performance, durability, and flexibility

    • Improved performance with new Adaptive Write Path

    • Optimized I/O processing for single VMDK/objects (vSAN ESA)

    • Enhanced durability in maintenance mode scenarios

    • Increased administrative storage capacity on vSAN datastores using customizable namespace objects

    • Witness appliance certification

  • Other key features

    • Simplified management and cloud native storage

For more information about these features and enhancements, see the VMware vSAN 8.0U1 Release Notes.

Components

Mandatory Add-On Components

Note: Additional license is required.

Optional Add-On Components

Note: Additional license is required.

Validated Patches

End of General Support Guidance

VMware Product Lifecycle Matrix outlines the End of General Support (EoGS) dates for VMware products. Lifecycle planning is required to keep each component of the VMware Telco Cloud Infrastructure solution in a supported state. Plan the component updates and upgrades according to the EoGS dates. To ensure that the component versions are supported, you may need to update the Telco Cloud Infrastructure solution to its latest maintenance release.

VMware pre-approval is required to use a product past its EoGS date. To discuss the extended support of products, contact your VMware representative.

Resolved Issues

Note: For information about the entire list of fixes in each Telco Cloud Infrastructure component, see the corresponding product release notes.

  • Unable to Retrieve Server Pool Details of the NSX Native Load Balancer After Adding NSX Advanced Load Balancer

    If you add NSX Advanced Load Balancer as an enforcement point, the status details of the native load balancer’s server pool are not retrieved. The operational status of the server pool in the NSX native load balancer appears blank, and the pool member details also do not appear in the NSX UI.

    This issue is fixed in NSX-T Data Center Release 3.2.3 and later.

  • Operation Failed Error Occurs While Changing the State of vROps Node to Offline or Online

    When you change the state of a vRealize Operations node to offline or online from the vROps 8.6.4 UI, the following error occurs:

    Operation failed. If the error persists, contact VMware support.

    This issue is fixed in VMware vRealize Operations Release 8.10 and later.

  • DHCPV6 Relay Fails to Provide IPv6 addresses from the DHCPv6 Server

    In NSX, the DHCPV6 relay fails to provide IPv6 addresses from the DHCPv6 server.

    This issue is fixed in NSX Release 4.0.0.1 and later.

Known Issues

Note: For information about the entire list of known issues in each Telco Cloud Infrastructure component, see the corresponding product release notes.

  • Statistics for IPv6 Route-Based IPsec Sessions Stop Suddenly in NSX

    In NSX 4.1.1, the traffic statistics for IPv6 route-based IPsec sessions stop after some time, even though the session status is up.

    Workaround: None

  • vSAN Data Query Error Shown in Real-time Performance Metrics of vCenter After ESXi Upgrade

    After upgrading ESXi from 7.0 U3c to 8.0 U1a, the real-time performance metrics at the host level show the following vSAN error in the vCenter UI:

    Unable to query data. Try again later.

    Workaround:

    1. Restart the vsanhealth service in the vCenter appliance:

      vmon-cli -r vsan-health

    2. Restart vsanmgmtd service in each ESXi host in the vSAN cluster:

      /etc/init.d/vsanmgmtd restart

  • NSX Advanced Load Balancer Shows Cluster Unavailable Error After NSX Upgrade

    After upgrading NSX from version 3.2.2 to 4.1.1, the NSX Advanced Load Balancer cluster status is reported as unavailable, displaying the following error message:

    Some error has occurred.

    Solution: You can ignore this error, as it has no functional impact.

  • Removal of External Network or Modification of its IP address From Edge Gateway in VMware Cloud Director Causes Issues in NSX Tier-1 Gateway

    When a static route is configured with the external network of the Edge Gateway in VMware Cloud Director 10.4.2, the removal of the external network from the Edge Gateway results in the removal of the service interface from NSX Tier-1 Gateway, causing a network outage. Additionally, modifying the IP address of the external network results in a failed state of the service interface in the NSX Tier-1 Gateway.

    Workaround: Before removing an external network or modifying its IP address from the Edge Gateway, ensure that no active static routes are dependent on the external network.

  • Unable to Select the Second vNic of the VM under Path & Topology in Aria Operations for Networks When vNIC is Attached to NSX Overlay Segment Without Gateway

    If a VM is configured with two vNics, with the first vNic attached to an NSX vlan segment with a gateway and the second vNic attached to an NSX overlay segment without a gateway (isolated segment), the VM's second vNic cannot be selected under "Path & topology" in Aria Operations for Networks. Hence, the VM path cannot be determined in Aria Operations for Networks.

    Workaround: None

  • vRNI Group Prevents Uninstallation of NSX from ESXi Host Cluster

    When you select the "Enable latency metric collection" option in the Aria Operations for Networks UI, a vRNI group for transport nodes is created in the NSX UI (manager mode). This vRNI group prevents the uninstallation of NSX from the ESXi host cluster.

    Workaround:  Before uninstalling NSX from the ESXi host cluster, remove the ESXi hosts manually from the vRNI group in the NSX UI (manager mode).

  • Custom Network Segment Profiles Not Listed in the Tenant Portal Network UI

    In the tenant portal’s network UI, the "Use a Custom set of Segment Profiles" drop-down list does not display the custom network segment profiles for selection.

    Workaround: Use the NSX UI to select the custom network segment profile.

  • Tier 1 Edge Gateway Moves to Error State in VMware Cloud Director if its HA Mode is Modified Directly from NSX

    If you use NSX to directly modify the HA mode of the Tier-1 edge gateway that is created in VMware Cloud Director, the edge gateway moves to error state in Cloud Director.

    Workaround:  Always use Cloud Director to modify the Tier-1 gateways that are created in Cloud Director. Do not use NSX.

  • Tier-1 Gateway Appears in Critical State When Deployed Using the Same Edge Cluster Connected to Tier-0 Gateway

    When you deploy a Tier-1 gateway using the same edge cluster that is connected to the Active-Active stateful Tier-0 gateway, the Tier-1 gateway appears in critical state in VMware Cloud Director. In addition, if you delete the Tier-1 gateway in Cloud Director, it is not deleted in NSX.

    Workaround:

    1. Delete the Tier-1 gateway manually in NSX.

    2. Re-deploy the Tier-1 gateway using a different edge cluster that is not connected to the Tier-0 gateway.

  • Linked Tier-0 Gateway Interface Statistics Shows the Edge Nodes That Are Not Part of the Network Topology

    In the Network Topology view of the NSX UI, the Linked Tier-0 Gateway Interface Statistics under the Tier-1 gateway shows all the edge nodes created under the same transport zones, instead of showing only the respective edge nodes created for the topology.

    Workaround: None

  • IPv4 North-South Communication Breaks When Adding IPv6 NIC Followed by IPv4

    If you add IPv6 NIC first and IPv4 NIC later to a virtual machine using the VMware Cloud Director UI, the IPv4 north-south communication breaks.

    Workaround: Always add IPv4 NIC first followed by IPv6 NIC to the VM.

  • ESXi Cluster Upgrade Fails When One or More VMs are Connected to a Standard Switch

    When you upgrade an ESXi cluster with one or more VMs connected to a Standard Switch, Distributed Resource Scheduler cannot move those VMs to active ESXi hosts. Hence, the ESXi hosts containing those VMs cannot be moved into maintenance mode resulting in upgrade failure.

    Workaround: Before you proceed with the ESXi cluster upgrade, do one of the following tasks:

    • Manually move the VMs from the Standard Switch to a Distributed Switch.

    • Power OFF the VMs connected to the Standard Switch.

  • New External Network Entry Created in VMware Cloud Director Edge Gateways Although the Entry Creation Task Fails

    When you create a new external network entry in the VMware Cloud Director Edge Gateway, the entry creation task fails with an error but the entry gets created.

    Solution: Ignore the error.

Release Notes Change Log

Date

Change

Feb 13, 2024

VMware NSX Advanced Load Balancer 22.1.5 is added to the Validated Patches section.

Dec 13, 2023

The following known issues are added to the Known Issues section:

  • Statistics for IPv6 Route-Based IPsec Sessions Stops Suddenly in NSX

  • vSAN Data Query Error Shown in Real-time Performance Metrics of vCenter After ESXi Upgrade

  • NSX Advanced Load Balancer Shows Cluster Unavailable Error After NSX Upgrade

  • Removal of External Network or Modification of its IP address From Edge Gateway in VMware Cloud Director Causes Issues in NSX Tier-1 Gateway

Oct 25, 2023

VMware vCenter Server 8.0 Update 1d is added to the Validated Patches section.

Support Resources

For additional support resources, see the VMware Telco Cloud Infrastructure documentation page.

check-circle-line exclamation-circle-line close-line
Scroll to top icon