This site will be decommissioned on January 30th 2025. After that date content will be available at techdocs.broadcom.com.

 

VMware Telco Cloud Platform 4.0.1 | 30 MAY 2024

Check for additions and updates to these release notes.

What's New

VMware Telco Cloud Platform Release 4.0.1 includes new features and enhancements across the carrier-grade network function automation and orchestration, and Kubernetes infrastructure areas.

This release streamlines the infrastructure and CNF management through GitOps support and the ability to deploy and manage Harbor as a CNF using Telco Cloud Automation (TCA). This release also enables the configuration of vSphere-CSI Container File Volume Security Configuration through the TCA UI.

Carrier-Grade Kubernetes Infrastructure

VMware Tanzu Standard for Telco supports the following Kubernetes versions as part of VMware Tanzu Kubernetes Grid 2.5.1:

  • 1.28.7

  • 1.27.11

  • 1.26.14

Important:

  • Kubernetes 1.28: The standard one-year support policy applies to Kubernetes 1.28.

  • Kubernetes 1.27: In addition to the standard one-year support policy, Kubernetes 1.27 receives an additional 1-year of support. The 2-year support clock for Kubernetes 1.27 begins on the release date of Telco Cloud Platform 4.0.1. The support clock does not reset with every patch release.

  • Kubernetes 1.26: When Tanzu Kubernetes Grid 2.3 reaches End of General Support, VMware will no longer support Kubernetes 1.26 with Tanzu Kubernetes Grid.

For more information about lifecycle support for specific Kubernetes versions in a Tanzu Kubernetes Grid release, see the VMware Tanzu Kubernetes Grid 2.5.x Release Notes.

For further information about support and extensions, contact Broadcom Support.

For more information about bug fixes and other updates, see the VMware Tanzu Kubernetes Grid 2.5.x Release Notes.

Carrier-Grade VNF and CNF Automation and Orchestration

VMware Telco Cloud Automation 3.1.1 supports various features and enhancements:

  • GitOps is introduced as an alternate CNF lifecycle management solution to ETSI. With this feature, you can manage GitOps-based CNFs and also upgrade multiple CNFs in parallel.

    Note: This feature was previously introduced as a Tech Preview in Telco Cloud Platform 4.0.

  • CNF LCM Helm version is updated to 3.13.0.

  • CaaS ENS RSS Spec readiness feature enables NetQ Receive Side Scaling (RSS) vNICs requests to be offloaded to a physical NIC through a feature flag using an API call. This feature improves the packet performance of the receive-side data. The RSS configuration is disabled by default.

    Note: This feature was previously introduced as a Tech Preview in Telco Cloud Platform 4.0.

  • vSphere-CSI Container File Volume Security Configuration can now be performed through the TCA UI.

  • VMware Aria Operations Management Pack for VMware Telco Cloud Platform is supported. Aria Operations Management Pack monitors the health status of TCA instances and their entities and generates alerts.

  • TCA Audit Events can now be forwarded automatically to VMware Aria Operations for Log Insights.

  • Harbor for CNFs: Introduces support for deploying Harbor (version 2.9.3) as a CNF and managing the Harbor CNF through TCA.

    Important:

    Harbor for CNFs is available as part of VMware Telco Cloud Platform 4.0.1 Advanced Edition.

For more information about these features and enhancements, see the VMware Telco Cloud Automation 3.1.1 Release Notes.

Components

Telco Cloud Platform Essentials

To download these components, see the Telco Cloud Platform 4.0.1 Essentials Product Downloads page.

Optional Add-On Components

Note:

An additional license is required.

Telco Cloud Platform Advanced

To download these components, see the Telco Cloud Platform 4.0.1 Advanced Product Downloads page.

Optional Add-On Components

Note:

Additional licenses are required.

Validated Patches

Deprecated Features

DPDK kernel modules (rte-kni and igb_uio): VMware has deprecated the support for DPDK kernel modules (rte-kni and igb_uio) and they will be removed from future releases.

Notes:

  • Use vfio-pci as an alternative to igb_uio. vfio-pci is available as part of Photon OS.

  • Use virtio_user as an alternative to rte-kni.

Support for Backward Compatibility of CaaS Layer with IaaS Layer

VMware Telco Cloud Platform supports backward compatibility of its CaaS layer components (Telco Cloud Automation and Tanzu Kubernetes Grid) with the IaaS Layer components (vSphere and NSX) in earlier versions of Telco Cloud Platform. With this feature, you can upgrade the CaaS layer components to their latest versions while using earlier versions of the IaaS layer components.

For more information, see Software Version Support and Interoperability in the Telco Cloud Automation Deployment Guide and Supported Features on Different VIM Types in the Telco Cloud Automation User Guide.

Known Issues

Note: For information about the entire list of known issues in each Telco Cloud Platform component, see the corresponding product release notes.

  • Bare Metal Edge Devices Experience Packet Drops Causing Network Performance Degradation

    Bare Metal Edge devices using NSX 4.1.2.1 experience packet drops, which affect the network traffic flow and result in degraded network performance.

    Workaround: Upgrade the NSX Edge version from 4.1.2.1 to 4.1.2.4.

  • VMware Aria Operations 8.16 Fails to Integrate with Tanzu Kubernetes Classy Standard Clusters 1.27 and 1.28 Running TLS 1.3

    VMware Aria Operations 8.16 fails to integrate with Tanzu Kubernetes Classy Standard Clusters (1.27 and 1.28) that are running Transport Layer Security (TLS) version 1.3. Hence, the Classy Standard Clusters 1.27 and 1.28 cannot be monitored.

    Note: TLS 1.3 is not supported in Aria Operations 8.16.

    Workaround: Follow one of these workarounds:

    • Upgrade to Aria Operations 8.18.

    • If you are using Aria Operations 8.16, change TLS 1.3 to TLS 1.2 on the Classy Standard clusters.

      1. Navigate to CaaS Infrastructure > Cluster Instances in the TCA UI.

      2. Select the cluster you want to modify, and add the following topology variable in the Edit Cluster Configuration > Configuration > Cluster Info section:

        • Key: Security

        • Value: minimumTLSProtocol: tls_1.2

  • Include Node Pool Toggle Button Gets Reset When Selecting Templates Individually

    The Include Node Pool toggle button in the Cluster Upgrade wizard gets reset when selecting templates individually.

    Workaround: Select all the templates first and then enable the Include Node Pool toggle button.

  • Edit DualStack Workload Cluster Shows Incorrect IP Version that Blocks Workload Cluster Upgrade

    The workload cluster upgrade from 1.28.4 to 1.28.7 is blocked due to an incorrect IP version that appears when using Edit DualStack Workload Cluster.

    Workaround: Upgrade workload clusters by selecting 'Upgrade Cluster' in the action drop-down list.

  • TCA Migration Validation Fails if vApp Options are Disabled in TCA VM

    If the vApp options are disabled or if the OVF properties values are lost for a TCA VM, TCA migration fails during deploy validation.

    Workaround: Ensure that the vApp Properties of TCA VM are enabled, and the Product Name under vApp Properties is VMware Telco Cloud Automation.

  • Existing Airgap Server and Harbor Appear as Disconnected in TCA Manager After Migrating TCA From 2.3 to 3.1.1

    After migrating Telco Cloud Automation (TCA) from 2.3 to 3.1.1, the existing Airgap server and Harbor appear as Disconnected under the Connected Endpoints tab in the TCA Manager.

    Workaround:

    Airgap server:

    1. Navigate to Administration > Certificate in the TCA Control Plane Appliance Management 9443 portal.

    2. Select the Trusted Certificate Option.

    3. If the ca certificate of the Airgap Server is missing in the trusted ca certificates, save the ca certificate of the Airgap Server using the file or content option.

    Harbor:

    After migrating TCA from 2.3 to 3.1.1, manually edit the existing Harbor and add the ca certificate.

  • Retry Can be Performed Only After a Four-Hour Timeout if Management Cluster Upgrade is Stuck Due to Missing TKG Template on vCenter

    If the Tanzu Kubernetes management cluster 1.24 upgrade is stuck due to the missing TKG template on vCenter, users need to wait approximately four hours for the timeout before retrying the cluster upgrade.

    Workaround: None

  • Retry Not Working if Management Cluster Upgrade Fails Due to a Missing TKG Template on vCenter

    If the Tanzu Kubernetes management cluster 1.24 upgrade fails due to a missing Tanzu Kubernetes Grid (TKG) template on vCenter, the retry operation does not work.

    Workaround:

    1. Log in to the Tanzu Kubernetes management cluster.

    2. Restart the Kubernetes pod.

    3. Retry the management cluster upgrade from the TCA Manager.

  • Incorrect Template Used for Control Plane and Worker Node Deployments When Upgrading Management Cluster from Kubernetes version 1.24 to 1.25

    If vCenter contains multiple templates of the same version during the management cluster upgrade from Kubernetes version 1.24 to 1.25 (Tanzu Kubernetes Grid 2.2), an incorrect template is used for control plane and worker node deployments.

    In this issue, node cloning might fail in vCenter if the auto-selected node is not accessible by the selected vSphere cluster. vCenter reports a Clone VM Task Failure with the following message:

    Cannot connect to host, and cluster creation fails.

    Workaround: Delete redundant Kubernetes cluster templates from vCenter.

  • vCenter Upgrade to 8.0 U2 Stuck for an Extended Time in Airgapped Environment

    vCenter upgrade to 8.0 U2 is stuck for an extended time in an airgapped environment.

    Workaround: Do one of the following:

    • If the upgrade is in progress, wait until the upgrade completes.

    • Retry the upgrade through VAMI or the full patch ISO.

  • Techsupport Bundle Generation for CaaS Clusters Might Fail When Run in Parallel

    The techsupport bundle generation for CaaS clusters might fail if it is run in parallel.

    In this issue, the Support bundle service allows a user to trigger multiple support bundle requests simultaneously, while KBS allows only one CaaS cluster log collection request at a time.

    Workaround: Wait until the previous techsupport bundle generation completes and then retry the subsequent bundle generation.

    Note: The Support bundle service displays a tooltip that a subsequent request to collect CaaS cluster logs will fail if one is already running.

  • Migration to TCA 3.0 or 3.1 Not Supported if Compute Cluster Domains Exist in TCA 2.3.x Infrastructure Automation

    If compute cluster domains exist in Telco Cloud Automation 2.3.x Infrastructure Automation, migration to TCA 3.0 or 3.1 is not supported.

    Workaround:

    1. Revert the partially migrated appliances using the tcamigctl tool.

    2. In the TCA Manager Web UI (443), delete compute cluster domains under Infrastructure Automation > Domains > Compute Cluster.

    3. Retry the migration.

    Important:

    Compute Clusters functionality in the Infrastructure Automation is deprecated and migration is not supported for Compute Clusters. Deleting Compute Clusters is a prerequisite for migration.

  • Multitenancy Not supported for Certificate Observability Service

    Unless a non-default Tenant shares the Endpoint with the default Tenant or the default Tenant inherits the Endpoint as a part of parent-child relationship, the Endpoint is not shown in the view for a Default Tenant login.

    For the Default Tenant login, though the Endpoint owned by other Tenants (non-default) is not listed in the portal, the Endpoint may get listed in the Connected Endpoints listing.

    Workaround: NA

  • CNF Upgrade Retry Skips Nodecustomization if Previous Nodecustomization Failed During CNF Upgrade

    The CNF upgrade retry skips nodecustomization if the previous nodecustomization failed during the CNF upgrade.

    Workaround:

    1. Roll back the failed CNF upgrade.

    2. Perform a fresh upgrade instead of retry.

  • Airgap rsync Operation Might Fail Occasionally if it is Run Multiple Times

    The airgap rsync operation might fail occasionally if it is run multiple times.

    Workaround: Run the following commands on the airgap server as a root user:

    1. Remove the existing content from the following location:

      rm -f /etc/yum.repos.d/*

    2. Copy the content from the backup location:

      cp /usr/local/airgap/backup_repo/* /etc/yum.repos.d/

    3. Run the rsync operation using the copied content:

      agctl rsync

  • Management Cluster Upgrade Might Fail Due to Default Timeout in TCA API

    The management cluster upgrade might fail due to the default timeout (about 3.5 hours) in the TCA API. If the upgrade task is running in the backend, inconsistent cluster status appears in the TCA UI and backend.

    Workaround:

    1. Identify the backend status of the management cluster upgrade from the k8s-bootstrapper pod.

      1. Log in to k8s-bootstrapper pod from TCA-CP:

        # kubectl exec -it <k8s-bootstrapper-pod-name> -ntca-cp-cn bash

      2. Identify the cluster ID using the management cluster name:

        # curl http://localhost:8888/api/v1/managementclusters 

      3. Identify the cluster upgrade status using the management cluster ID:

        # curl http://localhost:8888/api/v1/managementcluster/<target-mc-id>/status 

    2. Retry the management cluster upgrade based on the cluster upgrade status:

      • If the cluster upgrade is complete and its status is running, retry the management cluster upgrade from the TCA UI.

      • If the cluster upgrade is in progress and its status is upgrading, wait until the upgrade is complete and retry.

  • capv User Account Gets Locked After Three Unsuccessful Login Attempts in 15 Minutes

    The capv user account gets locked after three unsuccessful login attempts in 15 minutes. The following message appears in the Journal log:

    Mar 27 07:15:55 cp-stardard-cluster-1-control-plane-zdfgm sshd[3767202]: pam_faillock(sshd:auth): Consecutive login failures for user capv account temporarily locked

    In this issue, the Photon operating system automatically locks the user account as per the Photon 5 STIG requirement (PHTN-50-000108).

    Workaround:

    1. Log in to TCA-CP as an admin and change to the root user.

    2. SSH in to workload cluster endpoint as a capv user.

    3. Release the locked account:

      # faillock --user capv --reset

  • Workload Cluster Upgrade Might Leave a Few Pods in Terminating State

    Sometimes, the workload cluster upgrade might leave a few pods in the terminating state.

    Workaround:

    1. SSH into the workload cluster control plane.

    2. Identify the node with the status "NotReady,SchedulingDisabled":

      kubectl get node

    3. Clear the node:

      kubectl delete node <node_name>

  • Cluster Creation Might Fail if Too Many Kubernetes Node Templates are on vCenter

    If too many Kubernetes node templates are on vCenter, the cluster creation might fail with the following error in Tanzu Kubernetes Grid log reports.

    Workaround:

    1. Delete all unused Kubernetes node templates from vCenter Server.

    2. Retry the cluster creation operation.

    Note: You can also increase the resources allocated to the tkr-source-controller. For instructions on increasing resources, see KB92524.

  • Node Pool Forced Deletion Might Stuck in Processing State if Cell Site Host Goes Down

    If the cell site host goes down, the forced deletion of a node pool might get stuck in the processing state.

    Workaround: Remove the host from the vCenter inventory.

    Note: After the host is removed, the node pool is deleted successfully without requiring any additional action from TCA.

  • Management Cluster Creation Fails When vCenter Server Password Ends with Colon

    If the vCenter Server password ends with the colon character, the management cluster creation fails in Telco Cloud Automation.

    Workaround: Change the vCenter Server password so it does not end with a colon.

End of General Support Guidance

VMware Product Lifecycle Matrix outlines the End of General Support (EoGS) dates for VMware products. Lifecycle planning is required to keep each component of the VMware Telco Cloud Platform solution in a supported state. Plan the component updates and upgrades according to the EoGS dates. To ensure that the component versions are supported, you may need to update the Telco Cloud Platform solution to its latest maintenance release.

VMware pre-approval is required to use a product past its EoGS date. To discuss the extended support for products, contact your VMware representative.

Release Notes Change Log

Date

Change

Oct 1, 2024

VMware vCenter Server 8.0 Update 3b is added to the Validated Patches section.

Sep 10, 2024

VMware Telco Cloud Service Assurance 2.3.1 is replaced with VMware Telco Service Assurance 2.4.

Note: You can now use Telco Service Assurance 2.4 with Telco Cloud Platform 4.0.1.

Aug 9, 2024

The following issue is added to the Known Issues section;

  • Bare Metal Edge Devices Experience Packet Drops Causing Network Performance Degradation

Aug 7, 2024

VMware Aria Operations 8.16 is replaced with VMware Aria Operations 8.18 in the Components section.

Aug 7, 2024

The following issue is added to the Known Issues section:

  • VMware Aria Operations 8.16 Fails to Integrate with Tanzu Kubernetes Classy Standard Clusters 1.27 and 1.28 Running TLS 1.3

July 10, 2024

VMware vCenter Server is replaced with VMware vCenter.

June 14, 2024

Lifecycle details about the following Kubernetes versions are added to the What's New section:

  • Kubernetes 1.28

  • Kubernetes 1.27

  • Kubernetes 1.26

June 14, 2024

The following new feature is added to the What's New section:

  • Harbor for CNFs: Introduces support for deploying Harbor (version 2.9.3) as a CNF and managing the Harbor CNF through TCA.

June 14, 2024

Information about the following deprecated feature and alternatives is added to the Deprecated Features section:

  • DPDK Kernel Modules (rte-kni and igb_uio) are deprecated.

Support Resources

For additional support resources, see the VMware Telco Cloud Platform documentation page.

check-circle-line exclamation-circle-line close-line
Scroll to top icon