VMware Telco Cloud Service Assurance 2.0.0 Release Notes

VMware Telco Cloud Service Assurance 2.0.0 | 02 JUN 2022 Check for additions and updates to these release notes.

About VMware Telco Cloud Service Assurance

VMware Telco Cloud Service Assurance is a real-time automated service assurance solution designed to holistically monitor and manage complex 5G NFV virtual and physical infrastructure and services end to end, from mobile core to the RAN to the edge. From single pane of glass, VMware Telco Cloud Service Assurance provides cross‑domain, multi‑layer, automated assurance in a multi‑vendor and multi‑cloud environment. It provides operational intelligence to reduce complexity, perform rapid root cause analysis and see how problems impacts services and customers across all layers lowering costs, and improved customer experience.

For information about setting up and using VMware Telco Cloud Service Assurance, see the VMware Telco Cloud Service Assurance Documentation.

What's New

The VMware Telco Cloud Service Assurance v2.0 introduces the following enhancements:

• 5G Core and vRAN Assurance Support

  VMware Telco Cloud Service Assurance offers cross-domain and cross-layered service assurance: from core to edge, from physical infrastructure to network service. In addition, it has an intuitive wizard to configure data collectors required for 5G use case and provides flexibility in configuring the Physical, VIM, CaaS, NFV-Orchestrator, and Kafka collectors. It also acts as a single-pane of glass for CNF, CaaS, VIM, and Physical infrastructure notification views.

  Furthermore, VMware Telco Cloud Service Assurance provides visualization of cross-tier (Physical, VIM, CaaS and Network functions) topology, cross-tier correlation and Root-Cause Analysis, performance cross-launch, and access to the latest vRAN Assurance Dashboards and Reports.

• Integration with VMware vRealize Operations

  VMware Telco Cloud Service Assurance leverages vRealize Operations to build a complete end to end topology map and correlation. Additionally, VMware Telco Cloud Service Assurance provides specialized use cases for telecom providers, and technologies such as VNFs, CNFs and network slices.

• Integration with VMware Telco Cloud Automation

  VMware Telco Cloud Automation allows VMware Telco Cloud Service Assurance to get notified when a network service is installed. So, VMware Telco Cloud Service Assurance automatically discovers, modules and start capturing fault and performance data with correlation with all the known monitored elements.

  Additionally, integration with VMware Telco Cloud Service Assurance creates remediation capabilities on service level in case of service failure.

• New Kafka collector for data collection

  Kafka collector is now available to integrate commonly used Kafka supported devices with VMware Telco Cloud Service Assurance. Including collection of fault events and performance metrics, which are collected, correlated, and reported by VMware Telco Cloud Service Assurance.

• Kafka Connector Support

  Introduced Kafka Connector, this feature provides the capability to replicate metrics, events, and topology data on internal Kafka bus to an external Kafka bus to third party management systems and data lakes.

• Enhanced REST APIs to query notifications and topology of discovered network data
  • API to query Notification details, causes and caused-by, and historical data.
  • API to search for Notifications based on a filters.
  • API to retrieve Topology maps, Network class types, Relations, and Instances.
• Identity Management System Enhancements
  • High available identity brokering of authentication requests to VMware Identity Manager.
  • User federation to Microsoft Active Directory.

Additional features and enhancements

• Platform Usability and Deployability
  • VMware Tanzu Kubernetes Grid (TKG ) and Azure Kubernetes Service (AKS) deployment support in a cloud-native architecture.
  • Enhanced dashboards for core infrastructure services.
  • Improved Health Statistic dashboard for monitoring VMware Telco Cloud Service Assurance components.
  • Improved troubleshooting with centralized application log view.
  • Scaling of VMware Telco Cloud Service Assurance deployments across footprints, that is 25k to 50k/100k and 50k to 100k.
• Enhanced user interface for Kafka Mapper with rich user assistance features
  • Auto completion of JSONPath expression in the editor.
  • Catalog Integration to recommend model-based properties and metrics for Kafka Mapper definition.
  • Evaluate feature to view the output metric or event based on the sample JSON input and Kafka Mapper definition.
• Alarming Enhancements
  • User interface that allows user to create alarm definitions, deploy or remove alarms, and check alarm deployment status.
  • Receive and process relevant metrics data from Kafka Collector specific to 5G vRAN and Domain Manager.
  • Activate new alarms when the computed alarming conditions are met, or deactivate the existing alarms when the computed alarming conditions are no longer met.
• User Interface Enhancements
  • Health Dashboard to monitor health of an entity directly from the selected Notification or Topology entity.
  • Cross launch to VMware vRealize Operations user interface from Notification details view.
  • Data Model Catalogs to view VMware Telco Cloud Service Assurance metric models.
  • Uniform Topology tree across Topology Map and Topology Explorer.
  • Re-sizable details view in Topology Map.
  • Notification Summary view in the Notification consoles.
  • User interface for creating and configuring Kafka Connector instance.

Known Issues

• After editing the alarm definition, the legacy job active alarm is not getting cleared.

  Due to the behaviour of stream application on flink,  the newly redeployed alarm job are not able to process historic data retrospectively. The existing active alarms can be cleared only when future data causes the alarm to be activated again.

• Log level messages are empty in service logs (Kibana logs).

  There is no common logging between the services. And, some services do not add log level in the log message. Hence, logs are empty. The logging system is displaying the emitted information by the service.

  This is a cosmetic issue which will be fixed once common library is added.

• Prometheus restarts observed.

  Due to pod recreation the Prometheus services are restarting. No downtime seen and no functional impact.

• VMware Telco Cloud Service Assurance currently does not support connections to SAM server when Broker is configured in secure mode.

  Currently there is no workaround. Broker must configured in non-authenticate mode.

  Note: EDAA related operations including the Acknowledge, Ownership, Server Tools, Browse Details > Containment and Browse Details > Domain Manager are not supported when Broker is configured in secure mode.

• When the number of hops of connectivity is increased, you may experience performance issues in the topology maps.

  There might be performance issues in the rendering of Redundancy Group and SDN connectivity map types in the Map Explorer view. This issue is observed on deployments with a complex topology where the topology maps may stop working when the number of hops of connectivity is increased.

• Broker failover is not supported in VMware Telco Cloud Service Assurance.

  Primary Broker fails in the Domain manager failover environment.

  Currently when a Broker (multi-broker) failover happens, then it requires a manual intervention where you need to log in to VMware Telco Cloud Service Assurance  and change the Broker IP address to point to the new Broker IP.

  Procedure:

  1. Go to https://IPaddress of the Control Plane Node.
  2. Navigate to Administration > Configuration > Smarts Integration
  3. Delete the existing Smarts Integration Details.
  4. Re-add the Smarts Integration Details by pointing it to secondary Broker.

• Weekly indexes are not displayed while creating custom reports, only daily and hourly index are shown part of reports.

  Procedure for workaround:

  1. Select  Configurations > Data Sources from the left side menu bar
  2. Click Add Data Source.
  3. Select Elasticsearch.
  4. Enter relevant name based on the metric-type for which the weekly index needs to be created (for example: Week-Network-Interface) and the Elastic http url as  http://elasticsearch:9200, refer any other VMware Telco Cloud Service Assurance data sources
  5. Enter Index Name based on the metric type for which the weekly index needs to be create ([vsametrics-week-networkinterface-]YYYY.MM) and select Pattern "Monthly"
  6. Enter the Time Field Name timestamp and Version 7+.
  7. Keep the rest of the fields to default  value.
  8. Click Save & Test.

• Notification count mismatch between SAM and VMware Telco Cloud Service Assurance UI due to non-filtering of notification with Owner field set to SYSTEM​. By default in VMware Telco Cloud Service Assurance there are no filters set.

  Manually apply the filter to remove notifications with Owner field not containing SYSTEM in VMware Telco Cloud Service Assurance Notification Console window by following below steps:

  1. Go to Default Notification Console.
  2.  Click Customize View.
  3. Go to Filters and provide Filter Set Name, for example Filterout SYSTEM Notifications.
  4. Filter Section Add Attribute with below condition:

     Property = Owner

     Expression = regex

     Value = ~(SYSTEM+)

  5. Click Update.

  Verify the Default Notification Console has only those notifications whose owner not set to SYSTEM. The default notification count must match between SAM and VMware Telco Cloud Service Assurance UI.

• The Containment, Browse detail, Notification Acknowledge/Unacknowledge does not work when the primary Tomcat server fails in a HA environment.

  In a Failover deployment, when the primary Tomcat fails, the UI operations including the Notification Acknowledgement, Containment, Browse Detail, and Domain Managers fail.

  When the primary Tomcat instance fails in a failover environment, then you can manually point the VMware Telco Cloud Service Assurance to a secondary Tomcat instance.

  Procedure:

  1. Go to https://IPaddress of the Control Plane Node.
  2. Navigate to Administration > Configuration > Smarts Integration
  3. Delete the existing Smarts Integration Details.
  4. Re-add the Smarts Integration Details by editing the EDAA URL and pointing it to the secondary Tomcat Instance.

• The SAM server is getting listed in the Domain Manager section instead of Presentation SAM section.

  During Smart integration and configuration, INCHARGE SA (SAM server) is getting listed in the Domain Manager section. This problem occurs only when, the SAM server is started in Non-EDAA Mode.

  To get listed under Presentation SAM section, start the SAM server in EDAA Mode.

• While starting server, the Map error warning message appears for INCHARGE-SA and INCHARGE-OI in respective logs.

  No functional impacts.

• User needs to mandatorily  discover ESX Servers for getting Virtual Machine Down event. Currently the Virtual Machine Down event is not generated if the corresponding ESX Servers are not discovered in IP Server. So, its recommended to discover Virtual Machines to get proper Root cause events.

• On RHEL 7.8 version machine when SAM services are started, brcontrol shows IPv6 entry for servers due to which communication between servers is getting impacted.

  On RHEL 7.8 version, if you start any domain manager as a service, the domain gets registered to a broker using both v4 and v6 IP address space. Due to this issue domain manager v6 entry will go to DEAD state in brcontrol output and the communication between the servers is failing sometimes due to this issue.

  Note: Issue also detected on some machines with RHEL 7.2 and 7.6.

  To avoid a domain running in v6 mode, allow only v4, by setting the below flag in runcmd_env.sh file:

  SM_IP_VERSIONS=v4

  Restart the domain manager, after updating runcmd_env.sh file.

• Topology synchronization is taking more than 10 minutes for 25k devices, when latency between SAM and VMware Telco Cloud Service Assurance is more than 5 milliseconds.

  When the latency increases topology synchronization time increases. 

  Ensure that the latency between VMware Telco Cloud Service Assurance (Topology Collector) and SAM Presentation server is less than 5 milliseconds.

• Notification processing rate is slower, when the latency between SAM and VMware Telco Cloud Service Assurance is greater than 5milliseconds.

  When the latency between VMware Telco Cloud Service Assurance and SAM Presentation server increases, notification processing rate goes down.

  Ensure that the latency between VMware Telco Cloud Service Assurance (Notification Collector) and SAM Presentation server is less than 5 milliseconds.

• Cisco-ACI, VIMS, and KPI features will be supported in upcoming release of VMware Telco Cloud Service Assurance.

• Post Incremental scale from one footprint to another footprint, VMware Telco Cloud Service Assurance user interface shows older base footprint instead of destination upgraded footprint.

  Post Incremental scale from 50k to 100k, About option in the VMware Telco Cloud Service Assurance user interface shows Footprint : 50k instead of 100k.

  There is no workaround and it has no functional impact, since the actual incremental scale happens properly only in the About UI footprint information is not updated.

• In Service Logs, the dark theme is not working. When user selects the Service Logs in the user interface, the default theme is not changing.

   Kibana GUI dark theme support is not available in the opendistro version of Kibana.

• vROps Contextual Launch button's name in vRops notifications is Launch External Application instead of Launch vROps Application".

  vROps Contextual launch notifications have the launch button for vRops called as Launch External Application. This must be the application name which is getting launched (that is Launch vROps Application).