In this topic, you can find information about common issues and solutions from VMware Telco Cloud Operations User Interface.

Issues Causes Solution
The EDAA operations on Notifications like Acknowledge, UnAcknowlege, BrowseDetails, and Containment fails, even though the EDAA and SAM Presentation server is UP and RUNNING. The SAM configuration from a DM adapter is getting deleted because of which EDAA operations in VMware Telco Cloud Operations are failing with 404 error. Run the following command inside any of the Pod running in a Kubernetes cluster:
  1. To log in to POD, execute the following command in any of Node of a Kubernetes cluster:

    kubectl exec -it <POD NAME> bash

  2. To verify the Routes in a DM adapter, run the following command inside the pod:

    curl --location --request GET 'http://dm-adapter:9999/actuator/gateway/routes'

  3. If routes returned are empty, then run the following command to refresh the routes

    curl --location --request POST 'http://dm-adapter:9999/actuator/gateway/refresh'
While logging in to VMware Telco Cloud Operations, with any non-preconfigured users (other than admin, maint, default, or oper), following error message appears Unexpected error while handling authentication request to identity provider.
  1. LDAP integrated with keycloak is SSL configured, but LDAP certs were not imported in the CPN node.
  2. User is trying to log in with wrong user name.
  1. See LDAPS Integration with Keycloak in the VMware Telco Cloud Operations Deployment Guide.
  2. Ensure that the user name is available in Keycloak or LDPA. If not, create the user.
While logging in to VMware Telco Cloud Operations, with any non-preconfigured users (other than admin, maint, default, or oper), following error message appears Failed to process request, cause JSONObject["groups"] is not a JSONArray. User is trying to log in which is not associated to any group. Associate the user to a group and configure Role for the group in VMware Telco Cloud Operations.
User unable to log in to the VMware Telco Cloud Operations user interface.
  1. Check that Nginx and keycloak pods are ready and available using kubectl.
  2. User not provisioned in keycloak native user DB Or LDAP.
  3. LDAP not configured / connectivity issues between LDAP and keycloak.
  • If Native user management is preferred authentication mechanism(default):
    1. Log in to the Keycloak user interface at the <TCOpsUI_IP>/auth endpoint as admin.
    2. Verify that the relevant user details are present in the “Users” section. Reset password if necessary.
  • If LDAP is configured as Identity Provider:
    1. Log in to the Keycloak user interface at the <TCOpsUI_IP>/auth endpoint as an admin.
    2. Navigate to User federation section and check LDAP connection details.
    3. To validate connectivity between keycloak and LDAP, use the test option in the LDAP configuration page.
After successful login, VMware Telco Cloud Operations user interface displays that, the user’s group is not authorized / mapped to any VMware Telco Cloud Operations role. User is not associated to any usergroup in keycloak native Or LDAP.
  • If a Native User management:
    1. Log in to the Keycloak user interface and associate user to a usergroup in the keycloak user interface.
  • If an LDAP is the identity provider:
    1. Log in into the LDAP user interface and verify that user is having a provisioned "memberOf" attribute .
UserGroup is not mapped to a VMware Telco Cloud Operations. Log in to the VMware Telco Cloud Operations user interface as an admin user and create the VMware Telco Cloud Operations role to usergroup mapping.
A non-admin user is only able to see a subset of navigational links on the VMware Telco Cloud Operations user interface. User's Role does not have permissions to access all the VMware Telco Cloud Operations navigational links.
  1. Log in to the VMware Telco Cloud Operations user interface as an admin user.
  2. Validate the relevant role has the required RBAC permissions provisioned.
A non-admin user is only able to see a subset / none of the datapoints on the Grafana dashboards. 1.User's Role is restricted by a defined policy filter that allows access only to a subset of data.
  1. Log in to the VMware Telco Cloud Operations user interface as an admin user.
  2. Navigate to the policy associated with the Role.
  3. Validate the defined filters are appropriate for the role.
Sometimes Topology instance count does not match between VMware Telco Cloud Operations and SAM. Some of the error conditions are not handled. When such error appears, retry mechanism is not implemented, due to which some device are missed to add to the ArangoDB. Follow below procedure to sync the entire topology in VMware Telco Cloud Operations:

Admin user deletes SAM from Administration > Smarts Integration > Broker > SAM(delete). When delete operation is performed, associated notification or/and Topology objects related to SAM are deleted from VMware Telco Cloud Operations.

VMware Telco Cloud Operations deletes Notification/Topology asynchronously, and based on size of Topology and number of live notifications, clean-up could be time consuming. Typically for 100K events time taken for deletion would be around 30 minutes and Topology delete for about 20k devices would be around 15 minutes. Wait till both notifications and topology objects are deleted.

Verifying Clean-up of topology in VMware Telco Cloud Operations:
  1. Go to Topology > TopologyExplorer.
  2. Verify empty topology explorer is displayed indicating no devices are present.

Re-Adding SAM in VMware Telco Cloud Operations:

Re-add the SAM in VMware Telco Cloud Operations so that all topology objects are resynced and count match occurs between VMware Telco Cloud Operations and SAM:
  • Perform resync of notification/topology by adding SAM (Administration > Smarts Integration) that was deleted using Smarts Integration.
  • Once SAM is added re-sync would be performed which will resync notifications and Topology from SAM.
  • Wait for re-sync to be completed.
  • Verify once SAM is added in VMware Telco Cloud Operations topology syncs it would take upto 15 minutes depending on the number of devices.
  • Once sync completed verify topology object count between SAM and VMware Telco Cloud Operations should match.
Note:

Re-adding SAM in VMware Telco Cloud Operations procedure should not be done immediately after deletion is performed.