General troubleshooting methods for VMware Telco Cloud Automation.
Obtain Deployment List
- For VMware Telco Cloud Automation manager, use the command
kubectl get deployments -n tca-mgr. - For VMware Telco Cloud Automation Control Plane, use the command
kubectl get deployments -n tca-cp-<namespaceSuffix>.
$ kubectl get deployments -n tca-mgr NAME READY UP-TO-DATE AVAILABLE AGE istio-ingressgateway 1/1 1 1 5d20h network-slicing-nsmf 1/1 1 1 5d20h network-slicing-nssmf 1/1 1 1 5d20h network-slicing-nssmf-ran 1/1 1 1 5d20h network-slicing-sms 1/1 1 1 5d20h redisoperator 1/1 1 1 5d20h rfs-redisfailover 1/1 1 1 5d20h tca-api 1/1 1 1 5d19h tca-app 1/1 1 1 5d19h tca-catalog-parser 1/1 1 1 5d19h tca-helm-service 1/1 1 1 5d19h tca-k8s-event-collector 1/1 1 1 5d19h tca-keycloak 1/1 1 1 5d20h tca-platform-manager 1/1 1 1 5d19h tca-prometheus-proxy 1/1 1 1 5d19h tca-proxy 1/1 1 1 5d19h tca-resource-change-monitor 1/1 1 1 5d19h tca-ui 2/2 2 2 5d19h
$ kubectl get deployments -n tca-cp-01 NAME READY UP-TO-DATE AVAILABLE AGE istio-ingressgateway 1/1 1 1 5d18h k8s-bootstrapper 1/1 1 1 5d18h nfv-ccli 1/1 1 1 5d18h nfv-tca-diagnosis-api 1/1 1 1 5d18h redisoperator 1/1 1 1 5d19h rfs-redisfailover 1/1 1 1 5d19h tca-api 1/1 1 1 5d18h tca-app 1/1 1 1 5d18h tca-extension-platform 1/1 1 1 5d18h tca-helm-service 1/1 1 1 5d18h tca-k8s-event-collector 1/1 1 1 5d18h tca-platform-manager 1/1 1 1 5d18h tca-prometheus-proxy 1/1 1 1 5d18h tca-proxy 1/1 1 1 5d18h tca-resource-change-monitor 1/1 1 1 5d18h tca-ui 1/1 1 1 5d18h
$ kubectl get deployments -n tca-cp-02 NAME READY UP-TO-DATE AVAILABLE AGE istio-ingressgateway 1/1 1 1 5d19h k8s-bootstrapper 1/1 1 1 5d19h nfv-ccli 1/1 1 1 5d19h nfv-tca-diagnosis-api 1/1 1 1 5d19h redisoperator 1/1 1 1 5d19h rfs-redisfailover 1/1 1 1 5d19h tca-api 1/1 1 1 5d19h tca-app 1/1 1 1 5d19h tca-extension-platform 1/1 1 1 5d19h tca-helm-service 1/1 1 1 5d19h tca-k8s-event-collector 1/1 1 1 5d19h tca-platform-manager 1/1 1 1 5d19h tca-prometheus-proxy 1/1 1 1 5d19h tca-proxy 1/1 1 1 5d19h tca-resource-change-monitor 1/1 1 1 5d19h tca-ui 1/1 1 1 5d19h
Check Statefulset
To obtain the statefulset for VMware Telco Cloud Automation manager, use the command
kubectl get statefulset -n tca-mgr.- To obtain the statefulset for VMware Telco Cloud Automation Control Plane, use the command
kubectl get deployments -n tca-cp-<namespaceSuffix>.
$ kubectl get statefulset -n tca-mgr NAME READY AGE kafka 1/1 5d20h mongodb 1/1 5d20h postgres 1/1 5d20h postgres-monitor 1/1 5d20h rfr-redisfailover 1/1 5d20h tca-tcf-manager 1/1 5d19h zookeeper 1/1 5d20h
$ kubectl get statefulset -n tca-cp-01 NAME READY AGE kafka 1/1 5d19h mongodb 1/1 5d19h postgres 1/1 5d18h postgres-monitor 1/1 5d18h rfr-redisfailover 1/1 5d19h zookeeper 1/1 5d19h
$ kubectl get statefulset -n tca-cp-02 NAME READY AGE kafka 1/1 5d19h mongodb 1/1 5d19h postgres 1/1 5d19h postgres-monitor 1/1 5d19h rfr-redisfailover 1/1 5d19h zookeeper 1/1 5d19h
Modify CapbkBootstrapTokenTtl in kbs.conf
To modify the CapbkBootstrapTokenTtl in kbs.conf file, see knowledgebase.
Manually restoring the backup on a Network Slicing enabled setup
To know more about manually restoring the backup on a network slicing enabled setup, see knowledgebase.
CNVA deployment issues
When TCA-M and TCA-CP are not reachable with "this site can’t be reached" in CNVA deployment:
Login to VM using ssh.
Check the status of tca installation using command
Example output for tca-mgr
admin@172-1-0-181 [~]$ Kubetcl get tcxproduct
NAME STATUS READY MESSAGE AGE
tca-common updateCompleted True All App CRs 8d
reconciled
successfully
tca-mgr updateCompleted True All App CRs 8d
reconciled
successfully
Example output for tca-cp:
admin@172-1-0-182 [~]$ Kubetcl get tcxproduct
NAME STATUS READY MESSAGE AGE
tca-mgr updateCompleted True All App CRs 8d
reconciled
successfully
tca-mgr updateCompleted True All App CRs 8d
reconciled
successfully
In the above output, if the resources are not created, then check status of systemctl status tca-deploy.service check logs using journalctl -u tca-deploy.service .
In the above output, if the resources are created but Ready is False, check more details using kubectl get tcxproduct <name> -o yaml.
When the TCA Manager web UI is not reachable with "Unable to reach tca service from ingress." in CNVA deployment:
Login to VM using ssh.
Check the status of
tca-api using kubectl get pods -n tca-mgr | grep tca-api
When TCA CP web UI is not reachable with "Unable to reach tca service from ingress." in CNVA deployment:
Login to VM using ssh.
Check the status of
tca-api using kubectl get pods -n tca-cp-cn | grep tca-api
When TCA Manager platform manager UI is not reachable with "Unable to reach tca service from ingress." in CNVA deployment:
Login to VM using ssh.
Check the status of
tca-api using kubectl get pods -n tca-mgr | grep tca-platform-manager
When TCA Manager platform manager UI is not reachable with "Unable to reach tca service from ingress." in CNVA deployment:
Login to VM using ssh.
Check the status of
tca-api using kubectl get pods -n tca-mgr | grep tca-platform-manager
How to view logs in CNVA VM:
Option1
Login to VM using ssh.
cd /logsfolder
Option2
Launch a debug pod
Execute Debug Script
Get logs of a service
How to test networking inside pod:
Launch a debug pod
Execute Debug Script
Debug Service
How to take heapdump or some other process related activities inside pod:
Launch a debug pod
Execute Debug Script
Debug Service
How to connect to postgres:
Launch a debug pod
Execute Debug Script
Connect to Postgres
