General troubleshooting methods for VMware Telco Cloud Automation.
Getting kubeconfig of the VMware Telco Cloud Automation cluster
curl -XGET --user "bootstrapperVMUsername:bootstrapperVMPassword" "https://{bootstrapperVMIP}:9443/api/admin/clusters?clusterType=MANAGEMENT"
curl -XGET --user "bootstrapperVMUsername:bootstrapperVMPassword" "https://{bootstrapperVMIP}:9443/api/admin/clusters/{clusterName}/kubeconfig?clusterType=MANAGEMENT"
API returns JSON response, use kubeconfig to get the base64 encoded kubeconfig. Perform a bas64 decode of the kubeconfig and use decoded value for the kubectl, helm commands.
Obtain Deployment List
- For VMware Telco Cloud Automation manager, use the command
kubectl get deployments -n tca-mgr. - For VMware Telco Cloud Automation Control Plane, use the command
kubectl get deployments -n tca-cp-<namespaceSuffix>.
$ kubectl get deployments -n tca-mgr NAME READY UP-TO-DATE AVAILABLE AGE istio-ingressgateway 1/1 1 1 5d20h network-slicing-nsmf 1/1 1 1 5d20h network-slicing-nssmf 1/1 1 1 5d20h network-slicing-nssmf-ran 1/1 1 1 5d20h network-slicing-sms 1/1 1 1 5d20h redisoperator 1/1 1 1 5d20h rfs-redisfailover 1/1 1 1 5d20h tca-api 1/1 1 1 5d19h tca-app 1/1 1 1 5d19h tca-catalog-parser 1/1 1 1 5d19h tca-helm-service 1/1 1 1 5d19h tca-k8s-event-collector 1/1 1 1 5d19h tca-keycloak 1/1 1 1 5d20h tca-platform-manager 1/1 1 1 5d19h tca-prometheus-proxy 1/1 1 1 5d19h tca-proxy 1/1 1 1 5d19h tca-resource-change-monitor 1/1 1 1 5d19h tca-ui 2/2 2 2 5d19h
$ kubectl get deployments -n tca-cp-01 NAME READY UP-TO-DATE AVAILABLE AGE istio-ingressgateway 1/1 1 1 5d18h k8s-bootstrapper 1/1 1 1 5d18h nfv-ccli 1/1 1 1 5d18h nfv-tca-diagnosis-api 1/1 1 1 5d18h redisoperator 1/1 1 1 5d19h rfs-redisfailover 1/1 1 1 5d19h tca-api 1/1 1 1 5d18h tca-app 1/1 1 1 5d18h tca-extension-platform 1/1 1 1 5d18h tca-helm-service 1/1 1 1 5d18h tca-k8s-event-collector 1/1 1 1 5d18h tca-platform-manager 1/1 1 1 5d18h tca-prometheus-proxy 1/1 1 1 5d18h tca-proxy 1/1 1 1 5d18h tca-resource-change-monitor 1/1 1 1 5d18h tca-ui 1/1 1 1 5d18h
$ kubectl get deployments -n tca-cp-02 NAME READY UP-TO-DATE AVAILABLE AGE istio-ingressgateway 1/1 1 1 5d19h k8s-bootstrapper 1/1 1 1 5d19h nfv-ccli 1/1 1 1 5d19h nfv-tca-diagnosis-api 1/1 1 1 5d19h redisoperator 1/1 1 1 5d19h rfs-redisfailover 1/1 1 1 5d19h tca-api 1/1 1 1 5d19h tca-app 1/1 1 1 5d19h tca-extension-platform 1/1 1 1 5d19h tca-helm-service 1/1 1 1 5d19h tca-k8s-event-collector 1/1 1 1 5d19h tca-platform-manager 1/1 1 1 5d19h tca-prometheus-proxy 1/1 1 1 5d19h tca-proxy 1/1 1 1 5d19h tca-resource-change-monitor 1/1 1 1 5d19h tca-ui 1/1 1 1 5d19h
Check Statefulset
-
To obtain the statefulset for VMware Telco Cloud Automation manager, use the command
kubectl get statefulset -n tca-mgr. - To obtain the statefulset for VMware Telco Cloud Automation Control Plane, use the command
kubectl get deployments -n tca-cp-<namespaceSuffix>.
$ kubectl get statefulset -n tca-mgr NAME READY AGE kafka 1/1 5d20h mongodb 1/1 5d20h postgres 1/1 5d20h postgres-monitor 1/1 5d20h rfr-redisfailover 1/1 5d20h tca-tcf-manager 1/1 5d19h zookeeper 1/1 5d20h
$ kubectl get statefulset -n tca-cp-01 NAME READY AGE kafka 1/1 5d19h mongodb 1/1 5d19h postgres 1/1 5d18h postgres-monitor 1/1 5d18h rfr-redisfailover 1/1 5d19h zookeeper 1/1 5d19h
$ kubectl get statefulset -n tca-cp-02 NAME READY AGE kafka 1/1 5d19h mongodb 1/1 5d19h postgres 1/1 5d19h postgres-monitor 1/1 5d19h rfr-redisfailover 1/1 5d19h zookeeper 1/1 5d19h
Restart deployment
To restart the deployed service, use the command kubectl rollout restart deployment/<name-of-deployment> -n tca-cp- <namespaceSuffix>.
Status of restart
To check the status of the restart of service, use the command kubectl rollout status deployment/<name-of-deployment> -n tca-cp-<namespaceSuffix>..
Replace <name-of-statefulset> with the actual name of the deployment service.
Example
$ kubectl rollout status deployment/tca-api -n tca-cp-01 Waiting for deployment "tca-api" rollout to finish: 1 out of 2 new replicas have been updated... Waiting for deployment "tca-api" rollout to finish: 1 out of 2 new replicas have been updated... Waiting for deployment "tca-api" rollout to finish: 1 out of 2 new replicas have been updated... Waiting for deployment "tca-api" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "tca-api" rollout to finish: 1 old replicas are pending termination... deployment "tca-api" successfully rolled out
Restart statefulset
kubectl rollout restart statefulset/<name-of-statefulset> -n tca-cp-<namespaceSuffix>
Replace <name-of-statefulset> with the actual name of the statefulset service. You can obtain the name of the stateful server using the command get statefulset kubectl.
Status of Restart statefulset
kubectl rollout status statefulset/<name-of-statefulset> -n tca-cp-<namespaceSuffix>
Replace <name-of-statefulset> with the actual name of the statefulset service. You can obtain the name of the stateful server using the command get statefulset kubectl.
Modify CapbkBootstrapTokenTtl in kbs.conf
To modify the CapbkBootstrapTokenTtl in kbs.conf file, see knowledgebase.
Manually restoring the backup on a Network Slicing enabled setup
To manually restore the backup on a network slicing enabled setup, see knowledgebase.
