Demo Footprint of VMware Telco Cloud Service Assurance with Native Kubernetes Troubleshooting

In this chapter, you can find information about issues and solutions to deploying Demo footprint of VMware Telco Cloud Service Assurance in VMs with Native Kubernetes.

If some of the pods are getting into evicted state and the app reconciliation are failing in a Demo footprint with Native Kubernetes, then perform the following steps:

Run the following command to verify if the eviction of pod is due to disk pressure.
```
kubectl describe pod <POD-Name> | grep Message
```
If the output of the command shows the message as Disk Pressure or low on resource, then perform the following steps:
1. Delete all the evicted pods.
```
kubectl get pods -A -n kafka-edge | grep Evicted | cut -d " " -f 1 | xargs kubectl delete pod -n kafka-edge
kubectl get pods -A -n default | grep Evicted | cut -d " " -f 1 | xargs kubectl delete pod -n default
```
2. Configure the log roll over for all the Container logs on all the VMs where the native Kubernetes is deployed.
  Create a file called daemon.json in /etc/docker directory with the following contents on all the VMs where the Kubernetes cluster is deployed.
```
[tco@node1 ~]$ sudo cat /etc/docker/daemon.json
{
    "data-root": "/var/lib/docker",
    "log-driver": "json-file",
    "log-opts": {
        "max-file": "5",
        "max-size": "50m"
    }
}
```
3. Restart the docker systemd service.
```
sudo systemctl restart docker
```

If the reconciliation is failing for the Prometheus apps, perform the following steps:

$ kubectl exec -it -n tcsa-system prometheus-prometheus-kube-prometheus-prometheus-0 -- rm -rf /opt/bitnami/prometheus/data/wal/* 
$ kubectl delete pod -n tcsa-system prometheus-prometheus-kube-prometheus-prometheus-0