Backup and Restore of Metrics

Backup and restore for metrics can be configured in VMware Telco Cloud Service Assurance during or post deployment. Backup and Restore for metrics is not enabled by default. Also, there is no support for metrics backup and restore in local PV. If you want to backup and restore metrics, configure MinIO in gateway mode using AWS S3 or vSAN.

How to Backup Metrics

Connect to the deployer VM and execute the following command:
```
export KUBECONFIG=/root/.kube/<KubeConfig File>
```
In a text editor, create the configuration file for the backup in YAML format.
The example file is located in tcx-deployer/examples/backup-and-restore/backup.yaml.example directory.
Use the following template for the metrics backup:
```
apiVersion: tcx.vmware.com/v1alpha1
kind: Backup
metadata:
  name: metricsbackup-name
spec:
  components:
    elastic:
      enabled: true
      indices: "METRICS"
      timeout: "30m"
      removeAndAddRepository: false
      cleanUpIndices: ""
  retentionPolicy:
    numberOfDaysToKeep: 45
```
Based on the metrics size, update the timeout parameter.
METRICS="vsametrics-raw-*,vsametrics-hour-*,vsametrics-day-*,vsametrics-week-*,.vsa-metricservice-jobs-*"
- vsametrics-raw-*--<device>-<yyyy.mm.dd>: This entry gets updated with every received metrics from incoming Elasticsearch metrics data.
- vsametrics-hour-*--<device>-<yyyy.weekofyear>: Provides aggregated hourly entry for every week of metrics data.
- vsametrics-week-*--<device>-<yyyy>: Provides aggregated weekly entry for every year.
- vsametrics-day-*--<device>-<yyyy.month>: Provides aggregated daily entry for a month for a device.
- .vsa-metricservice-jobs-logs-*: Provides Domain Manager metric job logs.
- .vsa-metricservice-jobs: Provides Domain Manager metric job configuration.

To create backup of metrics, run the following command:

kubectl apply -f <configuration YAML file>

The output is:

backup.tcx.vmware.com/metricsbackup-name created

Verify the backup has been created successfully.
```
kubectl get backups.tcx.vmware.com
```

How to Restore Metrics

Connect to the VM deployer and execute the following command:
```
export KUBECONFIG=/root/.kube/<KubeConfig File>
```
In a text editor, create the configuration file for the backup in YAML format.
The example file is located in tcx-deployer/examples/backup-and-restore/restore.yaml.example directory.
Use the following template for the metrics restore:
```
apiVersion: tcx.vmware.com/v1alpha1
kind: Restore
metadata:
  name: restore-name
spec:
  restore:
    elastic:
      enabled: true
      timeout: "30m"
      cleanUpIndices: "METRICS"
  backupName: "metricsbackup-name"
```
To restore the backup, run the following command:
```
kubectl apply -f <configuration YAML file>
```
The following command gives the status of the restore CR:
```
~> kubectl get restore.tcx.vmware.com --watch
```

While restoring the data to the same cluster, ensure that the collectors are not running.

Note: 25 K managed devices, 250 K events, and 10 million metrics can generate approximately 3 GB of Elasticsearch data. Hence, the backup and restore time for every 1 GB of data is 20 seconds and 40 seconds. Accordingly, update the timeout value in backup and restore CRDs.