Backup and Restore of Metrics

Backup and restore for metrics can be configured in VMware Telco Cloud Service Assurance during or post deployment. Backup and Restore for metrics is not enabled by default. Also, there is no support for metrics backup and restore in local PV. If you want to backup and restore metrics, configure MinIO in gateway mode using NFS File Server.

How to Backup Metrics

1. Connect to the deployer VM and execute the following command:

   export KUBECONFIG=/root/.kube/<KubeConfig File>

2. In a text editor, create the configuration file for the backup in YAML format.

   The example file is located in tcx-deployer/examples/backup-and-restore/backup.yaml.example directory.

   Use the following template for the metrics backup:

   apiVersion: tcx.vmware.com/v1
   kind: Backup
   metadata:
     name: metricsbackup-name
   spec:
     storage:
       minio:
         bucket: bucket-name
         endpoint: minio.default.svc.cluster.local:9000
         secretRef:
           name: minio-secrets
           namespace: default
           accessKey:
             key: root-user
           secretKey:
             key: root-password
     acrossClusters:
       enabled: true
     cluster:
       name: tcsa2.1
       namespace: default
     components:
       elastic:
         timeout: 30m
         config:
           endpoint:
             host: elasticsearch
             port: 9200
             scheme: https
           region: ap-south-1
         tls:
           secretName: elasticsearch-cert
           namespace: default
           tlsCrt:
             key: tls.crt
           caCrt:
             key: ca.crt
         authentication:
           name: elasticsearch-secrets
           namespace: default
           usernameKey:
             key: username
           passwordKey:
             key: password
         indexList:
           - vsametrics-raw-*
           - vsametrics-hour-*
           - vsametrics-day-*
           - vsametrics-week-*
     retentionPolicy:
       numberOfDaysToKeep: 45

   Based on the metrics size, update the timeout parameter.

   These are th metrics indices "vsametrics-raw-*,vsametrics-hour-*,vsametrics-day-*,vsametrics-week-*,.vsa-metricservice-jobs-*"

   • vsametrics-raw-*-<yyyy.mm.dd>: This entry gets updated with every received metrics from incoming Elasticsearch metrics data.
   • vsametrics-hour-*-<yyyy.weekofyear>: Provides aggregated hourly entry for every week of metrics data.
   • vsametrics-week-*-<yyyy>: Provides aggregated weekly entry for every year.
   • vsametrics-day-*-<yyyy.month>: Provides aggregated daily entry for a month for a device.
   • .vsa-metricservice-jobs-logs-*: Provides Domain Manager metric job logs.
   • .vsa-metricservice-jobs: Provides Domain Manager metric job configuration.
3. To create backup of metrics, run the following command:

   kubectl apply -f <configuration YAML file>

   The output is:

   backup.tcx.vmware.com/metricsbackup-name created

4. Verify the backup has been created successfully.

   kubectl get backups.tcx.vmware.com

How to Restore Metrics

1. Connect to the VM deployer and execute the following command:

   export KUBECONFIG=/root/.kube/<KubeConfig File>

2. In a text editor, create the configuration file for the backup in YAML format.

   The example file is located in tcx-deployer/examples/backup-and-restore/restore.yaml.example directory.

   Use the following template for the metrics restore:

   apiVersion: tcx.vmware.com/v1
   kind: Restore
   metadata:
     name: metrics-restore
   spec:
     restore:
       elastic:
         timeout: 30m
         config:
           endpoint:
             host: elasticsearch
             port: 9200
             scheme: https
           region: ap-south-1
         tls:
           secretName: elasticsearch-cert
           namespace: default
           tlsCrt:
             key: tls.crt
           caCrt:
             key: ca.crt
         authentication:
           name: elasticsearch-secrets
           namespace: default
           usernameKey:
             key: username
           passwordKey:
             key: password
         indexList:
           - vsametrics-*
     backupName: "metricsbackup-name"

3. To restore the backup, run the following command:

   kubectl apply -f <configuration YAML file>

   The following command gives the status of the restore CR:

   ~> kubectl get restore.tcx.vmware.com --watch

While restoring the data to the same cluster, ensure that the collectors are not running.

Note: 25 K managed devices, 250 K events, and 10 million metrics can generate approximately 3 GB of Elasticsearch data. Hence, the backup and restore time for every 1 GB of data is 20 seconds and 40 seconds. Accordingly, update the timeout value in backup and restore CRDs.