Recovering from a Single Provider Failure

If your single Provider VM fails, you can recover it from the latest backup of its internal vPostgres database only if you have configured the Provider Backup repository in your VMware Data Services Manager installation.

Recovering involves deploying a new Provider VM. You must deploy the new VM with the same version of the .ova, and the same IP address, as that of the original.

VMware Data Services Manager automatically transfers all trusted certificates to the new Provider VM after the backup is restored.

Audience

The procedure in this topic is performed by a VMware Data Services Manager user in the Provider Administrator role.

Prerequisites

Before you recover the Provider VM, ensure that you can identify:

• The version of the original Provider VM, and that you have the corresponding .ova.
• The IP address of the original Provider VM.
• The S3 location and credentials for the Provider Backup Repo.

If you are restoring from an TLS-enabled S3 repository, also ensure that you have the CA certificate on hand, and that you can identify a location on the Provider VM for this file.

Procedure

Perform the following procedure to recover the Provider VM in your VMware Data Services Manager installation:

1. Deploy the new Provider VM(s) with the same IP addresses as the 1.3.x release of VMware Data Services Manager.

2. Get trusted certificates through the API /appliance/provider/get-trusted-certificates.

3. Append S3 trusted certificates at the end of the downloaded certificate file. For example, Minio Secure S3 public key can be found at the location /root/.minio/certs/.

4. Upload the provider-trusted-cert.pem file that you modified in step 6.

   APIs can be found in at https:// /swagger-ui.html.

5. SSH to the newly deployed Provider VM and verify that the vpostgresql service is running.

6. Edit the file /data/pgbackrest.conf, and then add the following configuration:

   [global]
   repo1-path=<Path of the backup taken (for example, provider-backups-{id})>
   repo1-type=s3
   repo1-s3-endpoint=<URL of the S3 of for which cert is uploaded in step#4>
   repo1-s3-bucket=<Bucket that contains provider backups>
   repo1-s3-uri-style=path
   repo1-s3-verify-tls=n
   repo1-s3-key=<Placeholder>
   repo1-s3-key-secret=<Placeholder>
   repo1-s3-region=<PlaceHolder of AWS S3 else us-east-1>
   
   repo1-retention-full=7
   process-max=2
   log-level-console=info
   log-level-file=error
   start-fast=y
   delta=y
   
   [main]
   pg1-path=/data/vpgsql
   

7. Save the pgbackrest.conf file.

8. Run the following command to restore the Provider VM:

   root@providervm$ restore-provider -c /data/pgbackrest.conf
   

   Wait until the restoration process is complete.

9. Log out of the VM.

   Note: Recovery make take some time.

10. Log in to the VMware Data Services Manager Provider console to verify the recovery process.

    Note: If log in fails, recovery may still be in-progress. Wait a few minutes and try again.

11. Select System Audit from the left navigation pane.

    This actions displays the System Audit Events view, a table that lists system audit events.

12. Examine recent events listed in the table, and locate an Event Type that specifies Provider Restore Success that has an Event Time near the time of recovery.

13. Manually trigger a Provider VM backup by updating the Provider backup settings.