In the event of a disk failure or another critical issue, you can recover the individual nodes in your VMware Integrated OpenStack deployment using the command-line interface.

When you recover a VMware Integrated OpenStack node, it returns to the state of a newly deployed node.

Prerequisites

  • If you want to recover all database nodes, you must have a backup of the OpenStack database. If you want to recover Swift nodes, you must have a backup of the Swift ring. See Back Up Your Deployment.
  • Ensure that the datastore has sufficient free space to contain the original and recovered nodes at the same time. The recovery process will delete the original node, but space for both nodes is temporarily required. To avoid this issue, you can power off and delete the existing node before recovering it.

Procedure

  1. Log in to the OpenStack Management Server as viouser.
  2. Recover OpenStack nodes by node or role.

    To display the nodes in your deployment, use the viocli show command. The values shown in the VM Name and Role columns can be used to recover nodes.

    1. To recover a non-database node, run the following command: 
      sudo viocli recover {-n node1... | -r role1... [-n node1...]}
      Option Description
      -n

      Enter the names of the nodes to recover.

      -r

      Enter the names of the roles to recover. All nodes assigned to the specified role will be recovered. You can specify -n in addition to this parameter to recover single nodes outside of the specified role.

    2. To recover a database node, run the following command: 
      sudo viocli recover {-n node1... | -r role} -dn backup-name -nfs nfs-host:/backup-folder
      Option Description
      -n

      Enter the names of the database nodes to recover. You can specify DB nodes for HA deployments or the ControlPlane node for compact or tiny deployments.

      -r

      Specify DB for HA deployments or ControlPlane for compact or tiny deployments. All database nodes will be recovered.

      -dn

      Enter the folder containing the OpenStack database backup.

      OpenStack database backup folders are in vio_os_db_yyyymmddhhmmss format.

      -nfs

      Specify the NFS host and directory where the backup is located in the format remote-host:/remote-dir.

    3. To recover a Swift node, run the following command: 
      sudo viocli recover -n node-name -dn backup-name -nfs nfs-host:/backup-folder
      Option Description
      -n

      Enter the name of a single Swift node.

      -dn

      Enter the folder containing the Swift ring backup.

      Swift backup folders are in vio_swift_ring_yyyymmddhhmmss format.

      -nfs

      Specify the NFS host and directory where the backup is located in the format remote-host:/remote-dir.

      Important: Swift nodes cannot be recovered by role or in batches. To avoid data loss, verify that the recovered node is operational and that data replication to the recovered node has completed before recovering other Swift nodes.

Results

The recovery process may take several minutes. You can check the status of your node by viewing your OpenStack deployment in the vSphere Client.