Recovering and upgrading Tanzu Operations Manager

This topic provides an overview of upgrading and recovering a VMware Tanzu Operations Manager using Platform Automation Toolkit, including common errors.

Important VMware recommends persisting the zip file exported from export-installation to an external file store (for example, S3) on a regular basis. The exported installation can restore the Tanzu Operations Manager to a working state if it is not working.

Upgrading Tanzu Operations Manager

When upgrading your Tanzu Operations Manager:

• always perform an export installation
• persist that exported installation
• separate installation from upgrade
• ensure that an initial installation is done, which maintains the state

Upgrade flowchart

The upgrade-opsman task follows the flow based on the state of a Tanzu Operations Manager VM. This flowchart gives a high level overview of how the task makes decisions for an upgrade.

On successive invocations of the task, it will offer different behavior compared to the previous run. This aids in recovering from failures (that is, from an IAAS) that occur.

Recovering the Tanzu Operations Manager VM

Using the upgrade-opsman task will always delete the VM. This is done to create a consistent and simplified experience across IAASs. For example, some IAASs have IP conflicts if there are multiple Tanzu Operations Manager VMs present.

If there is an problem during the upgrade, you might need to recover your Tanzu Operations Manager VM. Recovering your VM can be done in two different ways. Both methods require an exported installation.

• Recovery using the upgrade-opsman task. Depending on the error, the VM could be recovered by re-running upgrade-opsman. This may or may not require a change to the state file, depending on if there is an ensure set for the state file resource.

• Manual recovery. The VM can always be recovered manually

• by deploying the Tanzu Operations Manager OVA, raw, or YAML from the Broadcom Support portal.

This is a list of common errors when running upgrade-opsman.

• Error: The Tanzu Operations Manager API is inaccessible. Rerun the upgrade-opsman task. The task assumes that the Tanzu Operations Manager VM is not created, and runs the create-vm and import-installation tasks.

• Error: The CLI for a supported IAAS fails. (for example, bad network, outage, and so on) The specific error is returned as output. Most errors can be fixed by re-running the upgrade-opsman task.

Restoring the original Tanzu Operations Manager VM

There may be an instance in which you want to restore a previous Tanzu Operations Manager VM before completing the upgrade.

VMware recommends that you restore a previous Tanzu Operations Manager VM manually. See Running commands locally for information about the following manual process.

1. Run delete-vm on the failed or non-desired Tanzu Operations Manager using the state.yml if applicable. opsman.yml is required for this command.

   docker run -it --rm -v $PWD:/workspace -w /workspace platform-automation-image \
   p-automator delete-vm --state-file state.yml --config opsman.yml
   

2. Run create-vm using either an empty state.yml or the state that was output by the previous step. This command requires the image file from the originally deployed version from the Broadcom Support portal (yaml, ova, raw). opsman.yml is required for this command.

   docker run -it --rm -v $PWD:/workspace -w /workspace platform-automation-image \
   p-automator create-vm --config opsman.yml --image-file original-opsman-image.yml \
   --state state.yml
   

3. Run import-installation. This command requires the exported installation of the original Tanzu Operations Manager and the env.yml used by Platform Automation Toolkit.

   docker run -it --rm -v $PWD:/workspace -w /workspace platform-automation-image \
   om --env env.yml import-installation --installation installation.zip
   

Alternatively, you can complete these steps using the upgrade-opsman command. This command requires all of the inputs described above.

docker run -it --rm -v $PWD:/workspace -w /workspace platform-automation-image \
p-automator upgrade-opsman --state-file state.yml \
--config opsman.yml --image-file original-opsman-image.yml \
--installation installation.zip --env-file env.yml