Upgrade the VMware Cloud Director Appliance by Using an Update Package

You can upgrade the VMware Cloud Director appliance to the latest version or apply patches to the VMware Cloud Director appliance by using an update package.

During the upgrade of the VMware Cloud Director appliance deployment, the VMware Cloud Director service stops working and some downtime can be expected. The downtime depends on the time you need to upgrade each VMware Cloud Director appliance and to run the VMware Cloud Director database upgrade script. The number of working cells in the VMware Cloud Director server group reduces until you stop the VMware Cloud Director service on the last VMware Cloud Director appliance. A properly configured load balancer in front of the VMware Cloud Director HTTP endpoints should stop routing traffic to the cells that are stopped.

After you apply the upgrade to every VMware Cloud Director appliance and the database upgrade is complete, you must reboot each VMware Cloud Director appliance.

Prerequisites

Take a snapshot of the primary VMware Cloud Director appliance.

When upgrading from version 10.1 or later or when patching, if the automatic failover in case of a primary database service failure is enabled, change the failover mode to Manual during the upgrade. After the upgrade, you can set the failover mode to Automatic. See Automatic Failover of the VMware Cloud Director Appliance.
Log in to the vCenter Server instance on which resides the primary VMware Cloud Director appliance of your database high availability cluster.
Navigate to the primary VMware Cloud Director appliance, right-click it, and click Power > Shut Down Guest OS.
Right-click the appliance and click Snapshots > Take Snapshot. Enter a name and, optionally, a description for the snapshot, and click OK.
Right-click the VMware Cloud Director appliance and click Power > Power On.
Verify that all nodes in your database high availability configuration are in a good state. See View the VMware Cloud Director Appliance Cluster Health and Failover Mode.
Familiarize yourself with the backup procedure for the VMware Cloud Director appliance. See Back Up the Embedded Database of the VMware Cloud Director Appliance.

Procedure

In a Web browser, log in to the appliance management user interface of a VMware Cloud Director appliance instance to identify the primary appliance, https://appliance_ip_address:5480.
Make a note of the primary appliance name. You must upgrade the primary appliance before the standby and application cells. You must use the primary appliance when backing up the database.
On all appliances in the cluster, shut down VMware Cloud Director by running the following command:
```
/opt/vmware/vcloud-director/bin/cell-management-tool -u <admin_username> cell --shutdown
```
Download the update package to the appliance you are upgrading.

Note: You must upgrade the primary appliance first.
VMware Cloud Director is distributed as an executable file with a name of the form VMware_Cloud_Director_v.v.v.v- nnnnnnnn_update. tar.gz, where v. v. v. v represents the product version and nnnnnnnn the build number. For example, VMware_Cloud_Director_10.1.0.4424-14420378_update.tar.gz.
Create the local-update-package directory in which to extract the update package.
```
mkdir /tmp/local-update-package
```

Extract the update package in the newly created directory.

 tar -zxf VMware_Cloud_Director_v.v.v.v-nnnnnnnn_update.tar.gz \
				-C /tmp/local-update-package

Set the local-update-package directory as the update repository.
```
vamicli	update --repo file:///tmp/local-update-package
```
Check for updates to verify that you established correctly the repository.
```
vamicli update --check
```
The upgrade release appears as an Available Update.
Create a backup using the primary or standby appliance management UI.
You cannot use the application cell to perform a backup.
Apply the available upgrade.
```
vamicli update --install latest
```
Repeat steps 3-7 and step 9 on the remaining standby and application cells.
From any appliance, run the VMware Cloud Director database upgrade utility.
```
/opt/vmware/vcloud-director/bin/upgrade
```
Reboot each VMware Cloud Director appliance.
```
shutdown -r now
```

What to do next

Verify that the upgrade is successful.
- Log in to the VMware Cloud Director Service Provider Admin Portal.
- Log in to the VMware Cloud Director appliance management UI and confirm that all the appliances appear with a Healthy status.
For each cell, verify that there are no add-on upgrade errors in vcloud-container-info.log.
Verify that the logs do not show any certificate convert errors during the upgrade. If there are certificate convert errors in the logs, you cannot backup the VMware Cloud Director appliance. If any errors appear, do not attempt the upgrade again and depending on the error, fix the problem manually before running the convert command again.
```
/opt/vmware/vcloud-director/bin/cell-management-tool certificates --convert
```
There are different workarounds depending on the error, for example:
- If the following exception appears in /opt/vmware/var/log/vami/updatecli.log during the certificates' conversion process, see KB article 88372.
```
 <JAVA_HOME>/lib/ext exists, extensions mechanism no longer supported; Use -classpath instead.
             .Error: Could not create the Java Virtual Machine.
              Error: A fatal exception has occurred. Program will exit. 
```
- If the /opt/vmware/var/log/vami/updatecli.log has errors from the certificates' conversion process related to a bad key or if it does not state any reason for the failures, possible workarounds are:
  - If there are no .pem and .key files ready, verify that the keystore file exists. You can find the location of the keystore file in the global.properties file.
  - If VMware Cloud Director finds the keystore file, extract the .pem and .key files from the keystore using the keytool utility or the OpenSSL tool.
  - If VMware Cloud Director does not find the keystore file, reconfigure the certificates entirely. To learn more about creating certificates, see SSL Certificate Creation and Management of the VMware Cloud Director Appliance.
  - If there are existing .pem and .key files related to the wildcard certificates, to replace the existing certificates on the node with the wildcard certificates, use the cell management tool certificate command.
```
/opt/vmware/vcloud-director/bin/cell-management-tool certificates -j --cert /opt/vmware/vcloud-director/data/transfer/user.http.pem --key /opt/vmware/vcloud-director/data/transfer/user.http.key --key-password key_password
/opt/vmware/vcloud-director/bin/cell-management-tool certificates -p --cert /opt/vmware/vcloud-director/data/transfer/user.consoleproxy.pem --key /opt/vmware/vcloud-director/data/transfer/user.consoleproxy.key --key-password key_password
```
If the upgrade is successful, you can delete the snapshot of the VMware Cloud Director appliance.
If the upgrade is not successful, you can roll back the VMware Cloud Director appliance to the snapshot that you took before the upgrade. See Roll Back a VMware Cloud Director Appliance When an Upgrade Fails.