Perform an Offline Skip-Level Upgrade

If the computer on which you install the skip-level upgrade tool does not have access to depot.vmware.com, you can perform an offline skip-level upgrade to update SDDC Manager and its services to the correct version for the target VMware Cloud Foundation release.

The skip-level upgrade tool downloads the bundles required to upgrade SDDC Manager and its services and applies those bundles to achieve the target version. It downloads, but does not apply, the bundles required to upgrade the other VMware Cloud Foundation components. It also downloads, copies, and uploads install bundles for the target release that are required for new workload domain creation. You can apply those bundles from the SDDC Manager UI once SDDC Manager and its services are upgraded.

The skip-level upgrade tool does not download upgrade and install bundles for the vRealize Suite components. To download bundles for the vRealize Suite components, use the Bundle Transfer Utility. See Offline Bundle Download for VMware Cloud Foundation. For vRealize Suite install bundles, you need to download bundles only for the target release. See Download Bundles from SDDC Manager.

Note: The Bundle Transfer Utility and Skip-Level Upgrade Tool is the only supported method for downloading bundles. Do not use third-party tools or other methods to download bundles.

For information about which versions of VMware Cloud Foundation support skip-level upgrades, see Skip-Level Upgrade. For complete usage information for the VMware Cloud Foundation SDDC Skip-Level Upgrade Tool command line interface (CLI), see Skip-Level Upgrade Tool CLI.

If you performing a skip-level upgrade from VMware Cloud Foundation 3.7.2, see the KB article Skip Level Upgrade from VMware Cloud Foundation 3.7.2 fails.

You should always use the latest version of the skip-level tool to ensure that you can take advantage of all of its features. When you use any of the following options, the skip-level tool checks to see if you are using the latest version:

-c,--copy-bundles
-d,--download-bundles
--dm, --download-manifest
-e,--enable-cumulative-upgrades
-u,--upgrade
--up,--upgrade-precheck
--vb,--verify-bundles

Prerequisites

Ensure you have access to a Windows or Linux computer that meets the following requirements:
- Java 8 runtime environment.
- Connectivity to depot.vmware.com.
- Connectivity to SDDC Manager over SSH and HTTPS.
- Minimum free space of 180 GB for saving downloaded bundles. In addition, you must have 36 GB free space in your home directory.
SDDC Manager and the computer on which you run the skip-level upgrade tool should be in the same time zone and have the same NTP server configured.
The /nfs/vmware/vcf/nfs-mount/bundle/ directory on the SDDC Manager VM must have a minimum free speace of 180 GB.
Take a snapshot of the SDDC Manager VM. The skip-level upgrade tool takes this snapshot by default, unless you choose to skip it.
Take a snapshot of the management VMs.
Do not run any domain operations while an upgrade is in progress. Domain operations are creating a new VI domain, adding hosts to a cluster or adding a cluster to a workload domain, and removing clusters or hosts from a workload domain.
Ensure that there are no failed workflows in your system and none of the Cloud Foundation resources are in activating or error state. If any of these conditions are true, resolve the issue before starting the upgrade.
Confirm that the passwords for all Cloud Foundation components are valid. Keep a record of the passwords in a secure location.
If you are upgrading from VMware Cloud Foundation 3.7.1 or 3.8, you must configure dual authentication. This is not required if your starting release is VMware Cloud Foundation 3.8.1 or higher.
Run a precheck from SDDC Manager and resolve any issues before performing a skip-level upgrade. See Perform Upgrade Precheck. You can also run a precheck on the management domain using the skip-level upgrade tool. See Skip-Level Upgrade Tool CLI.

Procedure

Download the Bundle Transfer Utility & Skip Level Upgrade Tool from My VMware to a Windows or Linux computer.
You must be logged in to My VMware.
Extract the file lcm-tools-3x-build_number.tar.gz.
(Optional) To use a proxy server with the skip-level upgrade tool:

Note: The skip-level upgrade tool only supports proxy servers that do not require authentication.
1. Open the /conf/application-skiplevelupgrade.properties file.
2. Modify the Depot proxy properties section:
```
lcm.depot.adapter.proxyEnabled=true
lcm.depot.adapter.proxyHost=<proxy IP address or FQDN>
lcm.depot.adapter.proxyPort=<proxy port>
```
Download the skip-level upgrade manifest file and upgrade bundles from the VMware depot. The manifest file determines the VMware Cloud Foundation version to which a skip-level upgrade can be performed.
1. Open a command line utility on the Windows or Linux computer, change to the bin directory that contains the executables, and run the following command:
  For Windows:
```
sddcmanager-skip-level-upgrade.bat -d
```
  For Linux:
```
./sddcmanager-skip-level-upgrade -d
```
2. Enter the following information:
  - Current version of SDDC Manager. For example: 3.9.0.0.
  - Enter yes for VMware Cloud Foundation on VxRail.
  - Target version of SDDC Manager. Enter latest to upgrade to the latest version supported by the tool.
  - VMware Depot (My VMware) user name.
  - VMware Depot (My VMware) password.
  - Path to the directory for downloading the manifest and upgrade bundles.
  - Bundles to download. Enter all to download all bundles. Enter install to download all install bundles. Enter upgrade to download all upgrade bundles. Enter specific bundle names to download specific bundles, for example bundle-38371, bundle-38378.
  Once the bundles are downloaded, do not change any of the file names. The downloaded manifest file is required for future skip-level upgrade operations; do not delete it. In cases where the manifest is missing, it can be downloaded again using the ./sddcmanager-skip-level-upgrade -dm command.
After you have downloaded all the bundles, copy the complete download directory, along with the skip-level upgrade binaries (lcm-tools-3x-<build_number>), to a system that can access SDDC Manager.
Copy the manifest and bundles to SDDC Manager.
1. Run the following command:
  For Windows:
```
sddcmanager-skip-level-upgrade.bat -c
```
  For Linux:
```
./sddcmanager-skip-level-upgrade -c
```
2. Enter the following information:
  - Fully-qualified domain name (FQDN) of the SDDC Manager VM.
  - SDDC Manager super user name (primary user), for example: vcf.
  - SDDC Manager REST API user name (basic auth user), for example: admin.
  - Password for the primary user.
  - Password for the root user.
  - Password for the basic auth user.
  - Path to the directory containing the bundles and manifests.
  Once the bundles are downloaded, do not change any of the file names. The downloaded manifest file is required for future skip-level upgrade operations; do not delete it. In cases where the manifest is missing, it can be downloaded again using the ./sddcmanager-skip-level-upgrade -dm command.
After the bundles are successfully copied to SDDC Manager, run the following command to verify the bundles:
1. Run the following command:
  For Windows:
```
sddcmanager-skip-level-upgrade.bat --vb
```
  For Linux:
```
./sddcmanager-skip-level-upgrade --vb
```
2. Enter the following information:
- Fully-qualified domain name (FQDN) of the SDDC Manager VM.
- SDDC Manager super user name (primary user), for example: vcf.
- SDDC Manager REST API user name (basic auth user), for example: admin.
- Password for the primary user.
- Password for the root user.
- Password for the basic auth user.
- Local directory used for copying bundles.
The skip-level upgrade tool verifies the bundles and lists any missing bundles that you need to download before you start the upgrade. The skip-level upgrade tool also lists any missing install bundles required for the target release. You can download missing bundles using the ./sddcmanager-skip-level-upgrade -d command.
Run the skip-level upgrade.
1. Run the following command:
  For Windows:
```
sddcmanager-skip-level-upgrade.bat -u
```
  For Linux:
```
./sddcmanager-skip-level-upgrade -u
```
2. Enter the following information:
  - Fully-qualified domain name (FQDN) of the SDDC Manager VM.
  - SDDC Manager super user name (primary user), for example: vcf.
  - SDDC Manager REST API user name (basic auth user), for example: admin.
  - Password for the primary user.
  - Password for the root user.
  - Password for the basic auth user.
  - Local directory used for copying bundles.
  - Temporary IP Address of the target SDDC Manager VM. This may not be required depending on the version of VMware Cloud Foundation from which you are upgrading.
  - Subnet mask of the target SDDC Manager VM. This may not be required depending on the version of VMware Cloud Foundation from which you are upgrading.
  - Network gateway of the target SDDC Manager VM. This may not be required depending on the version of VMware Cloud Foundation from which you are upgrading.
  - Password of the primary user on the target SDDC Manager VM. This may not be required depending on the version of VMware Cloud Foundation from which you are upgrading.
  - Root password on the target SDDC Manager VM. This may not be required depending on the version of VMware Cloud Foundation from which you are upgrading.
  - Basic authentication password on the target SDDC Manager VM. This may not be required depending on the version of VMware Cloud Foundation from which you are upgrading.
You can monitor upgrade progress in the SDDC Manager Dashboard.
When the upgrade is complete, a message similar to following is displayed:
```
2021-03-08 07:57:31.360-0800 INFO  SDDC Manager Skip Level Upgrade completed with success
2021-03-08 07:57:31.360-0800 INFO  Archiving Skip Level Upgrade tool data and logs
```

What to do next

From the SDDC Manager UI, upgrade the VMware Cloud Foundation components required for the target VMware Cloud Foundation release. The components that you need to upgrade and the order in which you upgrade them will vary based on the version of VMware Cloud Foundation from which you are upgrading.

vCenter Server and Platform Services Controllers
ESXi
NSX Data Center for vSphere
NSX-T Data Center
vRealize Suite components
To download vRealize Suite upgrade bundles, see Offline Bundle Download for VMware Cloud Foundation.