You can run the Supportability and Serviceability (SoS) Utility on the VMware Cloud Builder appliance to generate a support bundle, which you can use to help debug a failed bring-up of VMware Cloud Foundation.

Note:

After a successful bring-up, you should only run the SoS Utility on the SDDC Manager VM. See Supportability and Serviceability (SoS) Tool in the VMware Cloud Foundation Administration Guide.

The SoS Utility is not a debug tool, but it does provide health check operations that can facilitate debugging a failed deployment.

To run the SoS Utility in VMware Cloud Builder, SSH in to the VMware Cloud Builder appliance using the admin administrative account, then enter su to switch to the root user, and navigate to the /opt/vmware/sddc-support directory and type ./sos followed by the options required for your desired operation.

./sos --option-1 --option-2 ... --option-n

SoS Utility Help Options

Use these options to see information about the SoS tool itself.

Option Description

--help

-h

Provides a summary of the available SoS tool options

--version

-v

Provides the SoS tool's version number.

SoS Utility Generic Options

These are generic options for the SoS Utility.

Option Description
--configure-sftp Configures SFTP for logs.
--debug-mode Runs the SoS tool in debug mode.
--force
Allows SoS operations from the VMware Cloud Builder appliance after bring-up.
Note: In most cases, you should not use this option. Once bring-up is complete, you can run the SoS Utility directly from the SDDC Manager VM.
--history Displays the last twenty SoS operations performed.
--log-dir LOGDIR Specifies the directory to store the logs.
--log-folder LOGFOLDER Specifies the name of the log directory.
--setup-json SETUP_JSON

Custom setup-json file for log collection.

SoS prepares the inventory automatically based on the environment where it is running. If you want to collect logs for a pre-defined set of components, you can create a setup.json file and pass the file as input to SoS. A sample JSON file is available on the VMware Cloud Builder in the /opt/vmware/sddc-support/ directory.
--skip-known-host-check Skips the specified check for SSL thumbprint for host in the known host.
--zip Creates a zipped tar file for the output.

SoS Utility Log File Options

Option Description
--api-logs

Collects output from APIs.

--cloud-builder-logs Collects Cloud Builder logs.
--esx-logs

Collects logs from the ESXi hosts only.

Logs are collected from each ESXi host available in the deployment.

--no-clean-old-logs

Use this option to prevent the tool from removing any output from a previous collection run.

By default, before writing the output to the directory, the tool deletes the prior run's output files that might be present. If you want to retain the older output files, specify this option.

--no-health-check

Skips the health check executed as part of log collection.

--nsx-logs

Collects logs from the NSX Manager instances only.

--rvc-logs

Collects logs from the Ruby vSphere Console (RVC) only. RVC is an interface for ESXi and vCenter.

Note: If the Bash shell is not enabled in vCenter, RVC log collection will be skipped .
Note: RVC logs are not collected by default with ./sos log collection.
--sddc-manager-logs

Collects logs from the SDDC Manager only.

--test

Collects test logs by verifying the files.

--vc-logs

Collects logs from the vCenter Server instances only.

Logs are collected from each vCenter server available in the deployment.

--vm-screenshots Collects screen shots from all VMs.

SoS Utility JSON Generator Options

The JSON generator options within the SoS Utility provide a method to execute the creation of the JSON file from a completed deployment parameter workbook. To run the JSON generator, you must provide, as a minimum, a path to the deployment parameter workbook and the design type using the following syntax:
./sos --jsongenerator --jsongenerator-input JSONGENERATORINPUT --jsongenerator JSONGENERATORDESIGN
Option Description
--jsongenerator Invokes the JSON generator utility.
--jsongenerator-input JSONGENERATORINPUT Specify the path to the input file to be used by the JSON generator utility. For example: /tmp/vcf-ems-deployment-parameter.xlsx.
--jsongenerator-design JSONGENERATORDESIGN Use vcf-ems for VMware Cloud Foundation.
--jsongenerator-supress Supress confirmation to force cleanup directory. (optional)
--jsongenerator-logs JSONGENERATORLOGS Set the directory to be used for logs. (optional)

SoS Utility Health Check Options

The SoS Utility can be used to perform health checks on various components or services, including connectivity, compute, and storage.

Note: The health check options are primarily designed to run on the SDDC Manager VM. Running them on the VMware Cloud Builder appliance requires the --force parameter, which instructs the SoS Utility to identify the SDDC Manager VM deployed by VMware Cloud Builder during the bring-up process, and then execute the health check remotely. For example:
./sos --health-check --force
Option Description
--certificate-health

Verifies that the component certificates are valid (within the expiry date).

--connectivity-health

Performs a connectivity health check to inspect whether the different components of the system such as the ESXi hosts, vCenter Servers, NSX Manager VMs, and SDDC Manager VM can be pinged.

--compute-health

Performs a compute health check.

--general-health

Verifies ESXi entries across all sources, checks the Postgres DB operational status for hosts, checks ESXi for error dumps, and gets NSX Manager and cluster status.

--get-host-ips

Returns server information.

--health-check

Performs all available health checks.

--ntp-health

Verifies whether the time on the components is synchronized with the NTP server in the VMware Cloud Builder appliance.

--services-health

Performs a services health check to confirm whether services are running

--run-vsan-checks Runs proactive vSAN tests to verify the ability to create VMs within the vSAN disks.

Sample Output

The following text is a sample output from an --ntp-health operation.

root@cloud-builder [ /opt/vmware/sddc-support ]# ./sos --ntp-health --skip-known-host --force
Welcome to Supportability and Serviceability(SoS) utility!

User passed --force flag, Running SOS from Cloud Builder VM, although Bringup is completed and SDDC Manager is available. Please expe           ct failures with SoS operations.
Health Check : /var/log/vmware/vcf/sddc-support/healthcheck-2020-02-11-23-03-53-24681
Health Check log : /var/log/vmware/vcf/sddc-support/healthcheck-2020-02-11-23-03-53-24681/sos.log
SDDC Manager : sddc-manager.vrack.vsphere.local
NTP : GREEN
+-----+-----------------------------------------+------------+-------+
| SL# |                   Area                  |   Title    | State |
+-----+-----------------------------------------+------------+-------+
|  1  |    ESXi : esxi-1.vrack.vsphere.local    |  ESX Time  | GREEN |
|  2  |    ESXi : esxi-2.vrack.vsphere.local    |  ESX Time  | GREEN |
|  3  |    ESXi : esxi-3.vrack.vsphere.local    |  ESX Time  | GREEN |
|  4  |    ESXi : esxi-4.vrack.vsphere.local    |  ESX Time  | GREEN |
|  5  | vCenter : vcenter-1.vrack.vsphere.local | NTP Status | GREEN |
+-----+-----------------------------------------+------------+-------+

Legend:

 GREEN - No attention required, health status is NORMAL
 YELLOW - May require attention, health status is WARNING
 RED - Requires immediate attention, health status is CRITICAL


Health Check completed successfully for : [NTP-CHECK]

The following text is sample output from a --vm-screenshots log collection operation.

root@cloud-builder [ /opt/vmware/sddc-support ]# ./sos --vm-screenshots  
     --skip-known-host --force
Welcome to Supportability and Serviceability(SoS) utility!

User passed --force flag, Running SOS from Cloud Builder VM, although Bringup is completed 
and SDDC Manager is available. Please expect failures with SoS operations.
Logs : /var/log/vmware/vcf/sddc-support/sos-2018-08-24-10-50-20-8013
Log file : /var/log/vmware/vcf/sddc-support/sos-2018-08-24-10-50-20-8013/sos.log
Log Collection completed successfully for : [VMS_SCREENSHOT]