You can run SoS tool operations in the Cloud Builder VM to debug a failed bring-up of Cloud Foundation.

The Cloud Builder VM is available only until the SDDC Manager service is running in a successfully deployed Cloud Foundation. After a successful bring-up, run the SoS tool through SDDC Manager. See Supportability and Serviceability (SoS) Tool in the VMware Cloud Foundation Operations and Administration Guide.

To run the SoS tool, SSH in to the Cloud Builder VM using the root account, and navigate to the /opt/vmware/sddc-support directory and type ./sos followed by the options required for your desired operation.

./sos --option-1 --option-2 ... --option-n

SoS Tool Help Options

Use these options to see information about the SoS tool itself.

Option Description

--help

-h

Provides a summary of the available SoS tool options

--version

-v

Provides the SoS tool's version number.

SoS Tool Generic Options

These are generic options for the SoS tool.

Option Description
--configure-sftp Configures SFTP for logs.
--debug-mode Runs the SoS tool in debug mode.
--force
Allows SoS operations from the Cloud Foundation Builder VM after bring-up.
Note: It is recommended that you do not use this option.
--history Displays the last twenty SoS operations performed.
--log-dir LOGDIR Specifies the directory to store the logs.
--log-folder LOGFOLDER Specifies the name of the log directory.
--setup-json SETUP_JSON

Custom setup-json file for log collection.

SoS prepares the inventory automatically based on the environment where it is running. If you want to collect logs for a pre-defined set of components, you can create a setup.json file and pass the file as input to SoS. A sample JSON file is available on the Cloud Builder VM in the /opt/vmware/sddc-support/ directory.
--zip Creates a zipped tar file for the output.

SoS Tool Options for JSON Generator

Option Description
--jsongenerator Invokes the JSON generator utility.
--jsongenerator-input JSON_GENERATOR_INPUT Specify the input file to be used by the JSON generator utility.
--jsongenerator-design JSON_GENERATOR_DESIGN Specify the design file for the SDDC architecture.
--jsongenerator-logs JSONGENERATORLOGS Set the directory to be used for logs. Optional.

SoS Tool Options for Platform Audit

Option Description
--platformaudit

Invokes the platform audit operation.

--platformaudit-input FILE

Specify the input file to be used by the platform audit utility.

--platformaudit-tree

Displays a list of available audit tests.

--platformaudit-modules MODULE1,MODULE2,MODULE3

Specify the specific audit tests to run. If specifying multiple tests, separate the modules with commas.

--platformaudit-dependency

Execute audit tests with dependencies.

--platformaudit-reason

Outputs reasons for failed or skipped tests.

--platformaudit-output OUTPUT

Saves the output to the specified file.

SoS Tool Options for Health Check

These SoS commands are used for checking the health status of various components or services, including connectivity, compute, and storage.

Option Description
--certificate-health

Verifies that the component certificates are valid (within the expiry date).

--connectivity-health

Performs a connectivity health check to inspect whether the different components of the system such as the ESXi hosts, Virtual Center Servers, Log Insight VM, NSX Manager VMs, PSC VMs, SDDC Manager VM can be pinged.

--compute-health

Performs a compute health check.

--general-health

Verifies ESXi entries across all sources, checks the Postgres DB operational status for hosts, checks ESXi for error dumps, and gets NSX Manager and cluster status.

--get-host-ips

Returns server information.

--health-check

Performs all available health checks.

--services-health

Performs a services health check to confirm whether services are running

SoS Tool Log File Options

Option Description
--api-logs

Collects output from APIs.

--cassandra-logs

Collects logs from the Apache Cassandra database only.

Apache Cassandra processes run in each of the infrastructure virtual machines, the ones with ISVM in their names. These ISVM VMs run in your installation's primary rack.

--cloud-builder-cassandra-logs

Collects Cassandra database logs from the Cloud Builder VM only.

--cloud-builder-logs Collects Cloud Builder logs.
--dump-only-sddc-java-threads

Collects only the Java thread information from the SDDC Manager.

--esx-logs

Collects logs from the ESXi hosts only.

Logs are collected from each ESXi host available in the deployment.

--no-clean-old-logs

Use this option to prevent the tool from removing any output from a previous collection run.

By default, before writing the output to the directory, the tool deletes the prior run's output files that might be present. If you want to retain the older output files, specify this option.

--no-health-check

Skips the health check executed as part of log collection.

--nsx-logs

Collects logs from the NSX Managerand NSX Controller instances only.

--psc-logs

Collects logs from the Platform Services Controller instances only.

--rvc-logs

Collects logs from the Ruby vSphere Console (RVC) only. RVC is an interface for ESXi and vCenter.

Note: If the Bash shell is not enabled in vCenter, RVC log collection will be skipped .
Note: RVC logs are not collected by default with ./sos log collection.
--sddc-manager-logs

Collects logs from the SDDC Manager only.

--test

Collects test logs by verifying the files.

--vc-logs

Collects logs from the vCenter Server instances only.

Logs are collected from each vCenter server available in the deployment.

--vm-screenshots Collects screen shots from all VMs.

Sample Output

The following text is a sample output from an --ntp-health operation.

root@cloud-builder [ /opt/vmware/sddc-support ]# ./sos --ntp-health --skip-known-host --force
Welcome to Supportability and Serviceability(SoS) utility!

User passed --force flag, Running SOS from Cloud Builder VM, although Bringup is completed 
and SDDC Manager is available. Please expect failures with SoS operations.
Health Check : /var/log/vmware/vcf/sddc-support/healthcheck-2018-08-24-10-49-05-7911
Health Check log : /var/log/vmware/vcf/sddc-support/healthcheck-2018-08-24-10-49-05-7911/sos.log
SDDC Manager : sddc-manager.vrack.vsphere.local
NTP : GREEN
+-----+-----------------------------------------+------------+-------+
| SL# |                   Area                  |   Title    | State |
+-----+-----------------------------------------+------------+-------+
|  1  |    ESXi : esxi-1.vrack.vsphere.local    |  ESX Time  | GREEN |
|  2  |    ESXi : esxi-2.vrack.vsphere.local    |  ESX Time  | GREEN |
|  3  |    ESXi : esxi-3.vrack.vsphere.local    |  ESX Time  | GREEN |
|  4  |    ESXi : esxi-4.vrack.vsphere.local    |  ESX Time  | GREEN |
|  5  |     PSC : psc-1.vrack.vsphere.local     | NTP Status | GREEN |
|  6  |     PSC : psc-2.vrack.vsphere.local     | NTP Status | GREEN |
|  7  | vCenter : vcenter-1.vrack.vsphere.local | NTP Status | GREEN |
+-----+-----------------------------------------+------------+-------+

Legend:

 GREEN - No attention required, health status is NORMAL
 YELLOW - May require attention, health status is WARNING
 RED - Requires immediate attention, health status is CRITICAL

Health Check completed successfully for : [NTP-CHECK]

The following text is sample output from a --vm-screenshots log collection operation.

root@cloud-builder [ /opt/vmware/sddc-support ]# ./sos --vm-screenshots  
     --skip-known-host --force
Welcome to Supportability and Serviceability(SoS) utility!

User passed --force flag, Running SOS from Cloud Builder VM, although Bringup is completed 
and SDDC Manager is available. Please expect failures with SoS operations.
Logs : /var/log/vmware/vcf/sddc-support/sos-2018-08-24-10-50-20-8013
Log file : /var/log/vmware/vcf/sddc-support/sos-2018-08-24-10-50-20-8013/sos.log
Log Collection completed successfully for : [VMS_SCREENSHOT]