Running mysql-diag

Learn how to use the mysql-diag tool in VMware Tanzu Operations Manager. mysql-diag prints the state of your MySQL highly available (HA) cluster and suggests solutions if your node fails. VMware recommends running this tool against your HA cluster before all deployments.

mysql-diag checks the following information about the status of your HA cluster:

• Membership status of all nodes
• Size as it appears to all nodes
• If it needs to be bootstrapped
• If replication is working
• Used disk space and inodes per server

Run mysql-diag using the BOSH CLI

To use the BOSH CLI to run mysql-diag, do the following:

1. Obtain the information needed to use the BOSH CLI by doing the procedure in Gather Credential and IP Address Information.

2. SSH into your Tanzu Operations Manager VM by doing the procedure in Log in to the Tanzu Operations Manager VM with SSH for your IaaS.

3. Log in to your BOSH Director by doing the procedure in Authenticate with the BOSH Director VM.

4. Identify the VM to log in to with SSH by running the following command:

   bosh -e MY-ENV -d MY-DEPLOYMENT vms
   

   Where:

   • MY-ENV is the name of your environment.
   • MY-DEPLOYMENT is the name of your deployment.

5. Record the GUID associated with the mysql-monitor VM, also known as the jumpbox VM.

6. SSH into your mysql-monitor VM by running the following command:

   bosh -e MY-ENV -d MY-DEP ssh mysql-monitor/GUID
   

   Where:

   • MY-ENV is the name of your environment.
   • MY-DEPLOYMENT is the name of your deployment.
   • GUID is the GUID you recorded in the previous step.

7. View the status of your HA cluster by running the following command:

   mysql-diag
   

Example healthy output

The mysql-diag command returns the following message if your canary status is healthy:

Checking canary status...healthy

Example mysql-diag output when the tool identifies a healthy HA cluster includes:

• The date
• The canary status, which shows as healthy
• The cluster status of the three service instances
• A table with the headings Host, Name/UUID, WSREP Local State, WSREP Cluster Status, and WSREP Cluster Size. The rows contain:
  • the value for WSREP Local State (‘Synced’)
  • the value for WSREP Cluster Status (‘Primary’)
  • the value for WSREP Cluster Size (‘3’)
• The disk status of the six services instances
• A table with the headings host, name/UUID, persistent disk used, and ephemeral disk used

View a larger version of this image.

Example unhealthy output

The mysql-diag command returns the following message if your canary status is unhealthy:

Checking canary status...unhealthy

In the event of a broken HA cluster, running mysql-diag outputs actionable steps meant to expedite the recovery of that HA cluster. The sample mysql-diag output after the tool identifies an unhealthy HA cluster is similar to the output described for the healthy case except that some of the clusters show getsockopt: connection refused errors. The output ends with messages marked as Critical containing instructions for what to do and what not do to resolve the issues.

View a larger version of this image.