Managing internal databases

Here are two procedures for managing the internal MySQL databases used by VMware Tanzu Application Service for VMs (TAS for VMs) system components: scaling down your MySQL cluster, and migrating the cluster to a database stack that uses TLS encryption. It also provides example sizing data from two environments that have significant load on their MySQL clusters.

The procedures do not apply to databases configured as external in the Databases pane of the TAS for VMs tile.

TAS for VMs components that use system databases include the Cloud Controller, Diego Brain, Gorouter, and the User Authorization and Authentication (UAA) server. For more information, see TAS for VMs Components.

Scale down your MySQL cluster

This procedure explains how to safely scale your MySQL cluster down to a single node. If you are already running the MySQL cluster with a single node, you do not have to perform these steps.

By default, internal MySQL deploys as a single node. To take advantage of the high availability features of MySQL, you might have scaled the configuration up to three or more server nodes.

Check the health of your cluster

Before scaling down your MySQL cluster, you must ensure the cluster is healthy. To check the health of your cluster:

Using the Cloud Foundry Command Line Interface (cf CLI), target the API endpoint of your Operations Manager deployment by running:
```
cf api api.YOUR-SYSTEM-DOMAIN
```
Where YOUR-SYSTEM-DOMAIN is the system domain you configured in the Domains pane of the TAS for VMs tile.
To obtain your User Account and Authentication (UAA) Administrator user credentials:
1. Select the Credentials tab in the TAS for VMs tile.
2. Go to UAA and locate the Admin Credentials.
3. Click Link to Credential.
Log in to the API endpoint of your Operations Manager deployment by running:
```
cf login -u admin
```
Create a test organization to verify the database across all nodes by running:
```
cf create-org data-integrity-test-organization
```
To obtain the IP addresses of your MySQL server:
1. Select the Status tab in the TAS for VMs tile.
2. Record the IP addresses for all instances of the MySQL Server job.
To retrieve Cloud Controller database credentials from CredHub using the VMware Tanzu Operations Manager API:
1. Follow the procedures in Using the Tanzu Operations Manager API to authenticate and access the Tanzu Operations Manager API.
2. Use the GET /api/v0/deployed/products endpoint to retrieve a list of deployed products by running:
```
curl "https://OPS-MAN-FQDN/api/v0/deployed/products" \
  -X GET \
  -H "Authorization: Bearer UAA-ACCESS-TOKEN"
```
  Where UAA-ACCESS-TOKEN is the access token you recorded in the previous step.
3. In the response to the request, locate the product with an installation_name starting with cf- and copy its guid.
4. Run:
```
curl "https://OPS-MAN-FQDN/api/v0/deployed/products/PRODUCT-GUID/variables?name=cc-db-credentials" \
  -X GET \
  -H "Authorization: Bearer UAA-ACCESS-TOKEN"
```
  Where: * OPS-MAN-FQDN is the fully-qualified domain name (FQDN) of your Tanzu Operations Manager installation. * PRODUCT-GUID is the guid value you recorded in the previous step. * UAA-ACCESS-TOKEN is the access token you recorded earlier.
5. Record the Cloud Controller database username and password from the response to the request.
SSH into the Tanzu Operations Manager VM. For instructions specific to your IaaS, see the SSH into Tanzu Operations Manager section of the Advanced Troubleshooting with the BOSH CLI topic.
For each of the MySQL server IP addresses previously recorded:
1. From the Tanzu Operations Manager VM, query the new organization by running:
```
mysql -h YOUR-IP -u YOUR-IDENTITY -D ccdb -p -e "select created_at, name from organizations where name = 'data-integrity-test-organization'"
```
  Where: * YOUR-IP is the IP address of the MySQL server. * YOUR-IDENTITY is the identity value of the Cloud Controller database credentials you obtained earlier.
2. When prompted, provide the password value of the Cloud Controller database credentials you obtained earlier.
3. Examine the output of the mysql command and verify the created_at date is recent.
```
+---------------------+----------------------------------+
| created_at          | name                             |
+---------------------+----------------------------------+
| 2016-05-28 01:11:42 | data-integrity-test-organization |
+---------------------+----------------------------------+
```
If each MySQL server instance does not return the same created_at result, contact Broadcom Support before proceeding further or making any changes to your deployment. If each MySQL server instance does return the same result, then you can safely proceed to scaling down your cluster to a single node by performing the steps in the following section.

Set server instance count to 1

From the Operations Manager Installation Dashboard, click the TAS for VMs tile.
Select Resource Config.
Use the drop-down menu to change the Instances count for MySQL Server to 1.
Click Save, then Review Pending Changes and Apply Changes to apply the changes.

Delete your test organization by running:

cf delete-org data-integrity-test-organization

MySQL cluster sizing example

This section describes a sizing example for internal MySQL in PAS. Use this data as guidance to ensure that your MySQL clusters are scaled to handle the number of app instances running on your deployment.

Example: Diego test environment

The information in this section comes from an environment used by Diego to test the MySQL cluster with a high load similar to one generated large deployment.

IAAS: GCP
App Instances: 250,000
Average SQL Queries Per Minute: ~5,100,000
Average IOPS:
- Reads: 3.63
- Writes: 363.98

VM sizing

The following table displays MySQL VM settings for this environment:

Setting	Value
VM Type	n1-standard-16
Storage Volume Type	pd-ssd
Storage Volume Size	1TB