Depending on the level of your GCP accounts, you can configure Tanzu CloudHealth to populate complete cost and usage information.
Tanzu CloudHealth is granted access to all the projects in the billing account at the project level.
For detailed instructions, refer to the Configuring GCP Accounts at the Project Level using gcloud section below.
Use When:
For detailed instructions, refer to the Configuring GCP Accounts at the Project Level section below.
Tanzu CloudHealth is granted access to all projects in the billing account at the organization level.
For detailed instructions, refer to the Configuring GCP Accounts at the Organization Level using gcloud section below.
Use When:
For detailed instructions, refer to the Configuring GCP Accounts at the Organization Level section below.
The following table lists the required permission in GCP for VMware Tanzu CloudHealth configuration:
Tanzu CloudHealth Configuration | Permission Required |
---|---|
Enable BigQuery export | Billing Account Admin or Billing Account Costs Manager |
View BigQuery export configuration | Billing Account Admin, Billing Account Costs Manager, or Billing Viewer |
Create IAM member and IAM role at Organization level | Organization Admin |
Enable APIs across all projects | Project Owner (at organization level) |
When you configure your Google cloud with the Tanzu CloudHealth Platform, you need to create a service account in the Google Console and assign the service account an IAM role.
You can assign one of two roles:
For example, if you are concerned about security, you can create a custom role that only grants Tanzu CloudHealth access to specific assets.
Custom roles must, at minimum, include the following permissions for the given asset.
For BigQuery Cost Data
Field | Details |
---|---|
Required IAM Permissions | resourcemanager.projects.get, bigquery.datasets.get, bigquery.jobs.create, bigquery.tables.get, bigquery.tables.getData |
Required APIs | bigquery-json.googleapis.com , cloudresourcemanager.googleapis.com |
Reports That Require the Permissions | History Invoice Report, Cost History Report |
Collection Frequency | Every 12 Hours: History Invoice Cost, History Invoice Metadata,Cost History Cost (By Partition),Cost History Metadata. Every 24 Hours: SKU Discovery |
For GCE Disks
Field | Details |
---|---|
Required IAM Permissions | compute.disks.get, compute.disks.list, compute.projects.get, compute.regions.get, compute.regions.list, compute.zones.get, compute.zones.list |
Optional IAM Permissions | compute.disks.delete -To delete GCE disks. |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | GCE Disk Asset Report |
Collection Frequency | Every 15 minutes |
For GCE Images
Field | Details |
---|---|
Required IAM Permissions | compute.images.get, compute.images.list, compute.projects.get, compute.regions.get, compute.regions.list, compute.zones.get, compute.zones.list |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | GCE Image Asset Report |
Collection Frequency | Every 15 minutes |
For GCE Instances
Field | Details |
---|---|
Required IAM Permissions | compute.instances.get, compute.instances.list, compute.projects.get, compute.regions.get, compute.regions.list, compute.zones.get, compute.zones.list |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | GCE Asset Report, GCE Rightsizing Report, Attached Disks Asset Report |
Collection Frequency | Every 15 minutes |
For GCE Snapshots
Field | Details |
---|---|
Required IAM Permissions | compute.snapshots.get, compute.snapshots.list, compute.projects.get, compute.regions.get, compute.regions.list, compute.zones.get, compute.zones.list |
Optional IAM Permissions | compute.snapshots.delete -To delete GCE snapshots. |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | GCE Snapshot Asset Report |
Collection Frequency | Every 15 minutes |
For GCE Static IPs
Field | Details |
---|---|
Required IAM Permissions | compute.addresses.list, compute.projects.get, compute.regions.get, compute.regions.list, compute.zones.get, compute.zones.list |
Optional IAM Permissions | compute.addresses.delete -To release unattached static IP addresses. |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | GCE Static IP Asset Report |
Collection Frequency | Every 15 minutes |
For GCS Bucket
Field | Details |
---|---|
Required IAM Permissions | storage.buckets.get, storage.buckets.list |
Required APIs | storage-component.googleapis.com |
Reports That Require the Permissions | GCS Bucket Asset Report |
Collection Frequency | Every 15 minutes |
For Dataproc Clusters
Field | Details |
---|---|
Required IAM Permissions | dataproc.clusters.list, compute.regions.get, compute.regions.list, compute.zones.get, compute.zones.list |
Required APIs | dataproc.googleapis.com |
Reports That Require the Permissions | Dataproc Cluster Asset Report |
Collection Frequency | Every 4 hours |
For GKE Clusters
Field | Details |
---|---|
Required IAM Permissions | container.clusters.list |
Required APIs | container.googleapis.com |
Reports That Require the Permissions | GKE Cluster Asset Report |
Collection Frequency | Every 15 minutes |
For GCE Rightsizing Recommendations
Field | Details |
---|---|
Required IAM Permissions | recommender.computeInstanceMachineTypeRecommendations.get, recommender.computeInstanceMachineTypeRecommendations.list, recommender.computeInstanceMachineTypeRecommendations.update, recommender.locations.get, recommender.locations.list |
Required APIs | recommender.googleapis.com |
Reports That Require the Permissions | GCE Rightsizing Report |
Collection Frequency | Every 12 hours |
For Project Health Status
Field | Details |
---|---|
Required IAM Permissions | resourcemanager.projects.get |
Required APIs | cloudresourcemanager.googleapis.com |
Collection Frequency | Project Test Access (Validate Project Credentials)- Every 4 hours |
For Attached Disks
Field | Details |
---|---|
Required IAM Permissions | compute.disks.get , compute.disks.list |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | Attached Disks Asset Report |
Collection Frequency | Every 15 minutes |
For Zones
Field | Details |
---|---|
Required IAM Permissions | compute.zones.get , compute.zones.list |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | Zones Asset Report |
Collection Frequency | Every 15 minutes |
For Machine Types
Field | Details |
---|---|
Required IAM Permissions | compute.instances.get , compute.instances.list |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | Machine Types Asset Report |
Collection Frequency | Every 24 hours |
For Disk Types
Field | Details |
---|---|
Required IAM Permissions | compute.disks.get , compute.disks.list |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | Disk Types Asset Report |
Collection Frequency | Every 24 hours |
For Regions
Field | Details |
---|---|
Required IAM Permissions | compute.regions.get , compute.regions.list |
Required APIs | compute.googleapis.com |
Reports That Require the Permissions | Regions Asset Report |
Collection Frequency | Every 24 hours |
Overview:
Create a custom role (or use the default Editor role) and IAM member at the project level and assign the role and IAM member to the service account. Identical roles and IAM members are automatically created and assigned to all projects in the billing account. If a new project is added to the billing account, a role and IAM member must be manually assigned to the new project.
Use When:
Configuring GCP Accounts in gcloud
The gcloud command-line interface (CLI) is the primary CLI to the Google Cloud Platform. You can use this CLI in place of the Google Portal interface to programmatically configure your billing account in the Tanzu CloudHealth platform. To use gcloud, you can either use Cloud Shell in the Google Console or Download and install the Google Cloud SDK.
To configure the GCP billing account using gcloud CLI, you need the project ID of a project assigned to that billing account. When you complete configuration, all projects assigned to the billing account are pulled into the Tanzu CloudHealth Platform.
If you have already enabled BigQuery, use the project that contains your BigQuery dataset. Otherwise, you can use any project assigned to the billing account. To ensure that billing data isn’t lost, use the project ID of a project that won’t be deleted.
The project ID is located in the Project Info pane of the Dashboard in the Google Cloud Console.
Note - If you have already enabled BigQuery in the Google Console, skip this step.
Enter the following command in the gcloud CLI to create a dataset to use later when enabling BigQuery.
bq mk project id:dataset name
project ID
with your project ID.dataset name
with your dataset name.Enter the following command in the gcloud CLI to allow Tanzu CloudHealth to gather cost and tagging information for your projects.
for project in $(gcloud projects list --format="value(projectId)")
do
if [[ $(gcloud beta billing projects describe $project --format="value(billingEnabled)") = "True" ]];
then
echo "ProjectId: $project - Enabling APIs..."
gcloud services enable compute.googleapis.com \
cloudresourcemanager.googleapis.com \
storage-component.googleapis.com \
recommender.googleapis.com \
container.googleapis.com \
dataproc.googleapis.com \
--project $project
fi;
done
This command enables the following APIs:
Note - This gcloud command only enables the APIs for existing projects. If you create new projects in the future, you must manually enable the above APIs for the new projects.
Enter the following command in the gcloud CLI to allow Tanzu CloudHealth to gather data from your BigQuery dataset. Replace project ID
with your project ID.
gcloud services enable bigquery.googleapis.com --project project id
Enter the following command in the gcloud CLI to create a service account.
gcloud iam service-accounts create service account name --project project id
project ID
with your project ID.service account name
with your new service account name.Enter the following command in the gcloud CLI to create a private key for the newly created service account and to save the key to the $HOME directory of your instance.
gcloud iam service-accounts keys create \
--iam-account <insert service account name>@<insert project id>.iam.gserviceaccount.com <insert private key name>.json
<insert project ID>
with your project ID.<insert service account name>
with your service account name.<insert private key name>
with your new private key name.Enter the following command in the gcloud CLI to download the service account key to your local machine. Replace private key name
with your private key name.
cloudshell download private key name.json
Depending on your cloud setup, you may choose to use the Editor role or create a custom role. Tanzu CloudHealth recommend using a custom role, but you can use the Editor role with Automated setup for easier maintenance.
To create a custom role, refer to the Create a Custom Role topic.
Enter the following command to assign your preferred role to the service account.
service account name
with your service account name.project ID
with your project ID.role path
with the file path to your preferred role:roles/editor
for the Editor roleprojects/project id/roles/custom role name>
for a custom rolegcloud projects add-iam-policy-binding project id --member=serviceAccount: service account name>@ project id.iam.gserviceaccount.com \
--role role path
Create your IAM members with the preferred role from Step 8.
service account name
with your service account name.project ID
with your project ID.role path
with the file path to your preferred role:
roles/editor
for the Editor roleprojects/project id>/roles/custom role name
for a custom role for project in $(gcloud projects list --format="value(projectId)")
do
echo "ProjectId: $project"
gcloud config set project $project
gcloud projects add-iam-policy-binding $project --member user:service account name>@project id>.iam.gserviceaccount.com \
--member serviceAccount:service account name>@project id>.iam.gserviceaccount.com \
--role role path> --project $project
done
Enabling BigQuery can only be completed in the Google Console.
Note - Use the dataset you created above as your BigQuery dataset. If you have already enabled BigQuery in the Google Console, skip this step.
Tanzu CloudHealth validates new Google Service Accounts and derived projects every 4 hours. You can view derived projects by going to Setup > Accounts > GCP Project. Projects you enable in the Tanzu CloudHealth Platform change status from Not Configured to Green, Yellow, Red, or Pending. See Status of Google Project to learn more about what each status represents.
Note - The newly configured GCP billing accounts will be onboarded automatically, and within 48 hrs, you will see the cost data in the Tanzu CloudHealth platform.
Overview: Create a custom role (or use the default Editor role) and IAM member at the project level and assign the role and IAM member to the service account. Identical roles and IAM members are automatically created and assigned to all projects in the billing account. If a new project is added to the billing account, a role and IAM member must be manually assigned to the new project.
Configuring GCP Accounts in the Google Console You can configure your GCP account using the Google Console. Tanzu CloudHealth recommends using the Google Console if you are a new Google Cloud user or are unfamiliar with gcloud.
Checkpoint: At this point, you have the following information in the text document.
Repeat this process for all derived projects.
NoteTanzu CloudHealth requires your service account role to be enabled with certain permissions to properly access and report on your BigQuery data. These permissions are not included in the default Viewer role in the Google Console. If you have assigned your service account a Viewer role, you cannot view your BigQuery data in Tanzu CloudHealth and must change your service account role to a custom role.
BigQuery is Google’s enterprise data warehouse. BigQuery provides billing data that contains more information on customer datasets and is easier to use for custom reporting than daily CSV exports.
BigQuery must be enabled for billing export in the Google Console before you can enable BigQuery in the Tanzu CloudHealth platform. Complete these instructions to do so.
Once you enable Billing export to Tanzu CloudHealth, ensure your GCP BigQuery table is not empty. To verify whether the table contains cost data, click the BigQuery table name from the left navigation and select the Preview tab. For the successful configuration of the GCP accounts, the GCP BigQuery table must include the cost data.
NoteThe Tanzu CloudHealth platform supports enabling both Standard usage cost and Detailed usage cost. The Standard option with the database table name
gcp_billing_export_v1_billing_account_ID
is selected by default. If you enable the Detailed option, you must update the database table name togcp_billing_export_resource_v1_BILLING_ACCOUNT_ID
using the Tanzu CloudHealth UI or API. Note that, currently, the Tanzu CloudHealth supports detailed data export only through FlexReports. For more details, see the Detailed Data Export topic.
Enable APIs that allow Tanzu CloudHealth to gather cost and tagging information.
You need to configure only the billing account. Tanzu CloudHealth discovers all derived projects associated with the billing account that have incurred costs.
Tanzu CloudHealth validates new Google Service Accounts and derived projects every 4 hours. You can view derived projects by going to Setup > Accounts > GCP Project. Projects you enable in the Tanzu CloudHealth Platform change status from Not Configured to Green, Yellow, Red, or Pending.
Note The newly configured GCP billing accounts will be onboarded automatically, and within 48 hrs, you will see the cost data in the Tanzu CloudHealth platform.
Overview: Create a custom role and IAM member at the organization level and assign the role and IAM member to the service account. When new projects are added to the billing account, the organization-level role and IAM member are automatically inherited by the new projects.
Note - You must create a new custom role for the organization level setup, even if you already have created a Owner role or your default Editor role has all the permissions.
Use When:
Configuring GCP Accounts in gcloud The gcloud command-line interface (CLI) is the primary CLI to the Google Cloud Platform. You can use this CLI in place of the Google Portal interface to programmatically configure your billing account in the Tanzu CloudHealth platform. To use gcloud, you can either use Cloud Shell in the Google Console or Download and install the Google Cloud SDK.
To configure the GCP billing account using gcloud CLI, you need the project ID of a project assigned to that billing account and your organization ID. When you complete configuration, all projects assigned to the billing account are pulled into the Tanzu CloudHealth Platform.
If you have already enabled BigQuery, use the project that contains your BigQuery dataset. Otherwise, you can use any project assigned to the billing account. To ensure that billing data isn’t lost, use the project ID of a project that won’t be deleted.
The project ID is located in the Project Info pane of the Dashboard in the Google Cloud Console.
The organization ID is located in IAM & admin > Manage resources in the Google Console.
Enter the following command to create a custom role with the permissions Tanzu CloudHealth requires at the organization level. * Replace org id
with the ID of your organization. * Replace custom role name
with the name of the new custom role.
```
gcloud iam roles create custom role name> --organization org id \
--title custom role name --description \
"Used to collect Tanzu CloudHealth Billing and Usage data" --permissions \
resourcemanager.projects.get,compute.disks.get,compute.disks.list,compute.images.get,compute.images.list,compute.instances.get,compute.instances.list,compute.projects.get,compute.regions.get,compute.regions.list,storage.buckets.get,storage.buckets.list,bigquery.datasets.get,bigquery.jobs.create,dataproc.clusters.list,compute.addresses.list,container.clusters.list,bigquery.tables.get,bigquery.tables.getData,compute.snapshots.get,compute.snapshots.list,compute.zones.get,compute.zones.list,recommender.computeInstanceMachineTypeRecommendations.get,recommender.computeInstanceMachineTypeRecommendations.list,recommender.computeInstanceMachineTypeRecommendations.update,recommender.locations.get,recommender.locations.list \
--stage GA
```
Note - If you have already enabled BigQuery in the Google Console, skip this step.
Enter the following command in the gcloud CLI to create a dataset to use later when enabling BigQuery.
project ID
with your project ID.dataset name
with your dataset name. bq mk project id:dataset name
Enter the following command in the gcloud CLI to allow Tanzu CloudHealth to gather cost and tagging information for your projects. This command enables the following APIs:
for project in $(gcloud projects list --format="value(projectId)")
do
if [[ $(gcloud beta billing projects describe $project --format="value(billingEnabled)") = "True" ]];
then
echo "ProjectId: $project - Enabling APIs..."
gcloud services enable compute.googleapis.com \
cloudresourcemanager.googleapis.com \
storage-component.googleapis.com \
recommender.googleapis.com \
container.googleapis.com \
dataproc.googleapis.com \
--project $project
fi;
done
NoteThis gcloud command only enables the APIs for existing projects. If you create new projects in the future, you must manually enable the above APIs for the new projects. Allow about 32 hours for the platform to collect and process the data from GCP to project the cost.
Enter the following command in the gcloud CLI to allow Tanzu CloudHealth to gather data from your BigQuery dataset. Replace project ID
with your project ID.
gcloud services enable bigquery-json.googleapis.com --project project id
Enter the following command in the gcloud CLI to create a service account.
project ID
with your project ID.Replace service account name
with your new service account name.
gcloud iam service-accounts create service account name --project project id
Enter the following command in the gcloud CLI to create a private key for the newly created service account and to save the key to the $HOME directory of your instance.
project ID
with your project ID.service account name
with your service account name.Replace private key name
with your new private key name.
gcloud iam service-accounts keys create \
--iam-account service account name@project id>.iam.gserviceaccount.com private key name.json
Enter the following command in the gcloud CLI to download the service account key to your local machine. Replace private key name
with your private key name.
```
cloudshell download private key name.json
```
Enter the following command to assign the custom role you created in Step 2 to the service account.
service account name
with your service account name.project ID
with your project ID.org id
with your organization ID.Replace custom role name
with the custom role name.
gcloud projects add-iam-policy-binding project id --member serviceAccount:service account name>@project id.iam.gserviceaccount.com \
--role organizations/org id>/roles/custom role name
Create an IAM member for your organization with the custom role you created in Step 2.
service account name
with your service account name.org id
with the ID of your organization.project ID
with your project ID.Replace custom role name
with the name of the new custom role.
gcloud organizations add-iam-policy-binding org id --member user:service account name>@project id.iam.gserviceaccount.com \
--member serviceAccount:service account name@project id.iam.gserviceaccount.com \
--role organizations/org id/roles/custom role name
Enabling BigQuery can only be completed in the Google Console.
Use the dataset you created previously as your BigQuery dataset. If you have already enabled BigQuery in the Google Console, skip this step.
You need to configure only the billing account. Tanzu CloudHealth discovers all derived projects associated with the billing account that have incurred costs.
Tanzu CloudHealth validates new Google Service Accounts and derived projects every 4 hours. You can view derived projects by going to Setup > Accounts > GCP Project. Projects you enable in the Tanzu CloudHealth Platform change status from Not Configured to Green, Yellow, Red, or Pending.
The newly configured GCP billing accounts will be onboarded automatically, and within 48 hrs, you will see the cost data in the Tanzu CloudHealth platform.
Benefit: Tanzu CloudHealth is granted access to all projects in the billing account at the organization level. Overview: Create a custom role and IAM member at the organization level and assign the role and IAM member to the service account. When new projects are added to the billing account, the organization-level role and IAM member are automatically inherited by the new projects.
NoteYou must create a new custom role for the organization level setup, even if you already have created a Owner role or your default Editor role has all the permissions.
Configuring GCP Accounts in the Google Console
You can configure your GCP account using the Google Console. Tanzu CloudHealth recommends using the Google Console if you are a new Google Cloud user or are unfamiliar with gcloud.
Create a custom role in the Google Console that you can later assign to your service account.2w
Checkpoint: At this point, you have the following information in the text document.
Enable APIs that allow Tanzu CloudHealth to gather cost and tagging information.
Note - Tanzu CloudHealth requires your service account role to be enabled with certain permissions to properly access and report on your BigQuery data. These permissions are not included in the default Viewer role in the Google Console. If you have assigned your service account a Viewer role, you cannot view your BigQuery data in Tanzu CloudHealth and must change your service account role to a custom role.
BigQuery is Google’s enterprise data warehouse. BigQuery provides billing data that contains more information on customer datasets and is easier to use for custom reporting than daily CSV exports.
NoteBigQuery must be enabled for billing export in the Google Console before you can enable BigQuery in the Tanzu CloudHealth Platform. Complete these instructions to do so.
Once you enable Billing export to Tanzu CloudHealth, ensure your GCP BigQuery table is not empty. To verify whether the table contains cost data, click the BigQuery table name from the left navigation and select the Preview tab. For the successful configuration of the GCP accounts, the GCP BigQuery table must include the cost data.
NoteThe Tanzu CloudHealth platform supports enabling both Standard usage cost and Detailed usage cost. The Standard option with the database table name
gcp_billing_export_v1_billing_account_ID
is selected by default. If you enable the Detailed option, you must update the database table name togcp_billing_export_resource_v1_BILLING_ACCOUNT_ID
using the Tanzu CloudHealth UI or API. Note that, currently, the Tanzu CloudHealth supports detailed data export only through FlexReports. For more details, see the Detailed Data Export topic.
You need to configure only the billing account. Tanzu CloudHealth discovers all derived projects associated with the billing account that have incurred costs.
Tanzu CloudHealth validates new Google Service Accounts and derived projects every 4 hours. You can view derived projects by going to Setup > Accounts > GCP Project. Projects you enable in the Tanzu CloudHealth Platform change status from Not Configured to Green, Yellow, Red, or Pending.
Note The newly configured GCP billing accounts will be onboarded automatically, and within 48 hrs, you will see the cost data in the Tanzu CloudHealth platform.