Google Dataproc Reference (Beta)

This topic gives you reference information about the Google Dataproc service (csb-google-dataproc). It details the plans, configuration parameters, and binding credentials.

Caution

The Google Dataproc service offering is currently in beta and is intended for evaluation and test purposes only. Do not use this product in a production environment.

Plans

The table below lists the plans for Google Dataproc:

Plan	Description
standard	Basic plan with 1 master and N workers
ha	High availability plan with 3 masters and N workers

Plan Configuration Parameters

When configuring Cloud Service Broker for GCP you can add additional plans. For how to configure plans, see Configure Services with Cloud Service Broker for GCP.

The following table lists parameters which can only be configured for additional plans:

Parameter Name	Values	Default
name*	The plan name	n/a
id*	A unique GUID	n/a
description*	Description of the new plan	n/a
display_name	Name to use when displaying plan in marketplace	n/a
free	When false, Service Instances of this Service Plan have a cost	true
bindable	Specifies whether Service Instances of the Service Plan can bind to applications	true
plan_updateable	Whether the Plan supports upgrade/downgrade/sidegrade to another version	true

^* Required

You can also add any of the configuration parameters listed in the parameters section to your plan. To create plans with specific replication we recommend setting master_count property.

Note

If you set a parameter at plan level, developers cannot change the value when creating or updating service instances.

Configuration Parameters

You can provision a service by running:

cf create-service csb-google-dataproc PLAN-NAME SERVICE-INSTANCE-NAME -c '{"PARAMETER-NAME": "PARAMETER-VALUE"}'

You can update the configuration parameters for a service instance by running:

cf update-service SERVICE-INSTANCE-NAME -c '{"PARAMETER-NAME": "PARAMETER-VALUE"}'

The following table lists the parameters that you can configure, by using the -c flag, when provisioning a csb-google-dataproc service:

Parameter Name	Type	Description	Default	Operation
name	string	Name of the cluster to be created.	pcf-sb-INSTANCE_COUNT-TIMESTAMP	provision and update
worker_machine_type	string	The name of a Google Compute Engine machine type to create for the worker(s).	n1-standard-2	provision and update
worker_count	number	Specifies the number of worker nodes to create.	2	provision and update
master_machine_type	string	The name of a Google Compute Engine machine type to create for the master(s).	n1-standard-2	provision and update
master_count	number	Specifies the number of master nodes to create.	1	provision and update
preemptible_count	number	Specifies the number of preemptible nodes to create. This is independent of the worker count, and the type is specified by `worker_machine_type`.	0	provision and update
region	string	The region of the bucket instance.	us-central1	provision and update
credentials	string	The GCP credentials.	The value the operator entered for Credentials in Ops Manager.	provision and update
project	string	The GCP project to use.	The value the operator entered for Project in Ops Manager.	provision and update

Binding Parameters

You can bind a service by running:

cf bind-service APP-NAME SERVICE-INSTANCE --binding-name BINDING-NAME -c '{"PARAMETER-NAME": "PARAMETER-VALUE"}'

The following table lists the parameters that you can configure, by using the -c flag, when binding to a csb-google-dataproc service:

Parameter Name	Type	Description	Default
project	string	The GCP project to use.	The value the operator entered for Project in Ops Manager.

Binding Credentials

The format for binding credentials for Google Dataproc is as follows:

{
    "bucket_name" : "OUTPUT-BUCKET-NAME",
    "cluster_name" : "DATAPROC-CLUSTER-NAME",
    "region" : "REGION",
    "email" : "SERVICE-ACCOUNT-EMAIL",
    "name" : "SERVICE-ACCOUNT-NAME",
    "private_key" : "BASE64-SERVICE-ACCOUNT-PRIVATE-KEY",
    "project_id" : "GCP-PROJECT-ID"
}