This topic for service authors gives you information about how to create a service adapter for an on-demand service tile using On-Demand Services SDK. For more information about service author responsibilities, see Service Author Deliverables.
A service adapter is an executable invoked by the on-demand broker (ODB) that fulfills service-specific functionality such as creating binding credentials and BOSH manifests for service instances.
To create a service adapter, do the following:
Use the On-Demand Services SDK to help you write your service adapter. See On-Demand Services Golang SDK.
Add any additional functionality from the following optional sections.
Note VMware recommends that you use the SDK to create your service adapter. If you choose not to, see Interfaces for the implementation that you need to provide.
VMware has published an SDK for teams writing their service adapters in Go. It covers command line invocation handling, parameter parsing, response serialization, and error handling so the adapter authors can focus on the service-specific logic in the adapter. For more information about the golang SDK, see the on-demand-services-sdk repository on GitHub.
The SDK supports properties in two levels for the generated BOSH manifest, manifest global and job level. Global properties are deprecated in BOSH, in favor of job level properties and job links.
For an example of property generation, see the kafka-example-service-adapter in GitHub.
Complete the following steps to install and invoke the SDK:
Install the SDK by running the following go get
command:
go get github.com/pivotal-cf/on-demand-services-sdk
Use the same version of the SDK as your ODB release. For example, if you are using v0.8.0 of the ODB BOSH release, you should check out the v0.8.0 tag of the SDK.
In the main function for the service adapter, call the HandleCLI
function:
package main
import (
"log"
"os"
"URL-FOR-SERVICE-ADAPTER-REPOSITORY"
"github.com/pivotal-cf/on-demand-services-sdk/serviceadapter"
)
func main() {
logger := log.New(os.Stderr, "[SERVICE-ADAPTER-NAME] ", log.LstdFlags)
manifestGenerator := adapter.ManifestGenerator{}
binder := adapter.Binder{}
dashboardUrlGenerator := adapter.DashboardUrlGenerator{}
handler := serviceadapter.CommandLineHandler{
ManifestGenerator: manifestGenerator,
Binder: binder,
DashboardURLGenerator: &adapter.DashboardUrlGenerator{},
SchemaGenerator: adapter.SchemaGenerator{},
}
serviceadapter.HandleCLI(os.Args, handler)
}
Where:
URL-FOR-SERVICE-ADAPTER-REPOSITORY
is the repository containing your service adapter, for example github.com/bar-org/foo-service-adapter/adapter
.SERVICE-ADAPTER-NAME
is the name of the service adapter, for example foo-service-adapter
.Note The HandleCommandLineInvocation function is deprecated, but to see its functionality, see Usage.
The HandleCLI
function accepts structs that implement the following interfaces. Your service adapter can implement these interfaces.
type CommandLineHandler struct {
ManifestGenerator ManifestGenerator
Binder Binder
DashboardURLGenerator DashboardUrlGenerator
SchemaGenerator SchemaGenerator
}
The ManifestGenerator
interface is required for all service adapters. This must generate a BOSH manifest for your service instance deployment, then output to stdout a JSON document containing the following:
The service adapter must generate the above items given information about the following:
For more information about the input parameters and expected output, see generate-manifest.
Note ODB requires generate-manifest
to be idempotent. Given the same arguments when a previous manifest is supplied—which happens during a deployment update—the command should always output the same BOSH manifest.
The following code snippet shows the interface signature:
type ManifestGenerator interface {
GenerateManifest(params GenerateManifestParams) (GenerateManifestOutput, error)
}
type GenerateManifestParams struct {
ServiceDeployment ServiceDeployment
Plan Plan
RequestParams RequestParameters
PreviousManifest *bosh.BoshManifest
PreviousPlan *Plan
PreviousSecrets ManifestSecrets
PreviousConfigs BOSHConfigs
ServiceInstanceUAAClient *ServiceInstanceUAAClient
}
type GenerateManifestOutput struct {
Manifest bosh.BoshManifest `json:"manifest"`
ODBManagedSecrets ODBManagedSecrets `json:"secrets"`
Configs BOSHConfigs `json:"configs"`
}
The Binder
interface is required for most service adapters. This includes:
create-binding
: This creates credentials for the service instance and print them to stdout as JSON. These should be unique, if possible.
For more information about binding credentials, see (Optional) Create Binding Credentials. For more information about the input parameters and expected output, see create-binding.
delete-binding
: This invalidates the created credentials, if possible. For single-user services, for example Redis, this endpoint does nothing.
For more information about the input parameters and expected output, see delete-binding.
To configure ODB to provide BOSH DNS addresses for service instances to the service adapter create-binding
and delete-binding
calls, see (Optional) Enable BOSH DNS Addresses for Bindings
The following code snippet shows the interface signature:
type Binder interface {
CreateBinding(params CreateBindingParams) (Binding, error)
DeleteBinding(params DeleteBindingParams) error
}
type CreateBindingParams struct {
BindingID string
DeploymentTopology bosh.BoshVMs
Manifest bosh.BoshManifest
RequestParams RequestParameters
Secrets ManifestSecrets
DNSAddresses DNSAddresses
}
type DeleteBindingParams struct {
BindingID string
DeploymentTopology bosh.BoshVMs
Manifest bosh.BoshManifest
RequestParams RequestParameters
Secrets ManifestSecrets
DNSAddresses DNSAddresses
}
The DashboardUrlGenerator
interface is optional. This generates an optional URL of a web-based management UI for the service instance. For more information about the input parameters and expected output, see dashboard-url.
The following code snippet shows the interface signature:
type DashboardUrlGenerator interface {
DashboardUrl(params DashboardUrlParams) (DashboardUrl, error)
}
type DashboardUrlParams struct {
InstanceID string
Plan Plan
Manifest bosh.BoshManifest
}
The SchemaGenerator
interface is optional. This generates a JSON schema to validate service-specific configuration parameters. For more information about the input parameters and expected output, see generate-plan-schemas.
The following code snippet shows the interface signature:
type SchemaGenerator interface {
GeneratePlanSchema(params GeneratePlanSchemaParams) (PlanSchema, error)
}
type GeneratePlanSchemaParams struct {
Plan Plan
}
The golang SDK provides the following helper functions.
The helper function GenerateInstanceGroupsWithNoProperties
can generate the instance groups for the BOSH manifest from the arguments passed to the adapter.
One of the inputs for this function is deploymentInstanceGroupsToJobs
, where instance groups are mapped to jobs for the deployment. Your service adapter must provide the following:
For an example job mapping, see the kafka-example-service-adapter on GitHub.
The SDK provides the methods ArbitraryContext
and Platform
. These are used to extract the context
property from the request parameters and the platform
property from within context
.
The context
property is a feature of Open Service Broker API (OSBAPI) v2.13 specification. It is used to pass through information about the environment Authentication the platform or app is in. For more information about the context
property, see the OSBAPI v2.13 specification on GitHub. If the platform does not provide a context
, the SDK returns empty values.
If a subcommand fails, the adapter must return a non-zero exit status. Any error returned by an interface function is printed to stdout, which is returned to the cf CLI user. The adapter code is responsible for performing any error logging to stderr that you think is relevant for the operator logs.
Both the stdout and stderr streams are printed in the broker log for the operator. For that reason, VMware recommends that the service adapter does not print the manifest or other sensitive details to stdout or stderr. ODB does no validation on this output.
The SDK provides three specific errors for the CreateBinding
function, which allow the adapter to exit with the appropriate code:
serviceadapter.NewBindingAlreadyExistsError()
serviceadapter.NewBindingNotFoundError()
serviceadapter.NewAppGuidNotProvidedError()
For more complete code examples, see the kafka-example-service-adapter on GitHub and the redis-example-service-adapter on GitHub.
Service authors can enable configuration of BOSH Features in their service adapters.
The SDK provides the following BoshFeatures
struct, with the option to add extra features using the ExtraFeatures
map:
type BoshFeatures struct {
UseDNSAddresses *bool `yaml:"use_dns_addresses,omitempty"`
RandomizeAZPlacement *bool `yaml:"randomize_az_placement,omitempty"`
UseShortDNSAddresses *bool `yaml:"use_short_dns_addresses,omitempty"`
ExtraFeatures map[string]interface{} `yaml:"extra_features,inline"`
}
For an example implementation, see the redis-example-service-adapter in GitHub.
For more information about BOSH Features, see Features Block in the BOSH documentation.
Secrets in the manifest can be formatted as:
You can avoid using plaintext secrets in the manifest by using ODB-managed secrets, which stores secrets on BOSH CredHub.
A service adapter might need to access secrets embedded in a service instance manifest when processing a create binding request. For example, the adapter might need credentials with sufficient privileges to add a new user to a service instance. ODB passes the service instance manifest to the adapter in the create-binding
call.
When using ODB-managed secrets, the service adapter generates secrets and uses ODB as a proxy to the BOSH CredHub config server.
The following sections provide information about how to use ODB-managed secrets to store, persist, and modify secrets in BOSH CredHub:
When you use ODB-managed secrets, ODB does the following during provision:
Generates a BOSH CredHub reference for each secret in the secrets
map in the format /odb/SERVICE-OFFERING-ID/SERVICE-INSTANCE-ID/SECRET-NAME
.
Stores the value of the secret in BOSH CredHub using the generated name.
Replaces all occurrences in the manifest of the ODB-managed secrets placeholder ((odb_secret:SECRET-NAME))
with the generated BOSH CredHub reference.
Deploys the updated manifest.
For example:{
"manifest": "password: ((odb_secret:SECRET-NAME))",
"secrets": {
"SECRET-NAME":"SOME-RANDOM-PASSWORD"
}
}
You can use the service adapter to migrate from plaintext secrets in the manifest to ODB-managed secrets that are stored in BOSH CredHub. When the generate-manifest
subcommand is provided with a previous manifest, the service adapter copies secrets from the previous deployment to the new manifest.
To migrate from plaintext secrets to ODB-managed secrets, write code in your service adapter that does the following:
<Note Secrets already stored in BOSH CredHub do not need placeholders. This is because ODB ignores BOSH CredHub references during generate-manifest
.
Detects whether a secret is a plaintext secret.
Replaces each plaintext secret from the previous manifest with an ODB-managed secrets placeholder ((odb_secret:SECRET-NAME))
in the new manifest.
Returns the value of the secrets in the secrets
map. For more information about the secrets
map, see Use ODB-Managed Secrets.
secrets
map.secrets
map at least one corresponding placeholder in the manifest.Only returns secrets in the secrets
map when the value of the secret is set for the first time, or if the value is changed. For example, this might be when:
For example:
import(
"github.com/pivotal-cf/on-demand-services-sdk/serviceadapter"
)
func extractSecret(oldValue, secretName string, secretsMap map[string]string, newValue string) {
if !( strings.HasPrefix(redisPassword, "((") && strings.HasSuffix(redisPassword, "))") ) {
// This is a plaintext secret
// Add the value to the secrets map,
secretsMap[secretName] = oldValue
// and return its placeholder to use in the manifest.
newValue = fmt.Sprintf( "((%s:%s))", serviceadapter.ODBSecretPrefix, secretName)
return newValue, secretsMap
} else {
// else: this secret could be one of the following:
// - a custom CredHub reference
// - a reference to the BOSH generated variables block
// - a CredHub reference to a secret already managed by the ODB
// In all cases, the ODB does not need to send the secret to CredHub, so it
// should not be included in the secrets map.
return oldValue
}
}
When dealing with properties that need to persist across updates, the service adapter must extract the existing name for any ODB-managed secrets from the previous manifest.
The following manifest snippet shows an ODB-managed secret with a BOSH CredHub name:
name: the-deployment
...
properties:
password: ((/odb/SERVICE-GUID/SERVICE-INSTANCE-GUID/SECRET-NAME))
If the previous manifest contains BOSH CredHub references for secrets, the generate-manifest
command must not replace properties.password
with the placeholder ((odb_secret:SECRET-NAME))
.
For more information about using the value during a bind, see create-binding.
Caution VMware discourages modifying the value of secrets without changing the secret name. If the BOSH deploy task fails during update or upgrade, ODB-managed secrets might be left in an inconsistent state. For more information, see Inconsistent Secrets after a Failed Update.
When updating or upgrading a service instance, operators might need to modify the value of an ODB-managed secret. These secrets are passed to the service adapter from the following:
cf update-service
commandTo regenerate the manifest with modified secrets, write code in your service adapter that does the following:
Replaces the property in the manifest with an ODB-managed secrets placeholder that uses a new secret name.
Uses the GenerateManifest
method to return the new secret in the secrets
map.
The service adapter must only insert the ODB-managed secrets placeholder if a secret has been modified. This is because ODB requires that the GenerateManifest
method is idempotent. When the service adapter generates a new manifest after a deployment update, it must be the same as the previous manifest when GenerateManifest
is given the same input.
ODB provides all the currently deployed secrets to the GenerateManifest
method using the previousSecrets
argument. For more information about the input to the previousSecrets
argument, see PREVIOUS-SECRETS-JSON.
To detect whether a secret has been modified, write code in your service adapter that does the following:
Compares the previous value of the secret to the new value.
If the secret has changed:
secrets
map.If the secret has remained the same:
For example code that does the above, see the redis-example-service-adapter in GitHub.
If you modify the value of a secret without providing a new secret name, ODB-managed secrets can be left in an inconsistent state if the update or upgrade of a BOSH deployment fails. This is because ODB updates the secrets in BOSH CredHub before updating the deployment.
The failed deployment might contain a mixture of old and new secrets depending on the stage that the deployment failed. When an operator attempts to troubleshoot this scenario by manually re-deploying the previous manifest, this manifest contains BOSH CredHub references that refer to the new secret values. This can cause errors with bindings.
VMware recommends that you avoid modifying secrets without using new names for new versions of secrets.
Binding credentials are a set of information that a user or app uses for permission to use a service instance. Use binding credentials to restrict the users and apps that can use your service instance.
Place binding credentials for a service instance together in the same namespace and make them unique, if possible, so that you can revoke access for specific apps and users without affecting the bindings of others.
You can store binding credentials in BOSH CredHub. For more information, see (Optional) Store Secrets on BOSH CredHub above.
VMware recommends that users are identified statelessly from the binding ID. The simplest way to do this is to name the user after the binding ID.
You can take one of three approaches to credentials for a service binding:
In this approach, the same credentials are used for all bindings. One option is to define these credentials in the service instance manifest.
This approach makes sense for services that use the same credentials for all bindings, such as Redis.
For example:
properties:
redis:
password: EXAMPLE-PASSWORD
In this approach, when the adapter generate-manifest
subcommand is invoked it generates random admin credentials and returns them as part of the service instance manifest. When the create-binding
subcommand is invoked, the adapter can use the admin credentials from the manifest to create unique credentials for the binding. Subsequent create-binding
calls create new credentials.
This approach makes sense for services with binding creation that resembles user creation, such as MySQL or RabbitMQ. For example, in MySQL the admin user can be used to create a new user and database for the binding:
properties:
admin_password: ADMIN-PASSWORD
In this approach, the author defines an agent responsible for handling the creation of credentials unique to each binding. The agent must be added as a BOSH release in the service manifest. Moreover, the service and agent jobs should be co-located in the same instance group.
This approach is useful for services where the adapter cannot, or tends not, to directly call out to the service instance and instead delegates responsibility for setting up new credentials to an agent.
For example:
releases:
- name: service-release
version: 1.5.7
- name: credentials-agent-release
version: 4.2.0
instance_groups:
- name: service-group
jobs:
- name: service-job
release: service-release
- name: credentials-agent-job
release: credentials-agent-release
Important This feature requires v266.3 or later of the BOSH Director. This is available in Tanzu Operations Manager v2.2 and later.
You can configure ODB to provide BOSH DNS addresses for service instances to the service adapter create-binding
and delete-binding
calls. This is useful when the binding for a service instance contains, or relies on, BOSH DNS addresses for that deployment. For more information about how DNS addresses are passed to the create-binding
and delete-binding
calls, see DNS-ADDRESSES-JSON.
To enable ODB to provide service instance DNS addresses to the create-binding
and delete-binding
calls, do the following:
Provide a link from the service instance’s BOSH release by choosing any job in the service release and adding the link to its spec
file.
For example:
name: redis-server-job
...
provides:
- name: example-link-1
type: example-type
For an example spec
file, see the redis-example-service-release in GitHub.
Write code in the service adapter that shares the link you provided in the BOSH manifest generated for your service instance deployment.
Share the link in the same job that you added the link to in step 1. Include the link in all instance groups that require a DNS address at binding time.
For example:
instance_groups:
- name: leader-node
jobs:
- name: redis-server-job
release: redis-cluster-release
provides: # add this section
example-link-1: {shared: true}
...
The service adapter can generate generic BOSH configs and use ODB to apply them to the BOSH Director before deployment of the service instance. This enables the service author to provide service instance-specific BOSH configs which exist only for the lifetime of the service instance.
For more detail, see Generic Configs in the BOSH documentation.
To return a BOSH config fragment specific to a service instance manifest, it must be included in the response from the generate-manifest
command, as in the following example:
{
"manifest": "
name: MY-SERVICE-INSTANCE
instance_groups:
- vm_type: MY-SERVICE-INSTANCE-small",
"configs": {
"cloud":"
vm_types:
- name: MY-SERVICE-INSTANCE-small
cloud_properties:
cpu: 1"
}
}
Where MY-SERVICE-INSTANCE
is your service instance.
ODB takes the configs in the output and for each entry creates a generic BOSH config on the director, using the map key as the config type and using the service instance deployment name as the config name. If the previous example were applied using the BOSH CLI, it would look similar to this:
$ echo <content of configs["cloud"]> > config.yml
$ bosh update-config --type cloud --name MY-SERVICE-INSTANCE config.yml
The valid config types are: cloud, cpi and runtime.
The configs are scoped for the BOSH team client ODB is deployed with and they are applied to subsequently deployed service instances.
Note VMware recommends that service adapters also namespace their configuration to avoid depending on a config value, which might be deleted in the future when the associated service instance is removed.
On updates, ODB passes the BOSH configs previously set for the instance to the service adapter on generate-manifest
. When the service adapter returns a new value for an existing type, the configuration is overwritten. When no value is returned for an existing type, that type remains as is. When new types are passed, those are set.
When the service instance is deleted, all the associated BOSH configs are also be deleted.
ODB supports a backup_agent
binding. When this binding is enabled, ODB clients can request the address of the backup agent.
To enable the backup_agent
binding:
Set support_backup_agent_binding
to true
in the broker job properties.
For example:
instance_groups:
- name: broker
jobs:
- name: broker
release: on-demand-broker-release
properties:
support_backup_agent_binding: true
...
backup_agent
binding by doing the following:
backup_agent
binding is requested, the service adapter generates a binding that only contains the backup_agent_url
in the response.backup_agent
binding is being requested. The service adapter can do this by inspecting whether the bind_resource.backup_agent
property is set to true
in the create-binding
input. SDK users can use the createBindingParams.RequestParams.BindResource()
function to do this.Package the service adapter as a BOSH release. The operator can co-locate the service adapter release with the ODB release in a BOSH manifest to place the adapter executable on the same VM as the ODB server. As a result, the adapter BOSH job’s monit
file can have no processes defined.
See the following example service adapter releases in GitHub:
For how to create a BOSH release, see Creating a Release in the BOSH documentation.