This topic for service authors gives you information about how to create a service adapter for an on-demand service tile using On-Demand Services SDK. For more information about service author responsibilities, see Service Author Deliverables.

A service adapter is an executable invoked by the on-demand broker (ODB) that fulfills service-specific functionality such as creating binding credentials and BOSH manifests for service instances.

To create a service adapter, do the following:

  1. Use the On-Demand Services SDK to help you write your service adapter. See On-Demand Services Golang SDK.

  2. Add any additional functionality from the following optional sections.

  3. Package the Service Adapter.

Note VMware recommends that you use the SDK to create your service adapter. If you choose not to, see Interfaces for the implementation that you need to provide.

On-Demand Services Golang SDK

VMware has published an SDK for teams writing their service adapters in Go. It covers command line invocation handling, parameter parsing, response serialization, and error handling so the adapter authors can focus on the service-specific logic in the adapter. For more information about the golang SDK, see the on-demand-services-sdk repository on GitHub.

The SDK supports properties in two levels for the generated BOSH manifest, manifest global and job level. Global properties are deprecated in BOSH, in favor of job level properties and job links.

For an example of property generation, see the kafka-example-service-adapter in GitHub.

Use the SDK

Complete the following steps to install and invoke the SDK:

  1. Install the SDK by running the following go get command:

    go get github.com/pivotal-cf/on-demand-services-sdk

    Use the same version of the SDK as your ODB release. For example, if you are using v0.8.0 of the ODB BOSH release, you should check out the v0.8.0 tag of the SDK.

  2. In the main function for the service adapter, call the HandleCLI function:

    package main

import (
	"log"
	"os"

	"URL-FOR-SERVICE-ADAPTER-REPOSITORY"
	"github.com/pivotal-cf/on-demand-services-sdk/serviceadapter"
)

func main() {
  logger := log.New(os.Stderr, "[SERVICE-ADAPTER-NAME] ", log.LstdFlags)
  manifestGenerator := adapter.ManifestGenerator{}
  binder := adapter.Binder{}
  dashboardUrlGenerator := adapter.DashboardUrlGenerator{}
	handler := serviceadapter.CommandLineHandler{
  	ManifestGenerator:     manifestGenerator,
  	Binder:                binder,
  	DashboardURLGenerator: &adapter.DashboardUrlGenerator{},
  	SchemaGenerator:       adapter.SchemaGenerator{},
	}
  serviceadapter.HandleCLI(os.Args, handler)
}

    Where:

    • URL-FOR-SERVICE-ADAPTER-REPOSITORY is the repository containing your service adapter, for example github.com/bar-org/foo-service-adapter/adapter.
    • SERVICE-ADAPTER-NAME is the name of the service adapter, for example foo-service-adapter.

    Note The HandleCommandLineInvocation function is deprecated, but to see its functionality, see Usage.

Interfaces

The HandleCLI function accepts structs that implement the following interfaces. Your service adapter can implement these interfaces.

type CommandLineHandler struct {
	ManifestGenerator     ManifestGenerator
	Binder                Binder
	DashboardURLGenerator DashboardUrlGenerator
	SchemaGenerator       SchemaGenerator
}

ManifestGenerator

The ManifestGenerator interface is required for all service adapters. This must generate a BOSH manifest for your service instance deployment, then output to stdout a JSON document containing the following:

The service adapter must generate the above items given information about the following:

  • BOSH Director (stemcells, release names)
  • Service instance (ID, request parameters, plan properties, IaaS resources)
  • Previous manifest, if this is an upgrade deployment

For more information about the input parameters and expected output, see generate-manifest.

Note ODB requires generate-manifest to be idempotent. Given the same arguments when a previous manifest is supplied—which happens during a deployment update—the command should always output the same BOSH manifest.

The following code snippet shows the interface signature:

type ManifestGenerator interface {
	GenerateManifest(params GenerateManifestParams) (GenerateManifestOutput, error)
}

type GenerateManifestParams struct {
	ServiceDeployment        ServiceDeployment
	Plan                     Plan
	RequestParams            RequestParameters
	PreviousManifest         *bosh.BoshManifest
	PreviousPlan             *Plan
	PreviousSecrets          ManifestSecrets
	PreviousConfigs          BOSHConfigs
        ServiceInstanceUAAClient *ServiceInstanceUAAClient
}

type GenerateManifestOutput struct {
    Manifest          bosh.BoshManifest `json:"manifest"`
    ODBManagedSecrets ODBManagedSecrets `json:"secrets"`
    Configs           BOSHConfigs       `json:"configs"`
}

Binder

The Binder interface is required for most service adapters. This includes:

  • create-binding: This creates credentials for the service instance and print them to stdout as JSON. These should be unique, if possible.
    For more information about binding credentials, see (Optional) Create Binding Credentials. For more information about the input parameters and expected output, see create-binding.

  • delete-binding: This invalidates the created credentials, if possible. For single-user services, for example Redis, this endpoint does nothing.
    For more information about the input parameters and expected output, see delete-binding.

To configure ODB to provide BOSH DNS addresses for service instances to the service adapter create-binding and delete-binding calls, see (Optional) Enable BOSH DNS Addresses for Bindings

The following code snippet shows the interface signature:

type Binder interface {
	CreateBinding(params CreateBindingParams) (Binding, error)
	DeleteBinding(params DeleteBindingParams) error
}

type CreateBindingParams struct {
	BindingID          string
	DeploymentTopology bosh.BoshVMs
	Manifest           bosh.BoshManifest
	RequestParams      RequestParameters
	Secrets            ManifestSecrets
	DNSAddresses       DNSAddresses
}

type DeleteBindingParams struct {
	BindingID          string
	DeploymentTopology bosh.BoshVMs
	Manifest           bosh.BoshManifest
	RequestParams      RequestParameters
	Secrets            ManifestSecrets
	DNSAddresses       DNSAddresses
}

(Optional) DashboardUrlGenerator

The DashboardUrlGenerator interface is optional. This generates an optional URL of a web-based management UI for the service instance. For more information about the input parameters and expected output, see dashboard-url.

The following code snippet shows the interface signature:

type DashboardUrlGenerator interface {
	DashboardUrl(params DashboardUrlParams) (DashboardUrl, error)
}

type DashboardUrlParams struct {
	InstanceID string
	Plan       Plan
	Manifest   bosh.BoshManifest
}

(Optional) SchemaGenerator

The SchemaGenerator interface is optional. This generates a JSON schema to validate service-specific configuration parameters. For more information about the input parameters and expected output, see generate-plan-schemas.

The following code snippet shows the interface signature:

type SchemaGenerator interface {
	GeneratePlanSchema(params GeneratePlanSchemaParams) (PlanSchema, error)
}

type GeneratePlanSchemaParams struct {
	Plan Plan
}

Helpers

The golang SDK provides the following helper functions.

GenerateInstanceGroupsWithNoProperties

The helper function GenerateInstanceGroupsWithNoProperties can generate the instance groups for the BOSH manifest from the arguments passed to the adapter.

One of the inputs for this function is deploymentInstanceGroupsToJobs, where instance groups are mapped to jobs for the deployment. Your service adapter must provide the following:

  • The mapping of instance groups to jobs for the deployment.
  • Job level properties for the generated instance groups. The helper function does not address these properties.

For an example job mapping, see the kafka-example-service-adapter on GitHub.

ArbitraryContext and Platform

The SDK provides the methods ArbitraryContext and Platform. These are used to extract the context property from the request parameters and the platform property from within context.

The context property is a feature of Open Service Broker API (OSBAPI) v2.13 specification. It is used to pass through information about the environment Authentication the platform or app is in. For more information about the context property, see the OSBAPI v2.13 specification on GitHub. If the platform does not provide a context, the SDK returns empty values.

Error Handling

If a subcommand fails, the adapter must return a non-zero exit status. Any error returned by an interface function is printed to stdout, which is returned to the cf CLI user. The adapter code is responsible for performing any error logging to stderr that you think is relevant for the operator logs.

Both the stdout and stderr streams are printed in the broker log for the operator. For that reason, VMware recommends that the service adapter does not print the manifest or other sensitive details to stdout or stderr. ODB does no validation on this output.

The SDK provides three specific errors for the CreateBinding function, which allow the adapter to exit with the appropriate code:

serviceadapter.NewBindingAlreadyExistsError()
serviceadapter.NewBindingNotFoundError()
serviceadapter.NewAppGuidNotProvidedError()

For more complete code examples, see the kafka-example-service-adapter on GitHub and the redis-example-service-adapter on GitHub.

BOSH Features

Service authors can enable configuration of BOSH Features in their service adapters.

The SDK provides the following BoshFeatures struct, with the option to add extra features using the ExtraFeatures map:

type BoshFeatures struct {
	UseDNSAddresses      *bool                  `yaml:"use_dns_addresses,omitempty"`
	RandomizeAZPlacement *bool                  `yaml:"randomize_az_placement,omitempty"`
	UseShortDNSAddresses *bool                  `yaml:"use_short_dns_addresses,omitempty"`
	ExtraFeatures        map[string]interface{} `yaml:"extra_features,inline"`
}

For an example implementation, see the redis-example-service-adapter in GitHub.

For more information about BOSH Features, see Features Block in the BOSH documentation.

(Optional) Store Manifest Secrets on BOSH CredHub

Secrets in the manifest can be formatted as:

  • Literal BOSH CredHub references
  • BOSH variables
  • Plaintext

You can avoid using plaintext secrets in the manifest by using ODB-managed secrets, which stores secrets on BOSH CredHub.

A service adapter might need to access secrets embedded in a service instance manifest when processing a create binding request. For example, the adapter might need credentials with sufficient privileges to add a new user to a service instance. ODB passes the service instance manifest to the adapter in the create-binding call.

When using ODB-managed secrets, the service adapter generates secrets and uses ODB as a proxy to the BOSH CredHub config server.

The following sections provide information about how to use ODB-managed secrets to store, persist, and modify secrets in BOSH CredHub:

How ODB Processes ODB-Managed Secrets

When you use ODB-managed secrets, ODB does the following during provision:

  1. Generates a BOSH CredHub reference for each secret in the secrets map in the format /odb/SERVICE-OFFERING-ID/SERVICE-INSTANCE-ID/SECRET-NAME.

  2. Stores the value of the secret in BOSH CredHub using the generated name.

  3. Replaces all occurrences in the manifest of the ODB-managed secrets placeholder ((odb_secret:SECRET-NAME)) with the generated BOSH CredHub reference.

  4. Deploys the updated manifest.

    For example: 
    {
  "manifest": "password: ((odb_secret:SECRET-NAME))",
  "secrets": {
    "SECRET-NAME":"SOME-RANDOM-PASSWORD"
  }
}

Migrate from Plaintext Secrets to ODB-Managed Secrets

You can use the service adapter to migrate from plaintext secrets in the manifest to ODB-managed secrets that are stored in BOSH CredHub. When the generate-manifest subcommand is provided with a previous manifest, the service adapter copies secrets from the previous deployment to the new manifest.

To migrate from plaintext secrets to ODB-managed secrets, write code in your service adapter that does the following:

<Note Secrets already stored in BOSH CredHub do not need placeholders. This is because ODB ignores BOSH CredHub references during generate-manifest.

  • Detects whether a secret is a plaintext secret.

  • Replaces each plaintext secret from the previous manifest with an ODB-managed secrets placeholder ((odb_secret:SECRET-NAME)) in the new manifest.

  • Returns the value of the secrets in the secrets map. For more information about the secrets map, see Use ODB-Managed Secrets.

    • Each placeholder in the manifest a corresponding entry in the secrets map.
    • Each key in the secrets map at least one corresponding placeholder in the manifest.

  • Only returns secrets in the secrets map when the value of the secret is set for the first time, or if the value is changed. For example, this might be when:

    • A new service is created.
    • You migrate plaintext secrets into BOSH CredHub.
    • You want to change the value for previously set secrets.

For example:

import(
  "github.com/pivotal-cf/on-demand-services-sdk/serviceadapter"
)

func extractSecret(oldValue, secretName string, secretsMap map[string]string, newValue string) {
  if !( strings.HasPrefix(redisPassword, "((") && strings.HasSuffix(redisPassword, "))") ) {
    // This is a plaintext secret
    // Add the value to the secrets map,
    secretsMap[secretName] = oldValue
    // and return its placeholder to use in the manifest.
    newValue = fmt.Sprintf( "((%s:%s))", serviceadapter.ODBSecretPrefix, secretName)
    return newValue, secretsMap
  } else {
    // else: this secret could be one of the following:
    // - a custom CredHub reference
    // - a reference to the BOSH generated variables block
    // - a CredHub reference to a secret already managed by the ODB
    // In all cases, the ODB does not need to send the secret to CredHub, so it
    // should not be included in the secrets map.
    return oldValue
  }
}

Persist Secrets across Updates

When dealing with properties that need to persist across updates, the service adapter must extract the existing name for any ODB-managed secrets from the previous manifest.

The following manifest snippet shows an ODB-managed secret with a BOSH CredHub name:

name: the-deployment
...
properties:
password: ((/odb/SERVICE-GUID/SERVICE-INSTANCE-GUID/SECRET-NAME))

If the previous manifest contains BOSH CredHub references for secrets, the generate-manifest command must not replace properties.password with the placeholder ((odb_secret:SECRET-NAME)).

For more information about using the value during a bind, see create-binding.

Modify ODB-Managed Secrets

Caution VMware discourages modifying the value of secrets without changing the secret name. If the BOSH deploy task fails during update or upgrade, ODB-managed secrets might be left in an inconsistent state. For more information, see Inconsistent Secrets after a Failed Update.

When updating or upgrading a service instance, operators might need to modify the value of an ODB-managed secret. These secrets are passed to the service adapter from the following:

  • Plan properties in the on-demand broker manifest
  • Adapter secrets given in the adapter config
  • Configuration parameters in the cf update-service command

To regenerate the manifest with modified secrets, write code in your service adapter that does the following:

  1. Replaces the property in the manifest with an ODB-managed secrets placeholder that uses a new secret name.

  2. Uses the GenerateManifest method to return the new secret in the secrets map.

Detect When Secrets Are Modified

The service adapter must only insert the ODB-managed secrets placeholder if a secret has been modified. This is because ODB requires that the GenerateManifest method is idempotent. When the service adapter generates a new manifest after a deployment update, it must be the same as the previous manifest when GenerateManifest is given the same input.

ODB provides all the currently deployed secrets to the GenerateManifest method using the previousSecrets argument. For more information about the input to the previousSecrets argument, see PREVIOUS-SECRETS-JSON.

To detect whether a secret has been modified, write code in your service adapter that does the following:

  1. Compares the previous value of the secret to the new value.

  2. If the secret has changed:

    1. Inserts the ODB-managed secrets placeholder.
    2. Adds the value to the secrets map.

    If the secret has remained the same:

    1. Inserts the BOSH CredHub reference from the previous manifest.

For example code that does the above, see the redis-example-service-adapter in GitHub.

Inconsistent Secrets after a Failed Update

If you modify the value of a secret without providing a new secret name, ODB-managed secrets can be left in an inconsistent state if the update or upgrade of a BOSH deployment fails. This is because ODB updates the secrets in BOSH CredHub before updating the deployment.

The failed deployment might contain a mixture of old and new secrets depending on the stage that the deployment failed. When an operator attempts to troubleshoot this scenario by manually re-deploying the previous manifest, this manifest contains BOSH CredHub references that refer to the new secret values. This can cause errors with bindings.

VMware recommends that you avoid modifying secrets without using new names for new versions of secrets.

(Optional) Create Binding credentials

Binding credentials are a set of information that a user or app uses for permission to use a service instance. Use binding credentials to restrict the users and apps that can use your service instance.

Place binding credentials for a service instance together in the same namespace and make them unique, if possible, so that you can revoke access for specific apps and users without affecting the bindings of others.

You can store binding credentials in BOSH CredHub. For more information, see (Optional) Store Secrets on BOSH CredHub above.

VMware recommends that users are identified statelessly from the binding ID. The simplest way to do this is to name the user after the binding ID.

You can take one of three approaches to credentials for a service binding:

Use Static Credentials

In this approach, the same credentials are used for all bindings. One option is to define these credentials in the service instance manifest.

This approach makes sense for services that use the same credentials for all bindings, such as Redis.

For example:

properties:
  redis:
    password: EXAMPLE-PASSWORD

Use credentials unique to each Binding

In this approach, when the adapter generate-manifest subcommand is invoked it generates random admin credentials and returns them as part of the service instance manifest. When the create-binding subcommand is invoked, the adapter can use the admin credentials from the manifest to create unique credentials for the binding. Subsequent create-binding calls create new credentials.

This approach makes sense for services with binding creation that resembles user creation, such as MySQL or RabbitMQ. For example, in MySQL the admin user can be used to create a new user and database for the binding:

properties:
  admin_password: ADMIN-PASSWORD

Use an Agent

In this approach, the author defines an agent responsible for handling the creation of credentials unique to each binding. The agent must be added as a BOSH release in the service manifest. Moreover, the service and agent jobs should be co-located in the same instance group.

This approach is useful for services where the adapter cannot, or tends not, to directly call out to the service instance and instead delegates responsibility for setting up new credentials to an agent.

For example:

releases:
  - name: service-release
    version: 1.5.7
  - name: credentials-agent-release
    version: 4.2.0

instance_groups:
  - name: service-group
    jobs:
      - name: service-job
        release: service-release
      - name: credentials-agent-job
        release: credentials-agent-release

(Optional) Enable BOSH DNS Addresses for Bindings

Important This feature requires v266.3 or later of the BOSH Director. This is available in Tanzu Operations Manager v2.2 and later.

You can configure ODB to provide BOSH DNS addresses for service instances to the service adapter create-binding and delete-binding calls. This is useful when the binding for a service instance contains, or relies on, BOSH DNS addresses for that deployment. For more information about how DNS addresses are passed to the create-binding and delete-binding calls, see DNS-ADDRESSES-JSON.

To enable ODB to provide service instance DNS addresses to the create-binding and delete-binding calls, do the following:

  1. Provide a link from the service instance’s BOSH release by choosing any job in the service release and adding the link to its spec file.

    For example:

    name: redis-server-job
...
provides:
  - name: example-link-1
    type: example-type


    For an example spec file, see the redis-example-service-release in GitHub.

  2. Write code in the service adapter that shares the link you provided in the BOSH manifest generated for your service instance deployment.

    Share the link in the same job that you added the link to in step 1. Include the link in all instance groups that require a DNS address at binding time.

    For example:

    instance_groups:
- name: leader-node
  jobs:
  - name: redis-server-job
    release: redis-cluster-release
    provides:                          # add this section
      example-link-1: {shared: true}
    ...

(Optional) Use Generic BOSH Configs with Service Instances

The service adapter can generate generic BOSH configs and use ODB to apply them to the BOSH Director before deployment of the service instance. This enables the service author to provide service instance-specific BOSH configs which exist only for the lifetime of the service instance.

For more detail, see Generic Configs in the BOSH documentation.

To return a BOSH config fragment specific to a service instance manifest, it must be included in the response from the generate-manifest command, as in the following example:

{
  "manifest": "
    name: MY-SERVICE-INSTANCE
    instance_groups:
    - vm_type: MY-SERVICE-INSTANCE-small",
  "configs": {
    "cloud":"
       vm_types:
       - name: MY-SERVICE-INSTANCE-small
         cloud_properties:
           cpu: 1"
  }
}

Where MY-SERVICE-INSTANCE is your service instance.

ODB takes the configs in the output and for each entry creates a generic BOSH config on the director, using the map key as the config type and using the service instance deployment name as the config name. If the previous example were applied using the BOSH CLI, it would look similar to this:

$ echo <content of configs["cloud"]> > config.yml
$ bosh update-config --type cloud --name MY-SERVICE-INSTANCE config.yml

The valid config types are: cloud, cpi and runtime.

The configs are scoped for the BOSH team client ODB is deployed with and they are applied to subsequently deployed service instances.

Note VMware recommends that service adapters also namespace their configuration to avoid depending on a config value, which might be deleted in the future when the associated service instance is removed.

On updates, ODB passes the BOSH configs previously set for the instance to the service adapter on generate-manifest. When the service adapter returns a new value for an existing type, the configuration is overwritten. When no value is returned for an existing type, that type remains as is. When new types are passed, those are set.

When the service instance is deleted, all the associated BOSH configs are also be deleted.

(Optional) Enable Backup Agent Bindings

ODB supports a backup_agent binding. When this binding is enabled, ODB clients can request the address of the backup agent.

To enable the backup_agent binding:

  1. Set support_backup_agent_binding to true in the broker job properties.
    For example:

    instance_groups:
- name: broker
  jobs:
  - name: broker
    release: on-demand-broker-release
    properties:
      support_backup_agent_binding: true
    ...
  2. Update your service adapter to support the backup_agent binding by doing the following:
    • Ensure that when the backup_agent binding is requested, the service adapter generates a binding that only contains the backup_agent_url in the response.
    • Ensure that the service adapter can detect when the backup_agent binding is being requested. The service adapter can do this by inspecting whether the bind_resource.backup_agent property is set to true in the create-binding input. SDK users can use the createBindingParams.RequestParams.BindResource() function to do this.

Package the Service Adapter

Package the service adapter as a BOSH release. The operator can co-locate the service adapter release with the ODB release in a BOSH manifest to place the adapter executable on the same VM as the ODB server. As a result, the adapter BOSH job’s monit file can have no processes defined.

See the following example service adapter releases in GitHub:

For how to create a BOSH release, see Creating a Release in the BOSH documentation.

check-circle-line exclamation-circle-line close-line
Scroll to top icon