Getting Started: ODB on a Local Development Environment

This topic tells you how to create and manage an on-demand service broker (ODB) on a local development machine using PCF Dev and BOSH Lite.

For more information about the components in this topic, see PCF Dev and BOSH-Lite.

Note The examples in this topic are based on Kafka open source messaging. See the following sample code directories:

• Kafka example service
• Kafka example service adapter
• Kafka example app

Prerequisites

Before you set up and use ODB on your local machine, install and configure the following components:

• BOSH Lite v9000.131.0 or later. To install BOSH Lite, see Install in the BOSH Lite documentation.

• PCF Dev file pcfdev-v0.19.1-rc.46. To install PCF Dev, see Installing PCF Dev. Record the PCF Dev domain for later. The default is local.pcfdev.io.

Part 1: Set Up

This section details how to prepare BOSH Lite and set up the Kafka example service, the Kakfa example service adapter, and ODB.

Step 1: Prepare BOSH Lite

To prepare BOSH Lite, do the following:

1. Target your BOSH Lite installation.

   bosh alias-env lite -e 192.168.50.4

2. Upload the BOSH Lite stemcell.

   bosh -e lite upload-stemcell \
   https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent?v=3262.2

Step 2: Set Up the Kafka Example Service

To set up the Kafka example service, do the following:

1. Clone the Kafka example service into your workspace.

   git clone \
   https://github.com/pivotal-cf-experimental/kafka-example-service-release.git

2. In the kafka-example-service-release directory, create and upload the Kafka example service.

   cd kafka-example-service-release
   bosh create-release --name kafka-example-service

3. Upload the service to the BOSH director.

   bosh -e lite upload-release

Step 3: Set Up the Kafka Example Service Adapter

To set up the Kafka example service adapter, do the following:

1. Clone the Kafka example service adapter.

   git clone \
   https://github.com/pivotal-cf-experimental/kafka-example-service-adapter-release.git

2. Update service adapter dependencies.

   cd kafka-example-service-adapter-release
   git submodule update --init --recursive

3. Create the example service adapter.

   bosh create-release --name kafka-example-service-adapter

4. Upload the example service adapter to the BOSH director.

   bosh -e lite upload-release

Step 4: Set Up ODB

To set up ODB:

1. Download the on-demand service broker from BOSH.io Releases. To download, see the BOSH documentation.

2. Upload the on-demand-service-broker release.

   bosh -e lite upload-release on-demand-service-broker-X.Y.Z.tgz

   Where X.Y.Z is the ODB release version.
   
   

   For example:

   $ bosh -e lite upload-release on-demand-service-broker-0.21.1.tgz
   

Part 2: Create

This section describes how to create a BOSH deployment and a service broker on PCF Dev.

Step 1: Create a BOSH Deployment

To create a BOSH Lite deployment, do the following:

1. Create a new directory in your workspace and a cloud_config.yml for the BOSH Lite Director. For example:

   vm_types:
   - name: container
     cloud_properties: {}
   
   networks:
   - name: kafka
     type: manual
     subnets:
     - range: 10.244.1.0/24
       gateway: 10.244.1.1
       az: lite
       cloud_properties: {}
   
   disk_types:
   - name: ten
     disk_size: 10_000
     cloud_properties: {}
   
   azs:
   - name: lite
     cloud_properties: {}
   
   compilation:
     workers: 2
     reuse_compilation_vms: true
     network: kafka
     az: lite
     cloud_properties: {}
   

2. Update the BOSH Lite cloud config using the deployment manifest.

   bosh -e lite update-cloud-config cloud\_config.yml

3. Record the URL and UUID of your BOSH Lite director.

   bosh environment

   See the following example output:

   $ bosh environment
   Config
                /Users/pivotal/.bosh_config
   
   Director
     Name       Bosh Lite Director
     URL        https://192.168.50.4:25555
     Version    1.3215.0 (00000000)
     User       admin
     UUID       17a45148-1d00-43bc-af28-9882e5a6535a
     CPI        warden\_cpi
     dns        disabled
     compiled\_package\_cache enabled (provider: local)
     snapshots  disabled
   

4. Create a BOSH Lite deployment manifest in a file called deployment_manifest.yml using the following as a base:

   name: kafka-on-demand-broker
   
   director_uuid: BOSH-LITE-UUID
   
   releases:
   - name: &broker-release on-demand-service-broker
     version: latest
   - name: &service-adapter-release kafka-example-service-adapter
     version: latest
   - name: &service-release kafka-example-service
     version: latest
   
   stemcells:
   - alias: trusty
     os: ubuntu-trusty
     version: STEMCELL-VERSION
   
   instance_groups:
   - name: broker
     instances: 1
     vm_type: container
     persistent_disk_type: ten
     stemcell: trusty
     azs: [lite]
     networks:
     - name: kafka
     jobs:
     - name: kafka-service-adapter
       release: *service-adapter-release
     - name: admin_tools
       release: *service-release
     - name: broker
       release: *broker-release
       properties:
         port: 8080
         username: broker    # or replace with your own
         password: password  # or replace with your own
         disable_ssl_cert_verification: true
         bosh:
           url: BOSH-LITE-URL
           authentication:
             basic:
               username: admin
               password: admin
         cf:
           url: https://api.PCF-DEV-DOMAIN
           authentication:
             url: https://uaa.PCF-DEV-DOMAIN
             user_credentials:
               username: admin
               password: admin
         service_adapter:
           path: /var/vcap/packages/odb-service-adapter/bin/service-adapter
         service_deployment:
           releases:
           - name: *service-release
             version: SERVICE-RELEASE-VERSION
             jobs: [kafka_server, zookeeper_server]
           stemcells:
             - os: ubuntu-trusty
               version: STEMCELL-VERSION
         service_catalog:
           id: D94A086D-203D-4966-A6F1-60A9E2300F72
           service_name: kafka-service-with-odb
           service_description: Kafka Service
           bindable: true
           plan_updatable: true
           tags: [kafka]
           plans:
           - name: small
             plan_id: 11789210-D743-4C65-9D38-C80B29F4D9C8
             description: A Kafka deployment with a single instance of each job and persistent disk
             instance_groups:
             - name: kafka_server
               vm_type: container
               instances: 1
               persistent_disk_type: ten
               azs: [lite]
               networks: [kafka]
             - name: zookeeper_server
               vm_type: container
               instances: 1
               persistent_disk_type: ten
               azs: [lite]
               networks: [kafka]
             properties:
               auto_create_topics: true
               default_replication_factor: 1
   
   update:
     canaries: 1
     canary_watch_time: 30000-180000
     update_watch_time: 30000-180000
     max_in_flight: 4
   

   Where:

   • BOSH-LITE-UUID is the UUID value you recorded in the BOSH environment step above.
   • BOSH-LITE-URL is the URL value you recorded in the BOSH environment step above.
   • PCF-DEV-DOMAIN is the PCF Dev domain you recorded in the Prerequisites above.

5. Deploy the broker.

   bosh -e lite -d kafka-on-demand-broker deployment\_manifest.yml

6. Record the IP address of the deployed broker.

   bosh -e lite -d kafka-on-demand-broker instances

   See the following example output:

   $ Using environment 'lite' as user 'admin' (openid, bosh.admin)
   
   Task 54727. Done
   
   Deployment 'redis-on-demand-broker-dev2'
   
   Instance                                     Process State  AZ  IPs
   broker/84294753-84b9-4be1-a338-37c1f3e71919  running        z1  10.244.1.2
   
   1 instances
   
   Succeeded
   

Step 2: Create a Service Broker on PCF Dev

To create a service broker on PCF Dev, do the following:

1. Create a service broker on PCF Dev and enable access to its service offering.

   cf create-service-broker kafka-broker USERNAME PASSWORD http://BROKER-IP:8080

   Where:

   • USERNAME and PASSWORD are the broker’s credentials set under properties in the broker job.
   • BROKER-IP is the value obtained in the step above. See the last step in Create a BOSH Deployment.

   For example:

   $ cf create-service-broker kafka-broker broker password http://10.244.1.2:8080
   

2. Enable access to the broker’s service plans.

   cf enable-service-access kafka-service-with-odb

3. View the broker-offered services in the Marketplace.

   cf marketplace

   See the following example output:

   Getting services from Marketplace in org pcfdev-org / space pcfdev-space as admin...
   OK
   
   service                  plans        description
   kafka-service-with-odb   small        Kafka Service
   p-mysql                  512mb, 1gb   MySQL databases on demand
   p-rabbitmq               standard     RabbitMQ is a robust and scalable high-performance multi-protocol messaging broker.
   p-redis                  shared-vm    Redis service to provide a key-value store
   

4. Create a service instance using the Kafka on-demand broker.

   cf create-service kafka-service-with-odb small k1

Part 3: Verify and Use

To verify and use your on-demand service, do the following:

Step 1: Verify Your BOSH Deployment and On-Demand Service

1. Check the status of your service.

   cf service k1

   See the state change from create in progress to create succeeded.

2. Verify that the on-demand service is provisioned in the BOSH deployment.

   bosh -e lite deployments

   See the following example output:

   Name                                                    Release(s)                              Stemcell(s)                                        Cloud Config
   kafka-on-demand-broker                                  kafka-example-service-adapter/0+dev.2   bosh-warden-boshlite-ubuntu-trusty-go\_agent/3262.2   latest
                                                           on-demand-service-broker/0.2.0+dev.1
   service-instance_2715262c-8564-4cd9-b629-0ae99e6aa4b9   kafka-example-service/0+dev.2           bosh-warden-boshlite-ubuntu-trusty-go\_agent/3262.2   latest
   

   This example shows that the service instance is provisioned and the service releases are specified in the ODB deployment manifest.

Step 2: Use Your On-Demand Service

To use the service instance that you created, do the following:

1. Clone the Kafka example app.

   git clone https://github.com/pivotal-cf-experimental/kafka-example-app.git

2. Push the app.

   cd kafka-example-app
   cf push --no-start

3. Bind the app to your service instance.

   cf bind-service kafka-example-app k1

4. Start the app.

   cf start kafka-example-app

Step 3: Read and write to your Service Instance

Now the app runs at https:/kafka-example-app.PCF-DEV-DOMAIN. You can use it to read and write to your on-demand Kafka service instance.

For example:

• To write data, run the following.

  curl -XPOST http://kafka-example-app.PCF-DEV-DOMAIN/queues/my-queue -d SOME-DATA
  

• To read data, run the following.

  curl http://kafka-example-app.PCF-DEV-DOMAIN/queues/my-queue