Installing VMware Spring Cloud® Data Flow for Kubernetes

Here you will learn how to install VMware Spring Cloud® Data Flow for Kubernetes (SCDF for Kubernetes).

Before beginning the installation process, review Configuring installation values to ensure that you have configured all required or recommended installation resources.

Create an image pull secret

Before installing SCDF for Kubernetes, you must create a Kubernetes secret containing credentials for the image registry that stores SCDF for Kubernetes container images. This secret will be used by SCDF for Kubernetes service accounts to pull images from the registry. Create the secret in the namespace where you will install SCDF for Kubernetes.

If you have not relocated the images, use credentials for the Tanzu Net registry. If you have relocated the images to another registry, use credentials for that registry.

To create the secret:

Set username and password environment variables. These variables should contain your login credentials for the registry that stores the application images.
```
$ registry=registry.packages.broadcom.com
$ username=<your user name>
$ password=<your password>
```

Run the kubectl create secret command, using the variables set in the previous step.

$ kubectl create secret \
docker-registry scdf-image-regcred \
--docker-server=${registry} \
--docker-username=${username} \
--docker-password=${password}

Note This secret is mounted by the Data Flow server. If you don't need credentials for relocated images, you must still create this secret with some dummy values.

Install on the development environment

Within the "Spring Cloud Data Flow for Kubernetes Legacy Installation" archive downloaded from the Broadcom Support portal, the bin directory contains an installation script named install-dev.sh. Given an existing Kubernetes cluster, the script automates installation steps for the following components for a development environment:

Database: PostgreSQL or MySQL
Broker: RabbitMQ or Kafka
Monitoring: Prometheus with Grafana
The Skipper server
The Data Flow server

Note This script installs all components in the namespace that is configured for the current context. The only exception are the monitoring config files in the `spring-cloud-data-flow/services/dev/monitoring` directory. They are hard coded to install in the `default` namespace and need to be manually changed for installtion in a different namespace.

Install the Spring Cloud Data Flow system

To install the components mentioned above, run the install-dev.sh script. This script has three option flags to specify database and broker plus whether monitoring should be included.

To list all available options use the -h or --help flag:

$ ./bin/install-dev.sh --help
Install for development help
Usage: ./bin/install-dev.sh [-d|--database <arg>] [-b|--broker <arg>] [-h|--help]
	-d, --database: The database to use -- postgresql (default) or mysql
	-b, --broker: The message broker to use -- rabbitmq (default) or kafka
	-m, --monitoring: The monitoring solution to install and enable -- none (default) or prometheus
	-o, --output-dir: The output directory for writing the config files that can be applied later
	-h, --help: Prints help

You can see that the defaults for the developer install is PostgreSQL for the database, RabbitMQ for the message broker and no monitoring (none). To override these defaults, provide the option flag with the alternate choice.

As an example, to install with MySQL, Kafka and Prometheus/Grafana enabled use the following command:

$ ./bin/install-dev.sh --database mysql --broker kafka --monitoring prometheus

Install ingress resource for the development environment

If you have an ingress controller and you want to use an ingress resource, run:

$ kubectl kustomize ./apps/ingress/kustomize/overlays/dev/ | kapp deploy -y -a ingress -f -

Accessing Grafana dashboard for the development environment

If you enabled monitoring for Prometheus and Grafana, then you can use port-forwarding to access the Grafana dashboard from youtr local browser.

Run the following command:

$ kubectl port-forward service/grafana 3000:3000

You can now access the dashboard at http://localhost:3000

Uninstall the development environment

To uninstall the components installed by the install-dev.sh script, run the uninstall-dev.sh script (also located in the bin directory):

$ ./bin/uninstall-dev.sh

To uninstall the Ingress resource, run the following command:

$ kapp delete -y -a ingress

Install on the production environment

Before installing SCDF for Kubernetes on the production environment, ensure that you have provisioned the database used by the Skipper and Data Flow servers and that you have properly configured the database settings in the Data Flow and Skipper applications' application.yaml files.

Note The following example commands pipe output to the kapp utility. If you wish, you can instead pipe the command output to a file for management in accordance with your organization's Kubernetes application deployment procedures.

To install the skipper application, run the following command:

$ kubectl kustomize ./apps/skipper/kustomize/overlays/production/ | kapp deploy -y -a skipper -f -

To install the data-flow application, run the following command:

$ kubectl kustomize ./apps/data-flow/kustomize/overlays/production/ | kapp deploy -y -a data-flow -f -

If you want to use an Ingress resource, run the following command:

$ kubectl kustomize ./apps/ingress/kustomize/overlays/production/ | kapp deploy -y -a ingress -f -

Next: Connect to Spring Cloud Data Flow for Kubernetes

For information about connecting to the newly-installed instance of SCDF for Kubernetes, see Connecting to SCDF for Kubernetes.