Here you will learn how to configure the multiple platform support in VMware Spring Cloud® Data Flow for Kubernetes (SCDF for Kubernetes).
Configure multiple platforms
SCDF for Kubernetes can deploy streaming and batch applications to different Kubernetes namespaces or clusters. For information about scenarios that benefit from this approach, see the Spring Cloud Data Flow documentation about the Role of Multiple Platform Deployments.
To configure multiple platform support, you must edit the application.yaml file for the relevant SCDF for Kubernetes application. The Data Flow server deploys Spring Cloud Task applications, and the Skipper server deploys Spring Cloud Stream applications.
To configure one or more deployment platforms for Spring Cloud Task applications, you must edit the application.yaml file for the Data Flow application. In the spring.cloud.dataflow.task.platform.kubernetes section, the accounts element is a map, where a key is an account name and a value is the properties of the KubernetesDeployerProperties Java object. For a list of object properties, with descriptions, see the Deployer Properties section of the Spring Cloud Data Flow Reference Documentation.
The following example configures two accounts: a default account that targets the default namespace and has a memory limit of 1024Mi, and a highmemory account that targets the highmemory-namespace namespace and has a memory limit of 4096Mi.
spring:
cloud:
dataflow:
task:
platform:
kubernetes:
accounts:
default:
namespace: default
limits:
memory: 1024Mi
highmemory:
namespace: highmemory-namespace
limits:
memory: 4096Mi
SCDF for Kubernetes uses the Fabric8 Kubernetes client library to connect to Kubernetes. The KubernetesDeployerProperties object contains a property named fabric8, which exposes the Fabric8 client configuration properties. You can use this to configure connection properties, such as the Kubernetes master location and certificate information.
The following example configures an account that references another cluster, which contains GPU resources.
spring:
cloud:
dataflow:
task:
platform:
kubernetes:
accounts:
default:
namespace: default
limits:
memory: 1024Mi
gpuzone:
fabric8:
masterUrl: <k8s-master-api-url>
namespace: gpuzone-namespace
trustCerts: true
To configure one or more deployment platforms for Spring Cloud Stream applications, you must edit the application.yaml file for the Skipper application. Add similar configuration to the accounts map in the spring.cloud.dataflow.stream.platform.kubernetes section.
The following example configures two accounts that reference both a default namespace and a namespace where a Kafka broker is available. Because this configuration sets environment variables at the account level, all Spring Cloud Stream applications deployed to the kafkazone account will be configured to use the Kafka broker available in the kafka-namespace namespace.
spring:
cloud:
skipper:
server:
platform:
kubernetes:
accounts:
default:
namespace: default
limits:
memory: 1024Mi
cpu: 500m
readinessProbeDelay: 120
livenessProbeDelay: 90
kafkazone:
namespace: kafka-namespace
environmentVariables: 'SPRING_CLOUD_STREAM_KAFKA_BINDER_BROKERS=${KAFKA_SERVICE_HOST}:${KAFKA_SERVICE_PORT},SPRING_CLOUD_STREAM_KAFKA_BINDER_ZK_NODES=${KAFKA_ZOOKEEPER_SERVICE_HOST}:${KAFKA_ZOOKEEPER_SERVICE_PORT}'
limits:
memory: 2048Mi
cpu: 500m
readinessProbeDelay: 180
livenessProbeDelay: 120
Next: Install Spring Cloud Data Flow for Kubernetes
After completing the configuration steps, you can install SCDF for Kubernetes. For installation instructions, see Installing SCDF for Kubernetes.
