This topic describes how to configure Kubernetes clusters with BOSH VM extensions using VMware Tanzu Kubernetes Grid Integrated Edition (TKGI).
BOSH VM extensions are VM configurations stored in the BOSH cloud config. BOSH VM extensions allow you to specify IaaS-specific configurations for your VMs such as custom security group and load balancer configurations.
TKGI supports configuring Kubernetes clusters using BOSH VM extensions. You can use BOSH VM extensions to configure TKGI-provisioned Linux and Windows clusters.
For information about BOSH VM Extensions, see VM Extensions Block in Usage in the Cloud Foundry BOSH documentation.
Warning: Configure VM Extensions only if you are already familiar with BOSH VM Extensions. If you use VM extensions, you might accidentally override more settings than you intend. For example, if you use a VM extension to add tags, the default tags are removed from all instance groups.
To create a new Kubernetes cluster configured with VM extensions:
To create a cluster using a VM extensions file:
tkgi create-cluster CLUSTER-NAME --config-file CONFIG-FILENAME
Where:
CLUSTER-NAME is the name of the cluster to create.CONFIG-FILENAME is the name of the VM extension configuration file created above.To configure an existing Kubernetes cluster with VM extensions:
To modify a cluster using a VM extensions file:
tkgi update-cluster CLUSTER-NAME --config-file CONFIG-FILENAME
Where:
CLUSTER-NAME is the name of the cluster to modify.CONFIG-FILENAME is the name of the VM extension configuration file created above.To remove a VM extensions instance group from a cluster:
Create a VM extension configuration file containing the following:
{
"name":"INSTANCE-GROUP",
"vm_extension":{}
}
Where INSTANCE-GROUP is the name of the instance group VM extensions to be removed. Specify either master, worker, windows-worker or a node pool VM extension instance group name.
To remove the VM extensions instance group using the CLI:
tkgi update-cluster CLUSTER-NAME --config-file CONFIG-FILENAME
Where:
CLUSTER-NAME is the name of the cluster to remove VM extensions from.CONFIG-FILENAME is the name of the VM extension configuration file created above.To create a VM extensions configuration file in JSON format:
Create a new configuration file containing the following content:
Linux Cluster VM Extension Configuration File Content
{
"instance_groups":[
{
"name":"master",
"vm_extension":{
VM-EXTENSIONS
"vmx_options": {
VMX-OPTIONS
"disk.enableUUID": "1"
}
}
},
{
"name":"worker",
"vm_extension":{
VM-EXTENSIONS
"vmx_options": {
VMX-OPTIONS
"disk.enableUUID": "1"
}
}
},
{
"name":"NODE-POOL-NAME",
"vm_extension":{
VM-EXTENSIONS
"vmx_options": {
VMX-OPTIONS
"disk.enableUUID": "1"
}
}
}
]
}
Where:
VM-EXTENSIONS are the custom BOSH VM extensions settings to apply to the Kubernetes cluster.VMX-OPTIONS are the custom BOSH VMX options settings to apply to the Kubernetes cluster.NODE-POOL-NAME is the name of the node-pool, a node-pool specified in the cluster’s compute profile. For Linux clusters the node pool name must be prefixed worker-. For example: worker-tiny-1.Windows Worker VM Extension Configuration File Content
{
"instance_groups":[
{
"name":"master",
"vm_extension":{
VM-EXTENSIONS
"vmx_options": {
VMX-OPTIONS
"disk.enableUUID": "1"
}
}
},
{
"name":"windows-worker",
"vm_extension":{
VM-EXTENSIONS
"vmx_options": {
VMX-OPTIONS
"disk.enableUUID": "1"
}
}
},
{
"name":"NODE-POOL-NAME",
"vm_extension":{
VM-EXTENSIONS
"vmx_options": {
VMX-OPTIONS
"disk.enableUUID": "1"
}
}
}
]
}
Where:
VM-EXTENSIONS are the custom BOSH VM extensions settings to apply to the Kubernetes cluster.VMX-OPTIONS are the custom BOSH VMX options settings to apply to the Kubernetes cluster.NODE-POOL-NAME is the instance group name for the node-pool VM extensions. For Windows Workers the node pool instance group name must be prefixed windows-worker-. For example: windows-worker-tiny-1.Replace the VM-EXTENSIONS and VMX-OPTIONS of each instance group with the desired BOSH VM extensions and VMX options.
Warning: Volumes will not attach to your nodes if you do not include the disk.enableUUID vmx_options parameter in your configuration.
For example:
{
"instance_groups":[
{
"name":"master",
"vm_extension":{
"cpu_hot_add_enabled": "true",
"vmx_options": {
"disk.enableUUID": "1"
}
}
},
{
"name":"windows-worker",
"vm_extension":{
"cpu_hot_add_enabled": "true",
"vmx_options": {
"disk.enableUUID": "1"
}
}
},
{
"name":"windows-worker-tiny-1",
"vm_extension":{
"cpu_hot_add_enabled": "true",
"vmx_options": {
"ctkEnabled": "TRUE",
"disk.enableUUID": "1"
}
}
}
]
}
The supported BOSH VM extensions are specific to each IaaS. For the names of the cloud properties you can use in your VM extension configurations, see the following topics in Usage in the Cloud Foundry BOSH documentation:
* For AWS, see: VM Types / VM extensions.
* For Azure, see: VM Types / VM extensions.
* For GCP, see: VM Types / VM extensions.
* For vSphere, see: VM Types / VM extensions.
