When adding Kubernetes components to an Automation Assembler cloud template, you can choose to add clusters or enable users to create namespaces in various configurations. Typically, this choice depends on your access control requirements, how you have configured your Kubernetes components, and your deployment requirements.
To add a Kubernetes component to a cloud template in Automation Assembler, select , click New, and then locate and expand the Kubernetes option on the left menu. Then, make the desired selection, either Cluster or KBS Namespace by dragging it to the canvas.
Adding a Kubernetes cluster that is associated with a project to a cloud template is the most straightforward method of making Kubernetes resources available to valid users. You can use tags on clusters to control where they are deployed just as you do with other Cloud Assembly resources. You can use tags to select a zone and a VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) plan during the allocation phase of cluster deployment.
Once you add a cluster in this way, it is automatically available to all valid users.
Cloud template examples
The first cloud template example shows a template for a simple Kubernetes deployment that is controlled by tagging. A Kubernetes zone was created with two deployment plans, configured on the New Kubernetes Zone page. In this case, a tag called placement:tag was added as a capability on the zone, and it was used to match the analogous constraint on the cloud template. If there were more than one zone configured with the tag, the one with the lowest priority number would be selected.
formatVersion: 1
inputs: {}
resources:
Cluster_provisioned_from_tag:
type: Cloud.K8S.Cluster
properties:
hostname: 109.129.209.125
constraints:
-tag: 'placement tag'
port: 7003
workers: 1
connectBy: hostname
The second cloud template examples shows how to set up a template with a variable called $(input.hostname) so that users can input the desired cluster hostname when requesting a deployment. Tags can also be used to select a zone and a TKGI plan during the resource allocation phase of cluster deployment.
formatVersion: 1
inputs:
hostname:
type: string
title: Cluster hostname
resources:
Cloud_K8S_Cluster_1:
type: Cloud.K8S.Cluster
properties:
hostname: ${input.hostname}
port: 8443
connectBy: hostname
workers: 1
If you want to use namespaces to mange cluster usage, you can set up a variable in the cloud template called name: ${input.name} to substitute for the namespace name which a user enters when requesting a deployment. For this sort of deployment, you would create a template something like the following example:
1 formatVersion: 1
2 inputs:
3 name:
4 type: string
5 title: "Namespace name"
6 resources:
7 Cloud_KBS_Namespace_1:
8 type: Cloud.K8S.Namespace
9 properties:
10 name: ${input.name}
Users can manage deployed clusters via kubeconfig files that are accessible from the page. Locate the card on the page for the desired cluster and click Kubeconfig.
Supervisor Namespaces in VMware Cloud Templates
The following is the schema for a basic Supervisor namespace in an Automation Assembler cloud template.
{
"title": "Supervisor namespace schema",
"description": "Request schema for provisioning of Supervisor namespace resource",
"type": "object",
"properties": {
"name": {
"title": "Name",
"description": "Alphabetic (a-z and 0-9) string with maximum length of 63 characters. The character ‘-’ is allowed anywhere except the first or last position of the identifier.",
"type": "string",
"pattern": "^.*\\$\\{.*\\}.*$|^((?!-)[a-z0-9-]{1,63}(?<!-))$",
"ignoreOnUpdate": true
},
"description": {
"title": "Description",
"description": "An optional description of this Supervisor namespace.",
"type": "string",
"ignoreOnUpdate": true
},
"content": {
"title": "Content",
"description": "Kubernetes Yaml Content",
"type": "string",
"maxLength": 65000
},
"constraints": {
"title": "Constraints",
"description": "To target the correct resources, blueprint constraints are matched against infrastructure capability tags. Constraints must include the key name. Options include value, negative [!], and hard or soft requirement.",
"type": "array",
"recreateOnUpdate": true,
"items": {
"type": "object",
"properties": {
"tag": {
"title": "Tag",
"description": "Constraint definition in syntax `[!]tag_key[:tag_value][:hard|:soft]` \nExamples:\n```\n!location:eu:hard\n location:us:soft\n!pci\n```",
"type": "string",
"recreateOnUpdate": true
}
}
}
},
"limits": {
"title": "Limits",
"description": "Defines namespace resource limits such as pods, services, etc.",
"type": "object",
"properties": {
"stateful_set_count": {
"title": "stateful_set_count",
"description": "This represents the new value for 'statefulSetCount' option which is the maximum number of StatefulSets in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"deployment_count": {
"title": "deployment_count",
"description": "This represents the new value for 'deploymentCount' option which is the maximum number of deployments in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"cpu_limit_default": {
"title": "cpu_limit_default",
"description": "This represents the new value for the default CPU limit (in Mhz) for containers in the pod. If specified, this limit should be at least 10 MHz.",
"type": "integer",
"recreateOnUpdate": false
},
"config_map_count": {
"title": "config_map_count",
"description": "This represents the new value for 'configMapCount' option which is the maximum number of ConfigMaps in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"pod_count": {
"title": "pod_count",
"description": "This represents the new value for 'podCount' option which is the maximum number of pods in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"job_count": {
"title": "job_count",
"description": "This represents the new value for 'jobCount' option which is the maximum number of jobs in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"secret_count": {
"title": "secret_count",
"description": "This represents the new value for 'secretCount' option which is the maximum number of secrets in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"cpu_limit": {
"title": "cpu_limit",
"description": "This represents the new value for 'limits.cpu' option which is equivalent to the maximum CPU limit (in MHz) across all pods in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"cpu_request_default": {
"title": "cpu_request_default",
"description": "This represents the new value for the default CPU request (in Mhz) for containers in the pod. If specified, this field should be at least 10 MHz.",
"type": "integer",
"recreateOnUpdate": false
},
"memory_limit_default": {
"title": "memory_limit_default",
"description": "This represents the new value for the default memory limit (in mebibytes) for containers in the pod.",
"type": "integer",
"recreateOnUpdate": false
},
"memory_limit": {
"title": "memory_limit",
"description": "This represents the new value for 'limits.memory' option which is equivalent to the maximum memory limit (in mebibytes) across all pods in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"memory_request_default": {
"title": "memory_request_default",
"description": "This represents the new value for the default memory request (in mebibytes) for containers in the pod.",
"type": "integer",
"recreateOnUpdate": false
},
"service_count": {
"title": "service_count",
"description": "This represents the new value for 'serviceCount' option which is the maximum number of services in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"replica_set_count": {
"title": "replica_set_count",
"description": "This represents the new value for 'replicaSetCount' option which is the maximum number of ReplicaSets in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"replication_controller_count": {
"title": "replication_controller_count",
"description": "This represents the new value for 'replicationControllerCount' option which is the maximum number of ReplicationControllers in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"storage_request_limit": {
"title": "storage_request_limit",
"description": "This represents the new value for 'requests.storage' which is the limit on storage requests (in mebibytes) across all persistent volume claims from pods in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"persistent_volume_claim_count": {
"title": "persistent_volume_claim_count",
"description": "This represents the new value for 'persistentVolumeClaimCount' option which is the maximum number of PersistentVolumeClaims in the namespace.",
"type": "integer",
"recreateOnUpdate": false
},
"daemon_set_count": {
"title": "daemon_set_count",
"description": "This represents the new value for 'daemonSetCount' option which is the maximum number of DaemonSets in the namespace.",
"type": "integer",
"recreateOnUpdate": false
}
},
"additionalProperties": false
},
"vm_classes": {
"title": "VM classes",
"description": "Defines set of Virtual Machine classes to be assigned to the namespace",
"type": "array",
"recreateOnUpdate": false,
"items": {
"type": "object",
"properties": {
"name": {
"title": "Name",
"description": "Name of the Virtual Machine class.",
"type": "string",
"recreateOnUpdate": false
}
}
}
},
"storage": {
"title": "Storage policies",
"description": "Defines set of storage profiles to be used to assign storage policies to the namespace.",
"type": "array",
"recreateOnUpdate": false,
"items": {
"type": "object",
"properties": {
"profile": {
"type": "object",
"title": "Storage profile",
"description": "Defines storage policies to be assigned to the namespace",
"recreateOnUpdate": false,
"properties": {
"constraints": {
"title": "Constraints",
"description": "To target the correct storage profiles, blueprint constraints are matched against storage profile capability tags.",
"type": "array",
"recreateOnUpdate": false,
"items": {
"type": "object",
"properties": {
"tag": {
"title": "Tag",
"description": "Constraint definition in syntax `[!]tag_key[:tag_value][:hard|:soft]` \nExamples:\n```\nlocation:eu:hard\n location:us:soft\n```",
"type": "string",
"recreateOnUpdate": false
}
}
},
"minItems":1
},
"limitMb": {
"title": "Limit",
"description": "The maximum amount of storage (in mebibytes) which can be utilized by the namespace for this storage policy. Optional. If unset, no limits are placed.",
"type": "integer"
}
},
"required": [
"constraints"
]
}
}
}
}
},
"required": [
"name"
]
}
VMware cloud templates support the use of limits with supervisor namespaces. Limits enable you to control resource usage for CPUs and memory as well as the maximum number of pods allowed in the namespace by deployed machines.
formatVersion: 1
inputs: {}
resources:
Cloud_SV_Namespace_1:
type: Cloud.SV.Namespace
properties:
name: '${env.deploymentName}'
limits:
- cpu_limit: 1000
cpu_request_default: 800
memory_limit: 2000
memory_limit_default: 1500
pod_count: 200
The following example shows how you could specify a storage policy using tags.
formatVersion: 1
inputs: {}
resources:
Cloud_SV_Namespace_1:
type: Cloud.SV.Namespace
properties:
name: 'ns-with-storage-policy'
description: 'sample'
storage:
- profile:
limitMb: 1000
constraints:
- tag: 'storage:fast'
- profile:
constraints:
- tag: 'storage:cheap'
Using arbitrary YAMLs with self-service namespace or cluster VCTs
As part of a cluster or namespace creation, you may want to run additional customizations. For example, you may want to add users (role/role binding) or create a pod security policy, or install agents. By using the YAML content property, you can define customized packages to provision on that cluster/namespace/supervisor namespace.
Each YAML content package associated with the content property must be separated with a triple dash (---). Also the content information must be a multi-line string. Refer to the following YAML example to see how content packages can be configured.
formatVersion: 1
inputs: {}
resources:
Cloud_Tanzu_Cluster_1:
type: Cloud.Tanzu.Cluster
properties:
name: ddonchev-tkc
plan: small
content: |-
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: psp:authenticated-from-yaml
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:authenticated
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: psp:vmware-system-privileged
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
# name must match the spec fields below, and be in the form: <plural>.<group>
name: crontabs.stable.example.com
spec:
# group name to use for REST API: /apis/<group>/<version>
group: stable.example.com
# list of versions supported by this CustomResourceDefinition
versions:
- name: v1
# Each version can be enabled/disabled by Served flag.
served: true
# One and only one version must be marked as the storage version.
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
cronSpec:
type: string
image:
type: string
replicas:
type: integer
# either Namespaced or Cluster
scope: Namespaced
names:
# plural name to be used in the URL: /apis/<group>/<version>/<plural>
plural: crontabs
# singular name to be used as an alias on the CLI and for display
singular: crontab
# kind is normally the CamelCased singular type. Your resource manifests use this.
kind: CronTab
# shortNames allow shorter string to match your resource on the CLI
shortNames:
- ct
The YAML defined in the content property also appears on the Properties tab for the deployment.
Automation Assembler can only create content resources in the scope of the resource being deployed. For example: if you provision a kubernetes namespace,Automation Assembler cannot create a deployment inside a different namespace. Users have the same rights as if they were using the kubeconfig with kubectl.
After the virtual machine is provisioned, an installation of the kubernetes objects inside the content property will begin. If one of the resources referenced in the YAML content property fails to provision, Automation Assembler will roll back and delete all the previous kubernetes objects from the resource and the deployment will have a Failed status. The resource will still be provisioned and visible. Also, you can still use day2 actions, including trying to apply the content again.
You can enhance the content property with inputs from the cloud template as shown in the following example.
formatVersion: 1
inputs: {}
resources:
Cloud_SV_Namespace_1:
type: Cloud.SV.Namespace
properties:
name: sv-namespace-with-vm-classes
vm_classes:
- name: best-effort-2xlarge
- name: best-effort-4xlarge
- name: best-effort-8xlarge
In addition, you can provision custom resources such as the TanzuKubernetesCluster. This would fail as a day 1 operation, because the supervisor namespace will not contain the required virtual machine classes and storage classes. When the virtual machine classes and storage classes are bound to the supervisor namespace you can create the TanzuKubernetesCluster (or another resource) using the day 2 action.
Note: You can provision a resource without content, and you will still be able to add kubernetes objects as YAML with the day 2 action.
The content that appears in the YAML property defines what is provisioned on the resource. When you edit this content, the following table shows the possible results:
| Action | Result |
|---|---|
| If you add a kubernetes object and submit. | The specified object is created on the resource. |
| If you remove a kubernetes object and submit. | The specified object is deleted from the resource. |
| If you modify a kubernetes object and submit. | The specified object is patched on the resource. |
It is important to clarify what actions are considered as a modification to the current object. For example: if you modify the namespace field of an object, then a new object is created instead of the old one being patched.
The uniqueness of a resource is defined by the following fields: apiVersion, kind, metadata.name, metadata.namespace
