Native Kubernetes Templates

VMware Cloud Director Container Service Extension supports Native Kubernetes cluster development from multiple Kubernetes templates. This section details key functions you can perform as a cloud administrator in Native Kubernetes template management.

Each template name is uniquely constructed based on the type of guest OS, Kubernetes version, and the weave software version. You can find the definitions of different templates in an official location hosted at a remote repository URL. The VMware Cloud Director Container Service Extension sample config file, out of the box, points to the official location of the templates definitions. The remote repository is officially managed by maintainers of the VMware Cloud Director Container Service Extension project.

Create Kubernetes Templates

In this section, as a cloud administrator you can learn how to create Kubernetes templates during or after VMware Cloud Director Container Service Extension server installation.

During the VMware Cloud Director Container Service Extension server installation, Kubernetes templates are created from the template definitions that are available at the remote repository URL that is specified in the configuration file.

To choose Kubernetes templates after the VMware Cloud Director Container Service Extension server installation, use the --skip-template-creation command during the installation. When you use this command, VMware Cloud Director Container Service Extension does not create any Kubernetes templates during the server installation. Once VMware Cloud Director Container Service Extension server installation is complete, use the following command to create selective Kubernetes templates:

cse template list
cse template install TEMPLATE_NAME TEMPLATE_REVISION

Using Kubernetes Templates

This sections details how to configure and change Kubernetes templates.

When you start the VMware Cloud Director Container Service Extension server, it is necessary to specify a template name and revision when a user deploys a native cluster.

Note: From VMware Cloud Director Container Service Extension 3.1.1, the default template no longer exists. It is necessary to specify template name and revision while deploying a native cluster in VMware Cloud Director Container Service Extension.

Once the VMware Cloud Director Container Service Extension server installation is complete, tenant users can use the following command in the specification file to change templates as they deploy native clusters:

vcd cse cluster apply

Update Kubernetes Templates

This sections details how you can implement updates to Kubernetes templates as a service provider administrator.

When updates to OS versions, Kubernetes major or minor versions, or Weave major or minor versions are available, you can access new templates. Revised templates are also available with updated Kubernetes patch versions. This is due to a change to the revision of existing templates.

Use the following command to refresh existing templates with revised versions or install new templates:

cse template list --display diff
cse template install TEMPLATE_NAME TEMPLATE_REVISION        

Note: The refreshed templates do not impact existing Kubernetes clusters in the environment.

Note: It is recommended to restart the VMware Cloud Director Container Service Extension server before you update templates.

Restrict Kubernetes Templates for Tenant Users

This section details how to restrict Kubernetes templates for tenants between different versions of VMware Cloud Director Container Service Extension and VMware Cloud Director.

VMware Cloud Director Container Service Extension 3.1 with VMware Cloud Director 10.3, VMware Cloud Director 10.2 running in non legacy mode

Starting VMware Cloud Director Container Service Extension 3.0 with VMware Cloud Director 10.2, Kubernetes templates are restricted for use by default.

When VMware Cloud Director Container Service Extension 3.1 connects to VMware Cloud Director 10.2 and newer versions, cse install or cse upgrade commands restrict native template usage by default. The provider has to explicitly allow organizational virtual data centers to host native deployments, by running the command: vcd cse ovdc enable.

VMware Cloud Director Container Service Extension 3.1 leverages VMware Cloud Director feature of placement policies to restrict native Kubernetes deployments to specific organization virtual data centers. During VMware Cloud Director Container Service Extension install or upgrade, it creates a provider VMware Cloud Director Container Service Extension level placement policy cse—-native and tags the native templates with the same. In effect, you can represent native clusters from these tagged templates, only onto organization virtual data centers (ovdc) that have the corresponding placement policy published.

Note: VMware Cloud Director Container Service Extension 3.1.4 is compatible with VMware Cloud Director 10.4.

Table 1. Provider and tenant commands

User type	Command	Description
Provider	cse install or cse upgrade	Creates native placement policy cse—-native and tags the relevant templates with the same placement policy. On running cse upgrade on older environments with template rules, VMware Cloud Director Container Service Extension 3.1 automatically adopts the new template restriction mechanism. For more information, refer CSE 3.1 upgrade command.
Provider	vcd cse ovdc enable	This command publishes the native placement policy on to the chosen ovdc.
Tenant	vcd cse cluster apply	During the cluster creation, VMware Cloud Director internally validates the ovdc eligibility to host the cluster VMs instantiated from the native templates, by checking if the template’s placement policy is published onto the ovdc or not.

VMware Cloud Director Container Service Extension 3.1 with VMware Cloud Director 10.1

By default, Kubernetes templates are not restricted for use. All tenant users have access to all the Kubernetes templates to deploy Kubernetes clusters, as long as they have sufficient permissions to interact with VMware Cloud Director Container Service Extension. However, starting from VMware Cloud Director Container Service Extension 2.5, cloud administrators have the option to selectively restrict Kubernetes templates from being used by tenants in order to prohibit them from deploying Kubernetes clusters.

This is accomplished with the use of VDC Compute Policies feature of VMware Cloud Director 10.0. VMware Cloud Director Container Service Extension 2.5 offers the capability to service providers to tag selected templates and organization virtual data centers with compute policy, which restricts Kubernetes cluster deployments from tagged templates to only tagged organization virtual data centers.

As a service provider administrator, you can perform the following actions:

• Activate restriction on Kubernetes Templates:
  • You can activate the restriction on Kubernetes templates by leveraging the template_rules section in the VMware Cloud Director Container Service Extension configuration file. Service providers can mark Kubernetes templates as protected by tagging them with a VdcComputePolicy. To do so, it is necessary for service providers to define a template rule in the template_rules section, whose target is the template to protect, and as action a value must be specified for the key compute_policy.
    Service provider administrators select the name of the compute policy per their choice, and VMware Cloud Director Container Service Extension creates that compute policy in VMware Cloud Director, if it is not present. During VMware Cloud Director Container Service Extension server startup, the template rule Photon Template Rule is processed and the defined Kubernetes template is tagged with the compute policy. At this point, the Kubernetes template is restricted from further use, until tenant organization VDCs are enabled with matching compute policy to permit Kubernetes cluster deployments.

    template_rules:
    - name: Photon Template Rule
      target:
        name: photon-v2_k8-1.12_weave-2.3.0
        revision: 1
      action:
        compute_policy: "Photon Template Policy"

• Grant tenants access to Kubernetes Templates:
  • You can select tenants to grant access of certain Kubernetes Templates based cluster deployments. You can activate selected tenants’ organization VDCs with the same compute policy as present on the Kubernetes Template. To perform this task, use the following command:

    vcd cse ovdc compute-policy add ORG_NAME OVDC_NAME POLICY_NAME

• Revoke Permission to use Kubernetes Templates from tenants:
  • Permission to use a protected template can be revoked at any time from the tenant, through the following command. If there are Kubernetes clusters in that organization VDC, use -f/--force flag to force the operation. The clusters remain deployed, and switch to System Default compute policy.

    vcd cse ovdc compute-policy remove ORG_NAME OVDC_NAME POLICY_NAME

• Remove restriction from Kubernetes templates:
  • In order to remove the restriction from Kubernetes templates, you can delete the template rule from the configuration file and restart the VMware Cloud Director Container Service Extension server. Alternatively, you can specify an empty policy name in the concerned rule.

    template_rules:
    - name: Rule1
      target:
        name: out_of_box_protected_tempalte
        revision: 1
      action:
        compute_policy: ""

Source .ova Files for Kubernetes Templates

The following table lists URLs of the OVA files that form the base for the Kubernetes templates.

Table 2. Source .ova Files for Kubernetes Templates

OS	OVA name	URL
Photon OS 2.0 GA	photon-custom-hw11-2.0-304b817.ova	http://dl.bintray.com/vmware/photon/2.0/GA/ova/photon-custom-hw11-2.0-304b817.ova SHA256: cb51e4b6d899c3588f961e73282709a0d054bb421787e140a1d80c24d4fd89e1
Ubuntu 16.04.4 LTS	ubuntu-16.04-server-cloudimg-amd64.ova	https://cloud-images.ubuntu.com/releases/xenial/release-20180418/ubuntu-16.04-server-cloudimg-amd64.ova SHA256: 3c1bec8e2770af5b9b0462e20b7b24633666feedff43c099a6fb1330fcc869a9