This topic describes how VMware Tanzu Kubernetes Grid Integrated Edition administrators and cluster managers create and use Kubernetes profiles for Kubernetes clusters provisioned by Tanzu Kubernetes Grid Integrated Edition.
This topic also lists verified use cases for Kubernetes profiles on Tanzu Kubernetes Grid Integrated Edition.
Kubernetes profiles enable cluster administrators and cluster managers to customize Kubernetes component settings for any clusters that they provision.
To use Kubernetes profiles, Tanzu Kubernetes Grid Integrated Edition users:
kube-apiserver
on the control plane or the kubelet
on each node.Verified uses for Kubernetes profiles include encrypting secrets in an etcd database, adding an OIDC provider, and using a ResourceQuota
admission control plugin.
Users with the pks.clusters.admin
or pks.clusters.manage
roles can create and use Kubernetes profiles.
If user with the pks.clusters.admin-read-only
role attempts to create a Kubernetes profile, they see the following error:
You do not have enough privileges to perform this action. Please contact the TKGI administrator.
A Kubernetes profile configures settings for Kubernetes components in two JSON code blocks, customizations
and experimental_customizations
. See Kubernetes Profile Format for details. The code blocks differ as follows:
customizations
block
tkgi create-k8s-profile
on a profile with invalid configurations in customizations
, the command returns an error.experimental_customizations
block
Warning: Experimental customizations are not validated or supported.
In the TKGI CLI, all commands that include k8s-profile
are aliased to also use kubernetes-profile
. For example, the tkgi k8s-profiles
and tkgi kubernetes-profiles
commands are equivalent.
For brevity, this documentation uses the k8s-
versions.
To create a Kubernetes profile in Tanzu Kubernetes Grid Integrated Edition, you:
Define a Kubernetes profile in a JSON configuration file, following the Kubernetes Profile Format below.
Use the TKGI CLI to define the Kubernetes profile within Tanzu Kubernetes Grid Integrated Edition, as described in The create-network-profile Command, below.
To create a Kubernetes profile, you must first define it as a JSON file that specifies network parameters, listed in Kubernetes Profile Parameters below.
Here is the basic structure of a Kubernetes profile.
{
"name": "my-profile-1",
"description": "My profile description",
"customizations": [
{
"component": "k8s-component-name",
"arguments": {
"key": "value,value,value"
},
"file-arguments": {
"key": "local file path"
}
}
],
"experimental_customizations": [
{
}
]
}
Note: This example Kubernetes profile is for illustration purposes only. It is not intended to be used as a template for Kubernetes profile definition.
The Kubernetes profile JSON can include the following parameters:
Parameter | Type | Description |
---|---|---|
name * |
String | Name of the Kubernetes profile. |
description |
String | Description of the Kubernetes profile. |
customizations * |
Map | A block that defines supported, validated K8s options using the component , arguments ,and file-arguments parameters below. |
experimental_ |
Map | A block that defines unsupported, unvalidated K8s options using the component , arguments ,and file-arguments parameters below. |
component * |
String | The name of a K8s component, e.g. kubelet , kube-apiserver . |
arguments * |
Map | Parameters for each component, as one or more Key:Value pairs. Multiple values must be separated by commas, without spaces. |
file-arguments * |
Map | Parameters whose values are stored as files on the local machine. |
*Parameters marked with an *
are mandatory.
Note: If you specify the same parameter in both customizations
and experimental_customizations
, the one in customization
takes precedence.
After defining a Kubernetes profile in a JSON file as described in Kubernetes Profile Format, a cluster administrator or manager creates the Kubernetes profile by running the following TKGI CLI command:
tkgi create-k8s-profile PATH-TO-YOUR-KUBERNETES-PROFILE-CONFIGURATION
Where PATH-TO-YOUR-KUBERNETES-PROFILE-CONFIGURATION
is the path to the JSON file you created when defining the Kubernetes profile.
For example:
cat profile3-docs.json { “name”: “my-profile3”, “description”: “My profile description”, “customizations”: [ { “component”: “kube-apiserver”, “arguments”: { “service-node-port-range”: “30000-40000” } } ], “experimental_customizations”: [ { “component”: “kubelet”, “arguments”: { “maximum-dead-containers”: “1000”, “feature-gates”: “APIListChunking=true,AttachVolumeLimit=false” } } ] } user ~/workspace: ./tkgi create-k8s-profile k8s-profile3.json Kubernetes profile my-profile3 successfully created
Tanzu Kubernetes Grid Integrated Edition cluster administrators and managers can perform the following operations on Kubernetes profiles and the clusters that use them.
To list available Kubernetes profiles, run the following command:
tkgi k8s-profiles
For example:
$ tkgi k8s-profiles K8s-profile Owner Created Date Basic-k8s-profile Alana Tue, 05 Nov 2019 16:28:15 PST
The command output differs by user role: - pks.cluster.admin
see a list of Kubernetes profiles that all users created - pks.cluster.manage
see a list of only the Kubernetes profiles that they created
To delete a Kubernetes profile, run the following command:
tkgi delete-k8s-profile KUBERNETES-PROFILE-NAME
Where KUBERNETES-PROFILE-NAME
is the name of the Kubernetes profile you want to delete.
For example:
$ user ~/workspace: ./tkgi delete-k8s-profile my-profile3 Are you sure you want to delete the kubernetes profile my-profile3? (y/n): y Deletion of my-profile3 completed
Note: You cannot delete a Kubernetes profile that is in use. Before deleting a Kubernetes profile, you must disassociate it from all clusters or delete all clusters it is associated with.
Both pks.clusters.admin
and pks.clusters.manage
users can delete Kubernetes profiles. If a pks.clusters.admin-read-only
user attempts to delete a Kubernetes profile, they see the following error:
You do not have enough privileges to perform this action. Please contact the TKGI administrator.
To view details of a Kubernetes profile, run the following command:
tkgi k8s-profile KUBERNETES-PROFILE-NAME
Where KUBERNETES-PROFILE-NAME
is the name of the Kubernetes profile you want to view.
For example:
tkgi k8s-profile Basic-k8s-profile Name: Basic-k8s-profile Owner: Alana Created Date Tue, 05 Nov 2019 16:28:15 PST Description: Kubernetes profile for customer A …. <KEY> : <VALUE>
pks.cluster.admin
can view the details of any Kubernetes profile; users with the pks.cluster.manage
role can view details of Kubernetes profiles that they created.You can assign a Kubernetes profile to a Kubernetes cluster at the time of cluster creation.
To create an Tanzu Kubernetes Grid Integrated Edition-provisioned Kubernetes cluster with a Kubernetes profile, run the following command:
tkgi create-cluster CLUSTER-NAME --external-hostname HOSTNAME --plan PLAN-NAME --kubernetes-profile KUBERNETES-PROFILE-NAME
Where:
CLUSTER-NAME
is a unique name for your cluster. Note: Use only lowercase characters when naming your cluster if you manage your clusters with Tanzu Mission Control (TMC). Clusters with names that include an uppercase character cannot be attached to TMC.
HOSTNAME
is your external hostname used for accessing the Kubernetes API.PLAN-NAME
is the name of the Tanzu Kubernetes Grid Integrated Edition plan you want to use for your cluster.KUBERNETES-PROFILE-NAME
is the name of the Kubernetes profile you want to use for your cluster.TKGI supports changing the Kubernetes profile for an already created cluster. You can use this procedure to:
This is the procedure to change a cluster’s Kubernetes profile:
Do one of the following
Run the following command to update the cluster with the new Kubernetes profile:
tkgi update-cluster CLUSTER-NAME --kubernetes-profile NEW-KUBERNETES-PROFILE-NAME
Where:
CLUSTER-NAME
is the name of the existing Kubernetes clusterNEW-KUBERNETES-PROFILE-NAME
is the name of the new Kubernetes profile you want to apply to the cluster.Kubernetes profiles let you customize Kubernetes configuration parameters at the time of cluster creation. Use cases for Kubernetes profiles include:
Topic | Description |
---|---|
Encrypt Secrets in an etcd Database | Use an encryption provider to encrypt secrets in a cluster’s etcd database. |
Admission Control: ResourceQuota | Use the ResourceQuota admission control plugin to restrict incoming requests by resource usage. |
Set Service Node Port Range | Use service-node-port-range to specify an IP range for for NodePort services. |
Adding an OIDC Provider | Customize a cluster’s OIDC provider by deploying a dex connector or other OIDC provider to its pod. |
Restrict Request Header Names | Set requestheader-allowed-names for Apiserver client authentication. |
To create a Kubernetes profile that includes the ResourceQuota
admission control plugin:
Include the following customizations
in your profile configuration file:
"customizations": [
{
"component": "kube-apiserver",
"arguments": {
"enable-admission-plugins": PLUGINS-LIST
}
}
],
Where PLUGINS-LIST
is one of the following: * The string "ResourceQuota"
* A comma-delimited string list of validated plugins that includes ResourceQuota
.
For more information, see ResourceQuota in the Kubernetes documentation.
To create a Kubernetes profile that uses service-node-port-range
for NodePort
type services:
Include the following customizations
in your profile configuration file:
"customizations": [
{
"component": "kube-apiserver",
"arguments": {
"service-node-port-range": PORT-RANGE
}
}
],
Where PORT-RANGE
is A CIDR notation IP range from which to assign service cluster IPs, such as 30000-40000
.
If the specified PORT-RANGE
is not valid, the tkgi create-k8s-profile
command returns an error invalid value for service-node-port-range
.
For more information, see Type NodePort in the Kubernetes documentation.
To create a Kubernetes profile that uses requestheader-allowed-names
for Apiserver client authentication:
Include the following customizations
in your profile configuration file:
"customizations": [
{
"component": "kube-apiserver",
"arguments": {
"requestheader-allowed-names": COMMON-NAMES
}
}
],
Where COMMON-NAMES
is a string list of valid Common Name values in the signed client certificate, such as "cn1.com,c2.com"
.
For more information, see Kubernetes Apiserver Client Authentication in the Kubernetes documentation.