You can customize the Tanzu Kubernetes Grid Service with global settings for key features, including the container network interface (CNI), proxy server, and TLS certificates. Be aware of trade-offs and considerations when implementing global versus per-cluster functionality.
TkgServiceConfiguration specification applies to all
Tanzu Kubernetes clusters provisioned by that service. If a rolling update is initiated, either manually or by upgrade, clusters are updated by the changed service spec.
TkgServiceConfiguration Specification
TkgServiceConfiguration specification provides fields for configuring the
Tanzu Kubernetes Grid Service instance.
apiVersion: run.tanzu.vmware.com/v1alpha1
kind: TkgServiceConfiguration
metadata:
name: tkg-service-configuration-example
spec:
defaultCNI: <antrea or calico>
proxy:
httpProxy: http://<user>:<pwd>@<ip>:<port>
httpsProxy: http://<user>:<pwd>@<ip>:<port>
noProxy: [<array of CIDRs to not proxy>]
trust:
additionalTrustedCAs:
- name: <first-cert-name>
data: <base64-encoded string of a PEM encoded public cert 1>
- name: <second-cert-name>
data: <base64-encoded string of a PEM encoded public cert 2>
TkgServiceConfiguration Specification Parameters
TkgServiceConfiguration specification parameters. For examples, see
Examples for Configuring the Tanzu Kubernetes Grid Service v1alpha1 API..
| Field | Value | Description |
|---|---|---|
defaultCNI |
antrea or calico |
Default CNI for clusters to use. The default is antrea. The other supported CNI is calico. |
proxy |
Section marker for proxy parameters. |
The proxy parameters are httpProxy, httpsProxy, and noProxy. All parameters are required. If any proxy parameter is missing, you cannot create Tanzu Kubernetes clusters. |
httpProxy |
URI in the form http://<user>:<pwd>@<ip>:<port> |
Does not allow the https protocol. If https is used, you cannot create Tanzu Kubernetes clusters. |
httpsProxy |
URI in the form http://<user>:<pwd>@<ip>:<port> |
Does not allow the https protocol. If https is used, you cannot create Tanzu Kubernetes clusters. |
noProxy |
Array of CIDR blocks to not proxy, for example: Get the required values from the Workload Network on the Supervisor Cluster: Refer to the image below for what values to include in the |
You must not proxy the subnets used by the Workload Network on the Supervisor Cluster for Pods, Ingress, and Egress. You do not need to include the Services CIDR from the Supervisor Cluster in the The endpoints The Pod and Service CIDRs for Tanzu Kubernetes clusters are automatically not proxied. You do not need to add them to the |
trust |
Section marker for trust parameters. |
Accepts no data. |
additionalTrustedCAs |
Accepts an array of certificates with name and data for each. |
Accepts no data. |
name |
String | The name of the TLS certificate. |
data |
String | The base64-encoded string of a PEM encoded public certificate. |
Get the required noProxy values from the Workload Network on the Supervisor Cluster as shown in the image.
When To Use Global or Per-Cluster Configuration Options
The TkgServiceConfiguration is a global specification that impacts all Tanzu Kubernetes clusters provisioned by the Tanzu Kubernetes Grid Service instance.
TkgServiceConfiguration specification, be aware of the per-cluster alternatives that might satisfy your use case instead of a global configuration.
| Setting | Global Option | Per-Cluster Option |
|---|---|---|
| Default CNI | Edit the TkgServiceConfiguration spec. See Examples for Configuring the Tanzu Kubernetes Grid Service v1alpha1 API. |
Specify the CNI in the cluster specification. For example, Antrea is the default CNI. To use Calico, specify it in the cluster YAML. See Examples for Provisioning Tanzu Kubernetes Clusters Using the Tanzu Kubernetes Grid Service v1alpha1 API |
| Proxy Server | Edit the TkgServiceConfiguration spec. See Examples for Configuring the Tanzu Kubernetes Grid Service v1alpha1 API. |
Include the proxy server configuration parameters in the cluster spec. See Examples for Provisioning Tanzu Kubernetes Clusters Using the Tanzu Kubernetes Grid Service v1alpha1 API. |
| Trust Certificates | Edit the TkgServiceConfiguration spec. There are two use cases: configuring an external container registry and certificate-based proxy configuration. See Examples for Configuring the Tanzu Kubernetes Grid Service v1alpha1 API |
Yes, you can include custom certificates on a per-cluster basis or override the globally-set trust settings in the cluster specification. See Examples for Provisioning Tanzu Kubernetes Clusters Using the Tanzu Kubernetes Grid Service v1alpha1 API. |
TkgServiceConfiguration, that proxy information is propagated to the cluster manifest after the initial deployment of the cluster. The global proxy configuration is added to the cluster manifest only if there is no proxy configuration fields present when creating the cluster. In other words, per-cluster configuration takes precedence and will overwrite a global proxy configuration. For more information, see
Configuration Parameters for the Tanzu Kubernetes Grid Service v1alpha1 API.
Before editing the TkgServiceConfiguration specification, be aware of the ramifications of applying the setting at the global level.
| Field | Applied | Impact on Existing Clusters If Added/Changed | Per-Cluster Overriding on Cluster Creation | Per-Cluster Overriding on Cluster Update |
|---|---|---|---|---|
defaultCNI |
Globally | None | Yes, you can override the global setting on cluster creation | No, you cannot change the CNI for an existing cluster; if you used the globally set default CNI on cluster creation, it cannot be changed |
proxy |
Globally | None | Yes, you can override the global setting on cluster creation | Yes, with U2+, you can override the global setting on cluster update |
trust |
Globally | None | Yes, you can override the global setting on cluster creation | Yes, with U2+, you can override the global setting on cluster update |
Propagating Global Configuration Changes to Existing Clusters
Settings made at the global level in the TkgServiceConfiguration may not be automatically propagated to existing clusters. For example, if you make changes to either the proxy or the trust settings in the TkgServiceConfiguration, such changes may not affect clusters that are already provisioned.
To manually propagate a global change to an existing cluster, you must patch the Tanzu Kubernetes cluster to make the cluster inherit the changes made to the TkgServiceConfiguration.
kubectl patch tkc <CLUSTER_NAME> -n <NAMESPACE> --type merge -p "{\"spec\":{\"settings\":{\"network\":{\"proxy\": null}}}}"
kubectl patch tkc <CLUSTER_NAME> -n <NAMESPACE> --type merge -p "{\"spec\":{\"settings\":{\"network\":{\"trust\": null}}}}"
