You can customize the TKG Service Configuration for TKG clusters provisioned using the v1alpha3 API, including the container network interface (CNI), proxy server, and TLS certificates.

About Customizing the TkgServiceConfiguration

You can configure global settings for Tanzu Kubernetes clusters by editing the TkgServiceConfiguration. This configuration lets you set the default CNI, add a global proxy server, and add one or more trusted TLS certificates.

Caution: Customizing the TkgServiceConfiguration is a global operation. Any change you make to the TkgServiceConfiguration object applies to all TKG clusters provisioned by that service. If a rolling update is initiated, either manually or by upgrade, clusters are updated by the changed service specification.
Note: Customizing the TkgServiceConfiguration applies to Tanzu Kubernetes clusters provisioned with the v1alpha3 API. It does not apply to Clusters provisioned with the v1beta1 API.

TkgServiceConfiguration Specification

The TkgServiceConfiguration specification provides fields for configuring the Tanzu Kubernetes Grid instance.
Important: A valid key name must consist only of alphanumeric characters, a dash (such as key-name), an underscore (such as KEY_NAME) or a dot (such as key.name). You cannot use a space in a key name.
apiVersion: run.tanzu.vmware.com/v1alpha3
kind: TkgServiceConfiguration
metadata:
  name: tkg-service-configuration-spec
spec:
  defaultCNI: string
  proxy:
    httpProxy: string
    httpsProxy: string
    noProxy: [string]
  trust:
    additionalTrustedCAs:
      - name: string
        data: string
  defaultNodeDrainTimeout: time

Annotated TkgServiceConfiguration Specification

The following YAML lists and described the configurable fields for each of the TkgServiceConfiguration specification parameters.
apiVersion: run.tanzu.vmware.com/v1alpha3
kind: TkgServiceConfiguration
#valid config key must consist of alphanumeric characters, '-', '_' or '.'
metadata:
  name: tkg-service-configuration-spec
spec:
  #defaultCNI is the default CNI for all Tanzu Kubernetes 
  #clusters to use unless overridden on a per-cluster basis 
  #supported values are antrea, calico, antrea-nsx-routed
  #defaults to antrea
  defaultCNI: string
  #proxy configures a proxy server to be used inside all 
  #clusters provisioned by this TKGS instance
  #if implemented all fields are required
  #if omitted no proxy is configured 
  proxy:
    #httpProxy is the proxy URI for HTTP connections 
    #to endpionts outside the clusters
    #takes the form http://<user>:<pwd>@<ip>:<port>
    httpProxy: string
    #httpsProxy is the proxy URI for HTTPS connections 
    #to endpoints outside the clusters
    #takes the frorm http://<user>:<pwd>@<ip>:<port>
    httpsProxy: string
    #noProxy is the list of destination domain names, domains, 
    #IP addresses, and other network CIDRs to exclude from proxying
    #must include from Workload Network: [Namespace Netowrk, Ingress, Egress]
    noProxy: [string]
  #trust configures additional trusted certificates 
  #for the clusters provisioned by this TKGS instance
  #if omitted no additional certificate is configured
  trust: 
    #additionalTrustedCAs are additional trusted certificates 
    #can be additional CAs or end certificates
    additionalTrustedCAs:
      #name is the name of the additional trusted certificate
      #must match the name used in the filename
      - name: string
        #data holds the contents of the additional trusted cert 
        #PEM Public Certificate data encoded as a base64 string
        data: string
  #defaultNodeDrainTimeout is the total amount of time the
  #controller spends draining a node; default is undefined
  #which is the value of 0, meaning the node is drained 
  #without any time limitations; note that `nodeDrainTimeout` 
  #is different from `kubectl drain --timeout`
  defaultNodeDrainTimeout: time
Figure 1. Required noProxy Values
Required noProxy Values
Note: If a global proxy is configured on the TkgServiceConfiguration, that proxy information is propagated to the cluster manifest after the initial deployment of the cluster. The global proxy configuration is added to the cluster manifest only if there is no proxy configuration fields present when creating the cluster. In other words, per-cluster configuration takes precedence and will overwrite a global proxy configuration.

Example TkgServiceConfiguration Specification

The following YAML lists and described the configurable fields for each of the TkgServiceConfiguration specification parameters.

apiVersion: run.tanzu.vmware.com/v1alpha3
kind: TkgServiceConfiguration
metadata:
  name: tkgserviceconfiguration_example
spec:
  defaultCNI: calico
  proxy:
    #supported format is `http://<user>:<pwd>@<ip>:<port>`
    httpProxy: http://admin:[email protected]:80
    httpsProxy: http://admin:[email protected]:80
    #noProxy vaulues are from Workload Network: [Namespace Network, Ingress, Egress]
    noProxy: [10.246.0.0/16,192.168.144.0/20,192.168.128.0/20]
  trust: 
    additionalTrustedCAs:
      #name is the name of the public cert
      - name: CompanyInternalCA-1
      #data is base64-encoded string of a PEM encoded public cert
        data: LS0tLS1C...LS0tCg==
        #where "..." is the middle section of the long base64 string
      - name: CompanyInternalCA-2
        data: MTLtMT1C...MT0tPg==        
  defaultNodeDrainTimeout: 0

Editing the TkgServiceConfiguration

Refer to the following procedure to edit the TkgServiceConfiguration specification.
  1. Configure Kubectl editing. See Configure a Text Editor for Kubectl.
  2. Authenticate with the Supervisor.
    kubectl vsphere login --server=SVC-IP-ADDRESS --vsphere-username USERNAME
  3. Switch context to the target vSphere Namespace.
    kubectl config use-context vSphere-Namespace
  4. Get the TkgServiceConfiguration specification.
    kubectl get tkgserviceconfigurations
  5. Load for editing the TkgServiceConfiguration specification.
    kubectl edit tkgserviceconfigurations tkg-service-configuration

    The system opens the tkg-service-configuration specification in the default text editor defined by your KUBE_EDITOR or EDITOR environment variable.

  6. Edit the TkgServiceConfiguration according to your requirements.
  7. To apply the changes, save the file in the text editor. To cancel, close the editor without saving.

    When you save the change in the text editor, kubectl updates the tkg-service-configuration service specification.

  8. Verify that the TkgServiceConfiguration specification is updated.
    kubectl get tkgserviceconfigurations

Propagating Global Configuration Changes to Existing Clusters

Settings made at the global level in the TkgServiceConfiguration may not be automatically propagated to existing clusters. For example, if you make changes to either the proxy or the trust settings in the TkgServiceConfiguration, such changes may not affect clusters that are already provisioned.

To manually propagate a global change to an existing cluster, you must patch the Tanzu Kubernetes cluster to make the cluster inherit the changes made to the TkgServiceConfiguration.

For example:
kubectl patch tkc <CLUSTER_NAME> -n <NAMESPACE> --type merge -p "{\"spec\":{\"settings\":{\"network\":{\"proxy\": null}}}}"
kubectl patch tkc <CLUSTER_NAME> -n <NAMESPACE> --type merge -p "{\"spec\":{\"settings\":{\"network\":{\"trust\": null}}}}"