You can customize the Tanzu Kubernetes Grid Service with global settings for key features, including the container network interface (CNI), proxy server, and TLS certificates. Be aware of trade-offs and considerations when implementing global versus per-cluster functionality.

Optionally you can configure the Tanzu Kubernetes Grid Service with global parameters.

Caution: Configuring the Tanzu Kubernetes Grid Service is a global operation. This means that any change you make to the TkgServiceConfiguration specification applies to all Tanzu Kubernetes clusters provisioned by that service. If a rolling update is initiated, either manually or by upgrade, clusters are updated by the changed service spec.

TkgServiceConfiguration Specification

The TkgServiceConfiguration specification provides fields for configuring the Tanzu Kubernetes Grid Service instance. 
apiVersion: run.tanzu.vmware.com/v1alpha1
kind: TkgServiceConfiguration
metadata:
  name: tkg-service-configuration-example
spec:
  defaultCNI: <antrea or calico>
  proxy:
    httpProxy: http://<user>:<pwd>@<ip>:<port>
    httpsProxy: http://<user>:<pwd>@<ip>:<port>
    noProxy: [<array of CIDRs to not proxy>]
  trust:
    additionalTrustedCAs:
      - name: <first-cert-name>
        data: <base64-encoded string of a PEM encoded public cert 1>
      - name: <second-cert-name>
        data: <base64-encoded string of a PEM encoded public cert 2>

TkgServiceConfiguration Specification Parameters

The table lists and describes each of the TkgServiceConfiguration specification parameters. For examples, see Examples for Configuring the Tanzu Kubernetes Grid Service v1alpha1 API..
Field Value Description
defaultCNI antrea or calico Default CNI for clusters to use. The default is antrea. The other supported CNI is calico.
proxy Section marker for proxy parameters. The proxy parameters are httpProxy, httpsProxy, and noProxy. All parameters are required. If any proxy parameter is missing, you cannot create Tanzu Kubernetes clusters.
httpProxy URI in the form http://<user>:<pwd>@<ip>:<port> Does not allow the https protocol. If https is used, you cannot create Tanzu Kubernetes clusters.
httpsProxy URI in the form http://<user>:<pwd>@<ip>:<port> Does not allow the https protocol. If https is used, you cannot create Tanzu Kubernetes clusters.
noProxy

Array of CIDR blocks to not proxy, for example: [10.246.0.0/16,192.168.144.0/20,192.168.128.0/20].

Get the required values from the Workload Network on the Supervisor Cluster: Pod CIDRs, Ingress CIDRs, and Egress CIDRs.

Refer to the image below for what values to include in the noProxy array field.

You must not proxy the subnets used by the Workload Network on the Supervisor Cluster for Pods, Ingress, and Egress.

You do not need to include the Services CIDR from the Supervisor Cluster in the noProxy field. Tanzu Kubernetes clusters do not interact with such services.

The endpoints localhost and 127.0.0.1 are automatically not proxied. You do not need to add them to the noProxy field.

The Pod and Service CIDRs for Tanzu Kubernetes clusters are automatically not proxied. You do not need to add them to the noProxy field.
trust Section marker for trust parameters. Accepts no data.
additionalTrustedCAs Accepts an array of certificates with name and data for each. Accepts no data.
name String The name of the TLS certificate.
data String The base64-encoded string of a PEM encoded public certificate.

Get the required noProxy values from the Workload Network on the Supervisor Cluster as shown in the image.

When To Use Global or Per-Cluster Configuration Options

The TkgServiceConfiguration is a global specification that impacts all Tanzu Kubernetes clusters provisioned by the Tanzu Kubernetes Grid Service instance.

Before editing the TkgServiceConfiguration specification, be aware of the per-cluster alternatives that might satisfy your use case instead of a global configuration.
Table 1. Global vs. Per-Cluster Configuration Options
Setting Global Option Per-Cluster Option
Default CNI Edit the TkgServiceConfiguration spec. See Examples for Configuring the Tanzu Kubernetes Grid Service v1alpha1 API. Specify the CNI in the cluster specification. For example, Antrea is the default CNI. To use Calico, specify it in the cluster YAML. See Examples for Provisioning Tanzu Kubernetes Clusters Using the Tanzu Kubernetes Grid Service v1alpha1 API
Proxy Server Edit the TkgServiceConfiguration spec. See Examples for Configuring the Tanzu Kubernetes Grid Service v1alpha1 API. Include the proxy server configuration parameters in the cluster spec. See Examples for Provisioning Tanzu Kubernetes Clusters Using the Tanzu Kubernetes Grid Service v1alpha1 API.
Trust Certificates Edit the TkgServiceConfiguration spec. There are two use cases: configuring an external container registry and certificate-based proxy configuration. See Examples for Configuring the Tanzu Kubernetes Grid Service v1alpha1 API Yes, you can include custom certificates on a per-cluster basis or override the globally-set trust settings in the cluster specification. See Examples for Provisioning Tanzu Kubernetes Clusters Using the Tanzu Kubernetes Grid Service v1alpha1 API.
Note: If a global proxy is configured on the TkgServiceConfiguration, that proxy information is propagated to the cluster manifest after the initial deployment of the cluster. The global proxy configuration is added to the cluster manifest only if there is no proxy configuration fields present when creating the cluster. In other words, per-cluster configuration takes precedence and will overwrite a global proxy configuration. For more information, see Configuration Parameters for the Tanzu Kubernetes Grid Service v1alpha1 API.

Before editing the TkgServiceConfiguration specification, be aware of the ramifications of applying the setting at the global level.

Field Applied Impact on Existing Clusters If Added/Changed Per-Cluster Overriding on Cluster Creation Per-Cluster Overriding on Cluster Update
defaultCNI Globally None Yes, you can override the global setting on cluster creation No, you cannot change the CNI for an existing cluster; if you used the globally set default CNI on cluster creation, it cannot be changed
proxy Globally None Yes, you can override the global setting on cluster creation Yes, with U2+, you can override the global setting on cluster update
trust Globally None Yes, you can override the global setting on cluster creation Yes, with U2+, you can override the global setting on cluster update

Propagating Global Configuration Changes to Existing Clusters

Settings made at the global level in the TkgServiceConfiguration are not automatically propagated to existing clusters. For example, if you make a changes to either the proxy or the trust settings in the TkgServiceConfiguration, such changes will not affect clusters that are already provisioned.

To propagate a global change to an existing cluster, you must patch the Tanzu Kubernetes cluster to make the cluster inherit the changes made to the TkgServiceConfiguration.

For example: 
kubectl patch tkc <CLUSTER_NAME> -n <NAMESPACE> --type merge -p "{\"spec\":{\"settings\":{\"network\":{\"proxy\": null}}}}"
kubectl patch tkc <CLUSTER_NAME> -n <NAMESPACE> --type merge -p "{\"spec\":{\"settings\":{\"network\":{\"trust\": null}}}}"