The cluster add-on framework was introduced as part of VMware Telco Cloud Platform 2.5. The configuration and additional elements of the management and workload clusters are delivered through the add-on framework.
The add-on framework moves some of the cluster configuration options into a modular framework. The modular framework can be used not only for generic cluster elements but to support an increasing number of the Tanzu Kubernetes Grid CLI managed packages.
The add-ons are categorized as follows:
Container Networking Interface (CNI) add-ons: Antrea and Calico. These primary CNI add-ons are selected during the cluster creation.
Container Storage Interface (CSI) add-ons: vSphere-CSI and NFS Client.
Monitoring add-ons: Prometheus and Fluent-bit. These add-ons are used for metric and syslog collection, and they can be added to a workload cluster.
Networking add-ons: Multus, Avi Kubernetes Operator (AKO), and Whereabouts.
System add-ons: System settings (cluster password and generic Syslog configuration), the partner harbor system connectivity, and cert-manager.
TCA-Core add-on: nodeconfig operator. This add-on is deployed automatically as part of Telco Cloud Automation.
Tools add-ons: HELM (v2) and Velero backup frameworks. By default, HELM 3.13.3 is used for NF deployment as part of Telco Cloud Platform. This can be patched to leverage HELM 3.13.0 if required.
Telco Cloud Platform also supports native deployment of Tanzu Kubernetes Grid through Telco Cloud Automation. However, the recommended approach is to use the add-on framework.
Prometheus Add-On
Prometheus is a monitoring and alerting platform for Kubernetes. It collects and stores metrics as time-series data. As part of the Prometheus deployment, cadvisor, kube-state-metrics, node exporters, and the Prometheus server components are deployed into the workload cluster.
When deploying Prometheus, an additional Custom Resource (CR) can be applied. The default configuration deploys Prometheus with a service type of clusterip and a PVC of 150 GB for metric retention. The default Prometheus configuration from the Tanzu Kubernetes add-on framework is deployed through the custom resource. The default configuration can be modified as required. For more information about the Prometheus deployment and configuration options, see Prometheus Configuration.
Prometheus provides the collected metrics for an upstream platform such as Aria Operations to consume. A recommended integration configuration is provided to capture and translate metrics into formats that Aria Operations can understand.
If you want to modify the integration configuration, contact your local VMware representative.
Fluent-Bit Add-On
Fluent-bit is a lightweight logging processor for Kubernetes. You can deploy fluent-bit through the add-on framework to forward logging information to an external syslog or the Security Information and Event Management (SIEM) platform.
Similar to other add-ons, the fluent-bit deployment uses an additional Custom Resource (CR) for its configuration. Specific fluent-bit configuration is required for the appropriate level of logging at the cluster level.
Similar to Prometheus, Telco Cloud Platform 4.0 introduces a recommended configuration for integrating Fluent-Bit with Aria Operations for Logs.
For more information about the Fluent-bit configuration options, see Fluent-bit Configuration. For more information about integrating fluent-bit with Aria Operations for Logs, contact your local VMware representative.
Whereabouts Add-On
Whereabouts is an IP Address Management (IPAM) CNI plugin. It is used with Multus to manage the IP address assignment to secondary pod interfaces in a cluster-wide configuration.
Whereabouts does not require configuration from the add-on framework Custom Resource (CR) screen. After the add-on is deployed, the NF must create a Network Attachment Definition with the IPAM type set to 'whereabouts'. The network definition can then be consumed through the pod or deployment specification.
For more information about Whereabouts consumption, see Multus and Whereabouts deployment.
Cert-Manager Add-On
cert-manager is an x.509 certificate controller for Kubernetes environments. It allows certificates or certificate issuers to be added as objects or resources within the Kubernetes cluster.
Cert-manager supports namespaced (Issuer) or cluster-wide (ClusterIssuers) configurations. Certificates can be self-signed, CA-signed, or integrated with external issuers. For more information about Cert-Manager deployment, see Cert-Manager Installation.
The default cert-manager deployment does not create any issuers or clusterissuers. Configure the issuers after deploying cert-manager. The configuration varies depending on the customer and application requirements.
Velero Add-On
Velero provides backup capabilities for Kubernetes. Velero administrators can perform backups of Kubernetes namespaces including any PVs, which can be restored upon a failure event.
The backups can be restored to the same cluster or to a new cluster. PVs are typically restored to the same cluster.
To restore the backup to a new cluster, you must first create a clone of the original (failed or deleted) cluster and then use Velero to restore the backup to the new cluster. A remediation option must be run on the NF that was instantiated to the original cluster to remediate the NF against the new cluster.
The remediation reconfigures any Dynamic Infrastructure Provisioning (also known as Late-Binding) that is implemented as part of the NF deployment.
