The cluster add-on framework was introduced as part of VMware Telco Cloud Automation 2.1 for v2 clusters. The configuration and additional elements of the Management and Workload clusters are delivered through the add-on framework.
The add-on framework moves some of the cluster configuration options into a modular framework. The modular framework can be used not only for generic cluster elements but to support an increasing number of the Tanzu Kubernetes Grid CLI Managed packages.
The add-ons are categorized as follows:
Container Networking Interface (CNI) add-ons: Antrea and Calico. These primary CNI add-ons are selected during the cluster creation.
Container Storage Interface (CSI) add-ons: vSphere-CSI and NFS Client
Monitoring add-ons: Prometheus and Fluent-bit. These add-ons are used for metric and syslog collection, and they can be added to a workload cluster.
Networking add-ons: Multus, Avi Kubernetes Operator (AKO), and Whereabouts.
System add-ons: Systemsettings (cluster password and generic Syslog configuration), partner harbor system connectivity, and cert-manager
TCA-Core add-on: nodeconfig operator. This add-on is deployed automatically as part of Telco Cloud Automation.
Tools add-ons: HELM (v2) and Velero backup frameworks.
The major new additions to this framework focus on monitoring, backup and system configuration with cert-manager and Whereabouts.
Prometheus Add-on
Prometheus is a monitoring and alerting platform for Kubernetes. It collects and stores metrics as time-series data. As part of the Prometheus deployment, cadvisor, kube-state-metrics, node exporters, and the Prometheus server components are deployed into the workload cluster.
When deploying Prometheus, an additional Custom Resource (CR) can be applied. The default configuration deploys Prometheus with a service type of clusterip and a PVC of 150GB for metric retention. The default Prometheus configuration from the Tanzu Kubernetes add-on framework is deployed through the custom resource. The default configuration can be modified as required. For more information about the Prometheus deployment and configuration options, see Prometheus Configuration.
Prometheus provides the collected metrics such as vROps to an upstream element to parse. For more information about integrating Prometheus with vRealize Operations, contact your local VMware representative.
Fluent-bit Add-on
Fluent-bit is a lightweight logging processor for Kubernetes. You can deploy fluent-bit through the add-on framework to forward logging information to an external syslog or the Security Information and Event Management (SIEM) platform.
Similar to other add-ons, the fluent-bit deployment uses an additional Custom Resource (CR) for its configuration. Specific fluent-bit configuration is required for the appropriate level of logging at the cluster level.
For more information about the Prometheus configuration options, see Fluent-bit Configuration. For more information about integrating fluent-bit with vRealize Log Insight, contact your local VMware representative.
Whereabouts Add-on
Whereabouts is an IP Address Management (IPAM) CNI plugin. It is used in conjunction with Multus to manage the IP address assignment to secondary pod interfaces in a cluster-wide configuration.
Whereabouts does not require configuration from the add-on framework Custom Resource (CR) screen. After the add-on is deployed, the NF must create a Network Attachment Definition with the IPAM type set to 'whereabouts'. The network definition can then be consumed through the pod or deployment specification.
For more information about Whereabouts consumption, see Multus and Whereabouts deployment.
Cert-Manager Add-on
cert-manager is an x.509 certificate controller for Kubernetes environments. It allows certificates or certificate issuers to be added as objects or resources within the Kubernetes cluster.
Cert-manager supports namespaced (Issuer) or cluster-wide (ClusterIssuers) configurations. Certificates can be self-signed, CA signed, or integrated with external issuers. For more information about Cert-Manager deployment, see Cert-Manager Installation.
The default cert-manager deployment does not create any issuers or clusterissuers. Configure the issuers after deploying cert-manager. The configuration varies depending on the customer and application requirements.
