Implement Log Forwarding with Fluent Bit

Fluent Bit is a lightweight log processor and forwarder that allows you to collect data and logs from different sources, unify them, and send them to multiple destinations. Tanzu Kubernetes Grid includes signed binaries for Fluent Bit, that you can deploy on management clusters and on Tanzu Kubernetes clusters to provide a log-forwarding service.

The Fluent Bit implementation provided in this release of Tanzu Kubernetes Grid allows you to gather logs from management clusters or Tanzu Kubernetes clusters running in vSphere, Amazon EC2, and Azure. You can then forward them to a log storage provider such as Elastic Search, Kafka, Splunk, or an HTTP endpoint.

You can deploy Fluent Bit on any management clusters or Tanzu Kubernetes clusters from which you want to collect logs. First, you configure an output plugin on the cluster from which you want to gather logs, depending on the endpoint that you use. Then, you deploy Fluent Bit on the cluster.


  • A bootstrap machine with the following installed:
  • A Tanzu Kubernetes Grid management cluster and workload cluster running on vSphere, Amazon EC2, or Azure, with the package repository installed. For information about deploying, see Deploy Tanzu Kubernetes Clusters.
  • You have deployed one of the logging management backends for storing and analyzing logs that Fluent Bit supports such as VMware vRealize Log Insight. For information, see the Fluent Bit documentation.


  • Support for IPv6 addresses in Tanzu Kubernetes Grid is limited; see Deploy Clusters on IPv6 (vSphere Only). If you are not deploying to an IPv6-only networking environment, you must provide IPv4 addresses in the following steps.

Prepare the Workload Cluster for Fluent Bit Deployment

Note: Deploying Fluent Bit on a management cluster does not mean that logs will also be collected from the Tanzu Kubernetes clusters that the management cluster manages. You must deploy Fluent Bit on each cluster from which you want to gather logs.

This procedure applies to all clusters, running on vSphere, Amazon EC2, and Azure.

To prepare the cluster:

  1. Get the admin credentials of the workload cluster into which you want to deploy Fluent Bit. For example:

    tanzu cluster kubeconfig get my-cluster --admin
  2. Set the context of kubectl to the cluster. For example:

    kubectl config use-context my-cluster-admin@my-cluster

Deploy Fluent Bit on a Tanzu Kubernetes Cluster

Follow the procedure below to deploy Fluent Bit on a workload cluster:

  1. Confirm that the Fluent Bit package is available in your workload cluster:

    tanzu package available list -A
  2. Retrieve the version of the available package:

    tanzu package available list -A
    | Retrieving package versions for
     NAME                           VERSION                          RELEASED-AT           NAMESPACE    1.7.5+vmware.1-tkg.1             2021-05-13T18:00:00Z  tanzu-package-repo-global
  3. Retrieve a data values file template by searching the VMware {code} Sample Exchange for Fluent Bit package configuration file for Tanzu Kubernetes Grid and downloading the file as fluent-bit-data-values.yaml.

  4. Based on the log management backend you wish to configure, add one or more [Output] configurations to the fluent_bit.config.outputs key in the fluent-bit-data-values.yaml file.

    The sample file contains examples for Syslog, ElasticSearch, Kafka, Splunk, and HTTP [Output] configurations for your reference. To use the provided Syslog Output plugin configuration, you must also uncomment the [Filter] entries under fluent-bit.config.filters under the heading “Optional - uncomment if using Syslog output plugin”

    The configuration blocks for some [Output], [Input], and [Filter] options are commented out. Uncomment and fill in values for any options you want to configure. You must provide values such as hostnames and ports for your log management backend, as these will be specific to your environment. Locations for providing these values appear in the format $PLUGINNAME_VARIABLENAME.

    For other Fluent-bit plugins, please refer to the Fluent Bit documentation.

  5. After you make any changes needed to your fluent-bit-data-values.yaml file, remove all comments in it:

    yq -i eval '... comments=""' fluent-bit-data-values.yaml
  6. Run tanzu package install to deploy the package:

    tanzu package install fluent-bit \
    --package-name \
    --values-file fluent-bit-data-values.yaml \
    --namespace TARGET-NAMESPACE


    • TARGET-NAMESPACE is the namespace in which you want to install the Fluent Bit package, Fluent Bit package app, and any other Kubernetes resources that describe the package. For example, the my-packages or tanzu-cli-managed-packages namespace. If the --namespace flag is not specified, the Tanzu CLI uses the default namespace. The Fluent Bit pods and any other resources associated with the Fluent Bit component are created in the namespace you provided in fluent-bit-data-values.yaml; do not install the Fluent Bit package into this namespace.
    • AVAILABLE-PACKAGE-VERSION is the version that you retrieved by running tanzu package available above.

    For example:

    tanzu package install fluent-bit --package-name --namespace my-packages --version 1.7.5+vmware.1-tkg.1
    - Installing package ''
    | Getting namespace 'my-packages'
    | Getting package metadata for ''
    | Creating service account 'fluent-bit-my-packages-sa'
    | Creating cluster admin role 'fluent-bit-my-packages-cluster-role'
    | Creating cluster role binding 'fluent-bit-my-packages-cluster-rolebinding'
    | Creating secret 'fluent-bit-my-packages-values'
    - Creating package resource
    \ Package install status: Reconciling
    Added installed package 'fluent-bit' in namespace 'my-packages'

    If the target namespace does not already exist in the cluster, append --create-namespace to the command above to create it.

Verify Fluent Bit Deployment

After you deploy Fluent Bit, you can verify that the deployment is successful:

  1. Confirm that the fluent-bit package is installed. For example:

    tanzu package installed list -A
    / Retrieving installed packages...
    NAME            PACKAGE-NAME                     PACKAGE-VERSION          STATUS               NAMESPACE
    fluent-bit      1.7.5+vmware.1-tkg.1     Reconcile succeeded  my-packages
    antrea                                   Reconcile succeeded  tkg-system

    The fluent-bit package and the fluent-bit app are installed in the namespace that you specify when running the tanzu package install command.

  2. Confirm that the fluent-bit app is successfully reconciled in your AVAILABLE-PACKAGE-NAMESPACE:

    kubectl get apps -A

    For example:

    NAMESPACE     NAME                           DESCRIPTION           SINCE-DEPLOY   AGE
    my-packages   fluent-bit                     Reconcile succeeded   38s            6m1s
    tkg-system    antrea                         Reconcile succeeded   3m50s          24h

    If the status is not Reconcile succeeded, view the full status details of the fluent-bit app. Viewing the full status can help you troubleshoot the problem:

    kubectl get app fluent-bit --namespace AVAILABLE-PACKAGE-NAMESPACE -o yaml

    Where AVAILABLE-PACKAGE-NAMESPACE is the namespace in which you installed the package.

  3. Confirm that the new services are running by listing all of the pods that are running in the cluster.

    kubectl get pods -A

    In the tanzu-system-logging namespace, you should see the fluent-bit service running in a Pod:

    NAMESPACE               NAME                     READY   STATUS    RESTARTS   AGE
    tanzu-system-logging    fluent-bit-zd2tn         1/1     Running   0          6m23s

    The Fluent Bit Pods and any other resources associated with the Fluent Bit component are created in the namespace you provided in fluent-bit-data-values.yaml. If you are using the default namespace, these are created in the tanzu-system-logging namespace.

Fluent Bit Package Configuration Parameters

Note: Configuration parameters have changed since TKG v1.3. Review your configurations before deployment. For information about migrating from the TKG v1.3 extension, see Migrate the Fluent Bit Extension to a Package.

Review Configuration Parameters

The following table lists configuration parameters of the Fluent Bit package and describes their default values.

You can set the following configuration values in your fluent-bit-data-values.yml file created in Deploy Fluent Bit on a Tanzu Kubernetes Cluster above.

Parameter Description Type Default
namespace Namespace where Fluent Bit will be deployed. String tanzu-system-logging
fluent_bit.config.outputs For information about the configuration for Fluent Bit outputs, see the Fluent Bit documentation. String Standard output
fluent_bit.config.parsers For information about the configuration for Fluent Bit parsers, see the Fluent Bit documentation. String JSON parser
fluent_bit.config.plugins Content for Fluent Bit plugins configuration. String <nil>
fluent_bit.config.service For information about the configuration for Fluent Bit service, see the Fluent Bit documentation. String Default fluent-bit service config.
fluent_bit.config.streams Content for Fluent Bit streams file. String <nil>
fluent_bit.config.filters For information about the configuration for Fluent Bit filters, see the Fluent Bit documentation. String Default kubernetes filter.
fluent_bit.config.inputs For information about the configuration for Fluent Bit inputs, see the Fluent Bit documentation. String Ingest Kubernetes container logs using the tail plugin and ingest systemd logs from Kubelet.
fluent_bit.daemonset.resources For information about the configuration for Fluent Bit containers resource requirements, see the Fluent Bit documentation. Object <nil>
fluent_bit.daemonset.podAnnotations The Fluent Bit deployments pod annotations. Object <nil>
fluent_bit.daemonset.podLabels The Fluent Bit deployments pod labels. Object <nil>

Update a Running Fluent Bit Deployment

To make changes to the configuration of the Fluent Bit package after deployment, update your deployed Fluent Bit package:

  1. Update the Fluent Bit configuration in the fluent-bit-data-values.yaml file.

  2. Update the installed package:

    tanzu package installed update fluent-bit \
    --version 1.7.5+vmware.1-tkg.1 \
    --values-file fluent-bit-data-values.yaml \
    --namespace my-packages

    Expected output:

    | Updating package 'fluent-bit'
    - Getting package install for 'fluent-bit'
    | Updating secret 'fluent-bit-my-packages-values'
    | Updating package install for 'fluent-bit'
     Updated package install 'fluent-bit' in namespace 'my-packages'

The Fluent Bit package is reconciled using the new value or values that you added. It can take up to five minutes for kapp-controller to apply the changes.

For information about updating, see Update a Package.

Delete a Fluent Bit Deployment

To remove the Fluent Bit package on your cluster, run:

tanzu package installed delete fluent-bit --namespace my-packages

For information about deleting, see Delete a Package.

check-circle-line exclamation-circle-line close-line
Scroll to top icon