This procedure describes how to configure Kafka as the output plugin on a cluster on which you have deployed Fluent Bit as the log forwarder.



  1. Open the file 04-fluent-bit-configmap.yaml in a text editor. For example, use vi to edit the file.

    • vSphere:

      vi tkg-extensions-v1.0.0/logging/fluent-bit/vsphere/output/kafka/04-fluent-bit-configmap.yaml
    • Amazon EC2:

      vi tkg-extensions-v1.0.0/logging/fluent-bit/aws/output/kafka/04-fluent-bit-configmap.yaml
  2. Update 04-fluent-bit-configmap.yaml to set the following environment variables:

    • <TKG_CLUSTER_NAME>: The name of the Tanzu Kubernetes Grid cluster.
    • Set <TKG_INSTANCE_NAME>: The name of Tanzu Kubernetes Grid instance. This name should be the same for the management cluster and all of the workload clusters that make up the Tanzu Kubernetes Grid deployment.
    • <KAFKA_BROKER_SERVICE_NAME>: The name of the Kafka broker service.
    • <KAFKA_TOPIC_NAME>: The name of the topic ingesting the logs in Kafka.
  3. Apply the configuration map to the cluster.

    • vSphere:
      kubectl apply -f logging/fluent-bit/vsphere/output/kafka/04-fluent-bit-configmap.yaml
    • Amazon EC2
      kubectl apply -f logging/fluent-bit/aws/output/kafka/04-fluent-bit-configmap.yaml
  4. Create Fluent Bit as a DaemonSet.

    • vSphere:
      kubectl apply -f logging/fluent-bit/vsphere/output/kafka/05-fluent-bit-ds.yaml
    • Amazon EC2:
      kubectl apply -f logging/fluent-bit/aws/output/kafka/05-fluent-bit-ds.yaml
check-circle-line exclamation-circle-line close-line
Scroll to top icon