Grafana is open-source software that allows you to visualize and analyze metrics data collected by Prometheus on your clusters. Tanzu Kubernetes Grid includes a Grafana package that you can deploy on your Tanzu Kubernetes clusters.

Prerequisites

IMPORTANT

  • Support for IPv6 addresses in Tanzu Kubernetes Grid is limited; see Deploy Clusters on IPv6 (vSphere Only). If you are not deploying to an IPv6-only networking environment, you must provide IPv4 addresses in the following steps.

Prepare the Workload Cluster for Grafana Deployment

This procedure applies to all clusters, running on vSphere, Amazon EC2, and Azure.

To prepare the cluster:

  1. Get the admin credentials of the workload cluster into which you want to deploy Grafana. For example:

    tanzu cluster kubeconfig get my-cluster --admin
    
  2. Set the context of kubectl to the cluster. For example:

    kubectl config use-context my-cluster-admin@my-cluster
    
  3. Enable Ingress for Grafana: By default, Grafana has Ingress enabled. This requires you to install the following packages:

Continue to Deploy Grafana on a Tanzu Kubernetes Cluster.

Deploy Grafana on a Tanzu Kubernetes Cluster

Follow the procedure below to deploy Grafana on a TKC:

  1. Confirm that the grafana package is available in your workload cluster:

    tanzu package available list -A
    
  2. Retrieve the version of the available package:

    tanzu package available list grafana.tanzu.vmware.com -A
    | Retrieving package versions for grafana.tanzu.vmware.com...
     NAME                           VERSION                          RELEASED-AT           NAMESPACE
     grafana.tanzu.vmware.com       7.5.7+vmware.1-tkg.1             2021-05-19T18:00:00Z  tanzu-package-repo-global
    
  3. Retrieve the template of the Grafana package's default configuration:

    image_url=$(kubectl -n tanzu-package-repo-global get packages grafana.tanzu.vmware.com.PACKAGE-VERSION -o jsonpath='{.spec.template.spec.fetch[0].imgpkgBundle.image}')
    
    imgpkg pull -b $image_url -o /tmp/grafana-package-PACKAGE-VERSION
    
    cp /tmp/grafana-package-PACKAGE-VERSION/config/values.yaml grafana-data-values.yaml
    

    For example:

    image_url=$(kubectl -n tanzu-package-repo-global get packages grafana.tanzu.vmware.com.7.5.7+vmware.1-tkg.1 -o jsonpath='{.spec.template.spec.fetch[0].imgpkgBundle.image}')
    
    imgpkg pull -b $image_url -o /tmp/grafana-package-7.5.7+vmware.1-tkg.1
    
    cp /tmp/grafana-package-7.5.7+vmware.1-tkg.1/config/values.yaml grafana-data-values.yaml
    

    This creates a configuration file named grafana-data-values.yaml that you can modify.

    For information about configuration parameters to use in grafana-data-values.yaml, see Grafana Package Configuration Parameters below.

  4. Set the namespace for the Grafana deployment in grafana-data-values.yaml as shown below:

    namespace: tanzu-system-dashboards
    
  5. Edit grafana-data-values.yaml and replace secret.admin_password with a Base64 encoded password.

    To generate a Base64 encoded password, run:

    echo -n 'mypassword' | base64
    

    You can also use the Base64 encoding tool at https://www.base64encode.org/ to encode your password. For example, using either method, mypassword results in the encoded password bXlwYXNzd29yZA==.

  6. (Optional) Modify the grafana datasource configuration in grafana-data-values.yaml.

    Grafana is configured with Prometheus as a default data source. If you have customized the Prometheus deployment namespace and it is not deployed in the default namespace (tanzu-system-monitoring), you need to change the grafana datasource configuration in grafana-data-values.yaml.

    To change the datasource configuration, copy the section below into the position shown and modify url as needed.

    #! The namespace in which to deploy grafana.
    namespace: tanzu-system-dashboards
    
    grafana:
      #! The grafana configuration.
      config:
        #! Refer to https://grafana.com/docs/grafana/latest/administration/provisioning/#example-data-source-config-file
        datasource_yaml: |-
          apiVersion: 1
          datasources:
            - name: Prometheus
              type: prometheus
              url: prometheus-server.<change-to-prometheus-namespace>.svc.cluster.local
              access: proxy
              isDefault: true
    

    For information about configuration parameters to use in grafana-data-values.yaml, see Grafana Package Configuration Parameters below.

  7. After you make any changes needed to your grafana-data-values.yaml file, remove all comments in it:

    yq -i eval '... comments=""' grafana-data-values.yaml
    
  8. Deploy the package:

    If the target namespace exists in the cluster, run:

    tanzu package install grafana \
    --package-name grafana.tanzu.vmware.com \
    --version AVAILABLE-PACKAGE-VERSION \
    --values-file grafana-data-values.yaml \
    --namespace TARGET-NAMESPACE
    

    Where:

    • TARGET-NAMESPACE is the namespace in which you want to install the package, such as my-packages.
    • AVAILABLE-PACKAGE-VERSION is the version that you retrieved above.

    For example:

    tanzu package install grafana --package-name grafana.tanzu.vmware.com --namespace my-packages --version 7.5.7+vmware.1-tkg.1
    
    - Installing package 'grafana.tanzu.vmware.com'
    | Getting namespace 'my-packages'
    | Getting package metadata for 'grafana.tanzu.vmware.com'
    | Creating service account 'grafana-my-packages-sa'
    | Creating cluster admin role 'grafana-my-packages-cluster-role'
    | Creating cluster role binding 'grafana-my-packages-cluster-rolebinding'
    | Creating secret 'grafana-my-packages-values'
    - Creating package resource
    \ Package install status: Reconciling
    
     Added installed package 'grafana' in namespace 'my-packages'
    

    If the target namespace does not exist in the cluster, run:

    tanzu package install grafana \
    --package-name grafana.tanzu.vmware.com \
    --version AVAILABLE-PACKAGE-VERSION \
    --values-file grafana-data-values.yaml \
    --namespace TARGET-NAMESPACE
    --create-namespace
    

    Where:

    • TARGET-NAMESPACE is the namespace in which you want to install the package, such as my-packages.
    • AVAILABLE-PACKAGE-VERSION is the version that you retrieved above.

    For example:

    tanzu package install grafana --package-name grafana.tanzu.vmware.com --namespace my-packages --version 7.5.7+vmware.1-tkg.1 -- create-namespace
    

Verify Grafana Deployment

After you deploy Grafana, you can verify that the deployment is successful:

  1. Confirm that the grafana package is installed. For example:

    tanzu package installed list -A
    / Retrieving installed packages...
    NAME            PACKAGE-NAME                     PACKAGE-VERSION          STATUS               NAMESPACE
    cert-manager    cert-manager.tanzu.vmware.com    1.1.0+vmware.1-tkg.2     Reconcile succeeded  my-packages
    contour         contour.tanzu.vmware.com         1.17.1+vmware.1-tkg.1    Reconcile succeeded  my-packages
    grafana         grafana.tanzu.vmware.com         7.5.7+vmware.1-tkg.1     Reconcile succeeded  tkg-system
    prometheus      prometheus.tanzu.vmware.com      2.27.0+vmware.1-tkg.1    Reconcile succeeded  tkg-system
    antrea          antrea.tanzu.vmware.com                                   Reconcile succeeded  tkg-system
    [...]
    

    The grafana package and its resources, such as the grafana app, are installed in the namespace that you specify when running the tanzu package install command.

  2. Confirm that the grafana app is successfully reconciled in your AVAILABLE-PACKAGE-NAMESPACE:

    kubectl get apps -A
    

    For example:

    NAMESPACE     NAME                           DESCRIPTION           SINCE-DEPLOY   AGE
    my-packages   cert-manager                   Reconcile succeeded   37s            21h
    my-packages   contour                        Reconcile succeeded   33s            129m
    my-packages   grafana                        Reconcile succeeded   19s            6m56s
    my-packages   prometheus                     Reconcile succeeded   46s            21h
    tkg-system    antrea                         Reconcile succeeded   3m50s          24h
    [...]
    

    If the status is not Reconcile succeeded, view the full status details of the grafana app. Viewing the full status can help you troubleshoot the problem:

    kubectl get app grafana --namespace AVAILABLE-PACKAGE-NAMESPACE -o yaml
    

    Where AVAILABLE-PACKAGE-NAMESPACE is the namespace in which you installed the package.

  3. Confirm that the new services are running by listing all of the Pods that are running in the cluster.

    kubectl get pods -A
    

    In the tanzu-system-dashboards namespace, you should see the grafana service running in a Pod:

    NAMESPACE               NAME                                    READY   STATUS    RESTARTS   AGE
    [...]
    tanzu-system-dashboards   grafana-6865dbb4f5-pk2qg              2/2     Running   0          7m7s
    [...]
    

The Grafana Pods and any other resources associated with the Grafana component are created in the namespace you provided in grafana-data-values.yaml.

Grafana Package Configuration Parameters

Note: Configuration parameters have changed since TKG v1.3. Review your configurations before deployment. For information about migrating from the TKG v1.3 extension, see Migrate the Grafana Extension to a Package.

There are two ways you can view configuration parameters of the Grafana package:

Review the Package Schema

You can review the package schema to expose configuration parameters:

tanzu package available get grafana.tanzu.vmware.com/7.5.7+vmware.1-tkg.1 -n AVAILABLE-PACKAGE-NAMESPACE --values-schema

You can use the output to expose values to set in your grafana-data-values.yml file created in Prepare the Grafana Package Configuration File above.

Review Configuration Parameters

The following table lists configuration parameters of the Grafana package and describes their default values.

You can set the following configuration values in your grafana-data-values.yml file created in Deploy Grafana on a Tanzu Kubernetes Cluster above.

Parameter Description Type Default
namespace Namespace where Grafana will be deployed. String tanzu-system-monitoring
grafana.deployment.replicas Number of Grafana replicas. Integer 1
grafana.deployment.containers.resources Grafana container resource requests and limits. Map {}
grafana.deployment.k8sSidecar.containers.resources k8s-sidecar container resource requests and limits. Map {}
grafana.deployment.podAnnotations The Grafana deployments Pod annotations. Map {}
grafana.deployment.podLabels The Grafana deployments Pod labels. Map {}
grafana.service.type Type of service to expose Grafana. Supported Values: ClusterIP, NodePort, LoadBalancer. (For vSphere set this to NodePort) String LoadBalancer
grafana.service.port Grafana service port. Integer 80
grafana.service.targetPort Grafana service target port. Integer 9093
grafana.service.labels Grafana service labels. Map {}
grafana.service.annotations Grafana service annotations. Map {}
grafana.config.grafana_ini For information about Grafana configuration, see Grafana Configuration Defaults in GitHub. Config file grafana.ini
grafana.config.datasource_yaml For information about datasource config, see the Grafana documentation. String prometheus
grafana.config.dashboardProvider_yaml For information about dashboard provider config, see the Grafana documentation. YAML file provider.yaml
grafana.pvc.annotations Storage class to use for persistent volume claim. By default this is null and default provisioner is used. String null
grafana.pvc.storageClassName Storage class to use for persistent volume claim. By default this is null and default provisioner is used. String null
grafana.pvc.accessMode Define access mode for persistent volume claim. Supported values: ReadWriteOnce, ReadOnlyMany, ReadWriteMany. String ReadWriteOnce
grafana.pvc.storage Define storage size for persistent volume claim. String 2Gi
grafana.secret.type Secret type defined for Grafana dashboard. String Opaque
grafana.secret.admin_user Username to access Grafana dashboard. String YWRtaW4=
grafana.secret.admin_password Password to access Grafana dashboard. String admin
ingress.enabled Enable/disable ingress for grafana. Boolean true
ingress.virtual_host_fqdn Hostname for accessing grafana. String grafana.system.tanzu
ingress.prefix Path prefix for grafana. String /
ingress.servicePort Grafana service port to proxy traffic to. Integer 80
ingress.tlsCertificate.tls.crt Optional certificate for ingress if you want to use your own TLS cert. A self signed certificate is generated by default. Note: tls.crt is a key and not nested. String Generated cert
ingress.tlsCertificate.tls.key Optional certificate private key for ingress if you want to use your own TLS certificate. Note: tls.key is a key and not nested. String Generated cert private key
ingress.tlsCertificate.ca.crt Optional CA certificate. Note: ca.crt is a key and not nested. String CA certificate

Update a Running Grafana Deployment

To make changes to the configuration of the Grafana package after deployment, update your deployed Grafana package:

  1. Update the Grafana configuration in the grafana-data-values.yaml file.

  2. Update the installed package:

    tanzu package installed update grafana \
    --version 7.5.7+vmware.1-tkg.1 \
    --values-file grafana-data-values.yaml \
    --namespace my-packages
    

    Expected output:

    | Updating package 'grafana'
    - Getting package install for 'grafana'
    | Updating secret 'grafana-my-packages-values'
    | Updating package install for 'grafana'
    
     Updated package install 'grafana' in namespace 'my-packages'
    

The Grafana package is reconciled using the new value or values that you added. It can take up to five minutes for kapp-controller to apply the changes.

For information about updating, see Update a Package.

Delete a Grafana Deployment

To remove the Grafana package on your cluster, run:

tanzu package installed delete grafana --namespace my-packages

For information about deleting, see Delete a Package.

Access the Grafana Dashboard

After Grafana is deployed, the grafana package creates a Contour HTTPProxy object with a Fully Qualified Domain Name (FQDN) of grafana.system.tanzu.

To use this FQDN to access the Grafana dashboard:

  1. Create an entry in your local /etc/hosts file that points an IP address to this FQDN:

    • Amazon EC2 or Azure: Use the IP address of the LoadBalancer for the Envoy service in the tanzu-system-ingress namespace.
    • vSphere: Use the IP address of a worker node.
  2. Navigate to https://grafana.system.tanzu.

Because the site uses self-signed certificates, you might need to navigate through a browser-specific security warning before you are able to access the dashboard.

check-circle-line exclamation-circle-line close-line
Scroll to top icon