Migrating Metrics from VMware RabbitMQ for Tanzu Application Service v1.x to v2.0

This topic tells you about the changes to metrics after upgrading from VMware Tanzu RabbitMQ for Tanzu Application Service v1.x to v2.0 and how to migrate to the new metrics.

About Metrics in Tanzu RabbitMQ for Tanzu Application Service v2.0

Tanzu RabbitMQ for Tanzu Application Service v2.0 includes improvements to the metrics available in RabbitMQ service instances. This version unifies metrics provided by service instances with the metrics provided by open-source RabbitMQ, and vastly increases the amount of information available to you for observability, and takes advantage of the more performance-optimised observability endpoint in RabbitMQ.

As a result of this refactor, the format of the metrics provided in v1.x of Tanzu RabbitMQ for Tanzu Application Service has changed. You might need to change any dashboards you used before upgrading. In many cases, there are simply one-to-one mappings of old metric names to new metric names, however some specific metrics might require some additional changes to achieve the same result. Unless otherwise stated, all metrics are available from both the Loggregator Firehose system and directly from the Prometheus endpoint on the RabbitMQ nodes themselves.

For the full list of exposed RabbitMQ and Erlang metrics, see the rabbitmq-server in GitHub.

Note

The majority of these metrics are exposed by therabbitmq_prometheusplug-in, and require you to have activated this plug-in on your service instance. The plug-in is always enabled for on-demand instances, however you can deactivate it for pre-provisioned instances. VMware recommends that you do not deactivate this plug-in.

On-Demand Service Broker Metrics

All metrics for the on-demand service broker have one-to-one equivalents in Tanzu RabbitMQ for Tanzu Application Service v2.0, and differ only in naming.

Old Metric	New Metric
/on-demand-broker/p-rabbitmq/quota_remaining	_on_demand_broker_p_rabbitmq_quota_remaining
/on-demand-broker/p-rabbitmq/total_instances	_on_demand_broker_p_rabbitmq_total_instances
/on-demand-broker/p-rabbitmq/<PLAN_NAME>/quota_remaining	_on_demand_broker_p_rabbitmq_<PLAN_NAME>_quota_remaining
/on-demand-broker/p-rabbitmq/<PLAN_NAME>/quota_remaining	_on_demand_broker_p_rabbitmq_<PLAN_NAME>_total_instances

Pre-provisioned Service Broker Metrics

All metrics for the pre-provisioned service broker have one-to-one equivalents in Tanzu RabbitMQ for Tanzu Application Service v2.0, and differ only in naming.

Old Metric	New Metric
/p-rabbitmq/service_broker/heartbeat	_p_rabbitmq_service_broker_heartbeat

HAProxy Metrics (Pre-provisioned Only)

All metrics for HAProxy have one-to-one equivalents in Tanzu RabbitMQ for Tanzu Application Service v2.0, and differ only in naming.

Old Metric	New Metric
/p-rabbitmq/haproxy/heartbeat	_p_rabbitmq_haproxy_heartbeat
/p-rabbitmq/haproxy/backend/qsize/amqp	_p_rabbitmq_haproxy_backend_qsize_amqp
/p-rabbitmq/haproxy/backend/retries/amqp	_p_rabbitmq_haproxy_backend_retries_amqp
/p-rabbitmq/haproxy/backend/ctime/amqp	_p_rabbitmq_haproxy_backend_ctime_amqp
/p-rabbitmq/haproxy/health/connections	_p_rabbitmq_haproxy_health_connections

RabbitMQ Metrics

RabbitMQ metrics are now consumed directly from the Prometheus metrics endpoint of the RabbitMQ server, rather than from the RabbitMQ Management API. In general, more information is now exposed in Tanzu RabbitMQ for Tanzu Application Service than before.

Old Metric	New Metric
/p-rabbitmq/erlang/heartbeat	No equivalent
/p-rabbitmq/erlang/erlang_processes	erlang_vm_process_count
/p-rabbitmq/erlang/reachable_nodes	count(erlang_vm_dist_node_state, where value==3)
/p-rabbitmq/rabbitmq/system/memory	rabbitmq_process_resident_memory_bytes
/p-rabbitmq/rabbitmq/system/mem_alarm	rabbitmq_alarms_memory_used_watermark
/p-rabbitmq/rabbitmq/system/disk_free_alarm	rabbitmq_alarms_free_disk_space_watermark
/p-rabbitmq/rabbitmq/system/disk_free	rabbitmq_disk_space_available_bytes
/p-rabbitmq/rabbitmq/system/disk_free_limit	rabbitmq_disk_space_available_limit_bytes
/p-rabbitmq/rabbitmq/system/file_descriptors	rabbitmq_process_open_fds
/p-rabbitmq/rabbitmq/connections/count	rabbitmq_connections
/p-rabbitmq/rabbitmq/consumers/count	rabbitmq_global_consumers
/p-rabbitmq/rabbitmq/messages/published	rabbitmq_global_messages_received_total
/p-rabbitmq/rabbitmq/messages/published_rate	rate(rabbitmq_global_messages_received_total)
/p-rabbitmq/rabbitmq/messages/delivered	rabbitmq_global_messages_delivered_consume_manual_ack_total
/p-rabbitmq/rabbitmq/messages/delivered_rate	rate(rabbitmq_global_messages_delivered_consume_manual_ack_total)
/p-rabbitmq/rabbitmq/messages/delivered_noack	rabbitmq_global_messages_delivered_consume_auto_ack_total
/p-rabbitmq/rabbitmq/messages/redelivered	rabbitmq_global_messages_redelivered_total
/p-rabbitmq/rabbitmq/messages/redelivered_rate	rate(rabbitmq_global_messages_redelivered_total)
/p-rabbitmq/rabbitmq/messages/get_no_ack	rabbitmq_global_messages_delivered_get_auto_ack_total
/p-rabbitmq/rabbitmq/messages/get_no_ack_rate	rate(rabbitmq_global_messages_delivered_get_auto_ack_total)
/p-rabbitmq/rabbitmq/messages/return_unroutable	rabbitmq_global_messages_unroutable_returned_total
/p-rabbitmq/rabbitmq/messages/return_unroutable_rate	rate(rabbitmq_global_messages_unroutable_returned_total)
/p-rabbitmq/rabbitmq/messages/pending	rabbitmq_queue_messages_unacked
/p-rabbitmq/rabbitmq/messages/pending_acknowledgements	rabbitmq_queue_messages_unacked
/p-rabbitmq/rabbitmq/queues/count	rabbitmq_queues
/p-rabbitmq/rabbitmq/messages/available	rabbitmq_queue_messages_ready
/p-rabbitmq/rabbitmq/channels/count	rabbitmq_channels
/p-rabbitmq/rabbitmq/messages/depth	rabbitmq_queue_messages

RabbitMQ Detailed Metrics

Some of the metrics from Tanzu RabbitMQ for Tanzu Application Service v1.x are only retrievable by using the /metrics/detailed endpoint on RabbitMQ servers. For more information on this endpoint, see the rabbitmq-server in GitHub.

In Tanzu RabbitMQ for Tanzu Application Service v2.0.11 and later you can configure your service instances to collect metrics from this endpoint.

If you want to collect the same metrics as in Tanzu RabbitMQ for Tanzu Application Service v1.x, do the following:

1. Set the detailed metrics endpoint to collect the relevant metrics:

   • If you use a Prometheus server: Ensure you are scraping the following endpoints on the RabbitMQ servers:
     • /metrics
     • /metrics/detailed?family=vhost_status&family=exchange_names&family=queue_consumer_count&family=queue_coarse_metrics
   • If you are using the Loggregator Firehose system to collect metrics: Configure additional metrics collection. You must set the value of detailed_metrics_query to ?family=vhost_status&family=exchange_names&family=queue_consumer_count&family=queue_coarse_metrics
     The procedure depends on the service offering you are using:
     • For the on-demand offering: You configure additional metrics when creating or updating a service instance. For more information, see Collect Additional RabbitMQ Metrics in Loggregator (on-demand instances).
     • For the pre-provisioned offering: You configure additional metrics in Ops Manager. For more information, see Collect Additional RabbitMQ Metrics in Loggregator (pre-provisioned instances).

2. Consult the following table for the equivalent metrics for this endpoint:

   Old Metric	New Metric	Endpoint Exposed on
   /p-rabbitmq/rabbitmq/vhosts/count	count(rabbitmq_cluster_vhost_status)	/metrics/detailed?family=vhost_status
   /p-rabbitmq/rabbitmq/exchanges/count	count(rabbitmq_cluster_exchange_name)	/metrics/detailed?family=exchange_names
   /p-rabbitmq/rabbitmq/exchange/<VHOST_NAME>/count	count(rabbitmq_cluster_exchange_name, aggregate by vhost)	/metrics/detailed?family=exchange_names
   /p-rabbitmq/rabbitmq/queues/<VHOST_NAME>/<QUEUE_NAME>/consumers	rabbitmq_detailed_queue_consumers	/metrics/detailed?family=queue_consumer_count
   /p-rabbitmq/rabbitmq/queues/<VHOST_NAME>/<QUEUE_NAME>/depth	rabbitmq_detailed_queue_messages	/metrics/detailed?family=queue_coarse_metrics