vRealize Operations Manager collects capacity, device, and summary metrics for datastore objects.

Capacity metrics can be calculated for datastore objects. See Capacity and Project-Based Metrics.

Metrics marked with an asterisk (*) provide the most relevant data to use when you troubleshoot the datastores in your environment.

Capacity Metrics for Datastores

Capacity metrics provide information about datastore capacity.

Metric

Description

* Capacity|Available Space (GB)

This metric shows the amount of free space that a datastore has available.

Use this metric to know how much storage space is unused on the datastore. Try to avoid having too little free disk space in order to accommodate unexpected storage growth on the datastore. The exact size of the datastore is based on company policy.

Key: capacity|available_space

Capacity|Data Store Capacity Contention

Datastore capacity contention.

Key: capacity|contention

* Capacity|Provisioned (GB)

This metric shows the amount of storage that was allocated to the virtual machines.

Use this metric to know how much storage space is currently being used on the datastore.

Check the metric trend to identify spikes or abnormal growth.

Key: capacity|provisioned

* Capacity|Total Capacity (GB)

This metric shows the overall size of the datastore.

Use this metric to know the total capacity of the datastore.

Typically the size of the datastore should not be too small. VMFS datastore size has grown over the years as virtualization matures and larger vitual machines are now onboard. Ensure the size can handle enough virtual machines to avoid datastore sprawl. A best practise is to use 5 TB for VMFS and more for vSAN.

Key: capacity|total_capacity

Capacity|Used Space (GB)

This metric shows the amount of storage that is being used on the datastore.

Key: capacity|used_space

Capacity|Workload (%)

Capacity workload.

Key: capacity|workload

Capacity|Uncommitted Space (GB)

Uncommitted space in gigabytes.

Key: capacity|uncommitted

Capacity|Total Provisioned Consumer Space

Total Provisioned Consumer Space.

Key: capacity|consumer_provisioned

* Capacity|Used Space (%)

This metric shows the amount of storage that is being used on the datastore.

Use this metric to know the percentage of storage space being used on the datastore.

When using this metric, verify that you have at least 20% of free storage. Less than this, and you may experience problems when a snapshot is not deleted. If you have more than 50% free storage space, you are not utilizing your storage in the best possible way.

Key: capacity|usedSpacePct

Device Metrics for Datastores

Device metrics provide information about device performance.

Metric

Description

Devices|Bus Resets

This metric shows the number of bus resets in the performance interval.

Key: devices|busResets_summation

Devices|Commands Aborted

This metric shows the number of disk commands aborted in the performance interval.

Key: devices|commandsAborted_summation

Devices|Commands Issued

This metric shows the number of disk commands issued in the performance interval.

Key: devices|commands_summation

Devices|Disk Command Latency (ms)

This metric shows the average time taken for a command from the perspective of a guest operating system. This metric is the sum of Kernel Disk Command Latency and Physical Device Command Latency metrics.

Key: devices|totalLatency_average

Devices|Disk Read Latency (ms)

This metric shows the average time taken for a read from the perspective of a guest operating system. This metric is the sum of the Kernel Disk Read Latency and Physical Device Read Latency metrics.

Key: devices|totalReadLatency_averag

Devices|Disk Write Latency (ms)

This metric shows the average amount of time for a write operation to the datastore. Total latency is the sum of kernel latency and device latency.

Key: devices|totalWriteLatency_average

Devices|Kernel Disk Command Latency (ms)

Average time spent in ESX Server V. Kernel per command.

Key: devices|kernelLatency_average

Devices|Kernel Disk Read Latency (ms)

Average time spent in ESX host VM Kernel per read.

Key: devices|kernelReadLatency_average

Devices|Kernel Disk Write Latency (ms)

Average time spent in ESX Server VM Kernel per write.

Key: devices|kernelWriteLatency_average

Devices|Number of Running Hosts

Number of running hosts that are powered on.

Key: devices|number_running_hosts

Devices|Number of Running VMs

Number of running virtual machines that are powered on.

Key: devices|number_running_vms

Devices|Physical Device Command Latency (ms)

Average time taken to complete a command from the physical device.

Key: devices|deviceLatency_average

Devices|Physical Device Read Latency (ms)

Average time taken to complete a read from the physical device.

Key: devices|deviceReadLatency_average

Devices|Queue Command Latency (ms)

Average time spent in the ESX Server VM Kernel queue per command.

Key: devices|queueLatency_average

Devices|Queue Read Latency (ms)

Average time spent in the ESX Server VM Kernel queue per read.

Key: devices|queueReadLatency_average

Devices|Queue Write Latency (ms)

Average time spent in the ESX Server VM Kernel queue per write.

Key: devices|queueWriteLatency_average

Devices|Read Rate (KBps)

Amount of data read in the performance interval.

Key: devices|read_average

Devices|Read Requests

Number of times data was read from the disk in the defined interval.

Key: devices|numberRead_summation

Devices|Reads per second

Average number of read commands issued per second to the datastore during the collection interval.

Key: devices|numberReadAveraged_average

Devices|Usage Average (KBps)

Average use in kilobytes per second.

Key: devices|usage_average

Devices|Write Rate (KBps)

Amount of data written to disk in the performance interval.

Key: devices|write_average

Devices|Write Requests

Number of times data was written to the disk in the defined interval.

Key: devices|numberWrite_summation

Devices|Writes per second

Average number of write commands issued per second to the datastore during the collection interval.

Key: devices|numberWriteAveraged_average

Devices|Commands per second

Average number of commands issued per second during the collection interval.

Key: devices|commandsAveraged_average

Devices|Physical Device Write Latency (ms)

Average time taken to complete a write from the physical disk.

Key: devices|deviceWriteLatency_average

Datastore Metrics for Datastores

Datastore metrics provide information about datastore use.

Metric

Description

* Datastore|Disk Command Latency (ms)

This metric shows the adjusted read and write latency at the datastore level. Adjusted means that the latency is taking into account the number of IOs. If your IO is read-dominated, the combined value is influenced by the reads.

This is the average of all the VMs running in the datastore. Because it is an average, some VMs logically experience higher latency that the value shown by this metric. To see the worst latency experienced by any VM, use the Maximum VM Disk Latency metric.

Use this metric to see the performance of the datastore. It is one of two key performance indicators for a datastore, the other being the Max Read Latency. The combination of Maximum and Average gives better insight into how well the datastore is coping with the demand.

The number should be lower than the performance you expect.

Key: datastore|totalLatency_average

Datastore|Usage Average (KBps)

Average use in kilobytes per second.

Key: datastore|usage_average

Datastore|Read Latency (ms

Average amount of time for a read operation from the datastore. Total latency = kernel latency + device latency.

Key: datastore|totalReadLatency_average

* Datastore|Write Latency (ms)

Average amount of time for a write operation to the datastore. Total latency = kernel latency + device latency.

Key: datastore|totalWriteLatency_average

Datastore|Demand

Demand.

Key: datastore|demand

Datastore|Demand Indicator

Demand Indicator.

Key: datastore|demand_indicator

Datastore|Max Observed Reads per Second

Maximum observed average number of read commands issued per second during the collection interval.

Key: datastore|maxObserved_NumberRead

Datastore|Max Observed Read Rate (KBps)

Max observed rate of reading data from the datastore.

Key: datastore|maxObserved_Read

* Datastore|Max Observed Read Latency (ms)

This metric displays the maximum observed average amount of time for a read operation from the datastore.

Use this metric to troubleshoot when the overall datastore latency is higher than expected. Look for a number that matches the overall latency.

Total latency = kernel latency + device latency.

Key: datastore|maxObserved_ReadLatency

Datastore|Max Observed Writes per second

Max observed average number of write commands issued per second during the collection interval.

Key: datastore|maxObserved_NumberWrite

Datastore|Max Observed Write Rate (KBps)

Max observed rate of writing data from the datastore.

Key: datastore|maxObserved_Write

Datastore|Max Observed Write Latency (ms)

This metric displays the maximum observed average amount of time for a write operation from the datastore.

Use this metric to troubleshoot when the overall datastore latency is higher than expected. Look for a number that matches the overall latency.

Total latency = kernel latency + device latency.

Key: datastore|maxObserved_WriteLatency

Datastore|Max Observed Number of Outstanding IO Operations

Maximum observed number of outstanding IO operations.

Key: datastore|maxObserved_OIO

Datastore|Outstanding IO requests

OIO for datastore.

Key: datastore|demand_oio

* Datastore|Reads per second (IOPS)

This metric displays the average number of read commands issued per second during the collection interval.

Use this metric when the total IOPS is higher than expected. Drill down to see if the metric is read or write dominated. This helps determine the cause of the high IOPS. Certain workloads such as backups, anti-virus scans, and Windows updates carry a Read/Write pattern. For example, an anti-virus scan is heavy on read since it is mostly reading the file system.

Key: datastore|numberReadAveraged_average

* Datastore|Writes per second (IOPS)

This metric displays the average number of write commands issued per second during the collection interval.

Use this metric when the total IOPS is higher than expected. Drill down to see if the metric is read or write dominated. This helps determine the cause of the high IOPS. Certain workloads such as backups, anti-virus scans, and Windows updates carry a Read/Write pattern. For example, an anti-virus scan is heavy on read since it is mostly reading the file system.

Key: datastore|numberWriteAveraged_average

Datastore|Read rate

This metric displays the amount of data read in the performance interval.

Key: datastore|read_average

Datastore|Write rate

This metric displays the amount of data written to disk in the performance interval.

Key: datastore|write_average

About Datastore Metrics for Virtual SAN

The metric named datastore|oio|workload is not supported on Virtual SAN datastores. This metric depends on datastore|demand_oio, which is supported for Virtual SAN datastores.

The metric named datastore|demand_oio also depends on several other metrics for Virtual SAN datastores, one of which is not supported.

  • The metrics named devices|numberReadAveraged_average and devices|numberWriteAveraged_average are supported.

  • The metric named devices|totalLatency_average is not supported.

As a result, vRealize Operations Manager does not collect the metric named datastore|oio|workload for Virtual SAN datastores.

Disk Space Metrics for Datastores

Disk space metrics provide information about disk space use.

Metric

Description

Diskspace|Not Shared (GB)

Unshared space in gigabytes.

Key: diskspace|notshared

Diskspace|Number of Virtual Disk

Number of virtual disks.

Key: diskspace|numvmdisk

Diskspace|Provisioned Space (GB)

Provisioned space in gigabytes.

Key: diskspace|provisioned

Diskspace|Shared Used (GB)

Shared used space in gigabytes.

Key: diskspace|shared

* Diskspace|Snapshot Space (GB)

This metric shows the amount of space taken by snapshots on a given database.

Use this metric to know how much storage space is being used by virtual machine snapshots on the datastore.

Check that the snapshot is using 0 GB or minimal space. Anything over 1 GB should trigger a warning. The actual value depends on how IO intensive the virtual machines in the datastore are. Run a DT on them to detect anomaly. Clear the snapshot within 24 hours, preferably as soon as you have finished backing up, or patching.

Key: diskspace|snapshot

Diskspace|Virtual Disk Used (GB)

Virtual disk used space in gigabytes.

Key: diskspace|diskused

Diskspace|Virtual machine used (GB)

Virtual machine used space in gigabytes.

Key: diskspace|used

Diskspace|Total disk space used

Total disk space used on all datastores visible to this object.

Key: diskspace|total_usage

Diskspace|Total disk space

Total disk space on all datastores visible to this object.

Key: diskspace|total_capacity

Diskspace|Total provisioned disk space

Total provisioned disk space on all datastores visible to this object.

Key: diskspace|total_provisioned

Diskspace|Total used (GB)

Total used space in gigabytes.

Key: diskspace|disktotal

Diskspace|Swap File Space (GB)

Swap file space in gigabytes.

Key: diskspace|swap

Diskspace|Other VM Space (GB)

Other virtual machine space in gigabytes.

Key: diskspace|otherused

Diskspace|Freespace (GB)

Unused space available on datastore.

Key: diskspace|freespace

Diskspace|Capacity (GB)

Total capacity of datastore in gigabytes.

Key: diskspace|capacity

Diskspace|Overhead

Amount of disk space that is overhead.

Key: diskspace|overhead

Summary Metrics for Datastores

Summary metrics provide information about overall performance.

Metric

Description

* Summary|Total Number of Hosts

This metric shows the number of hosts that the datastore is connected to.

Use this metric to know how many clusters the datastore is attached to.

The number should not be too high, as a datastore should not be mounted by every host. The datastore and cluster should be paired to keep operations simple.

Key: summary|total_number_hosts

* Summary|Total Number of VMs

This metric shows the number of virtual machines which save their VMDK files on the datastore. If a VM has four VMDKs stored in four datastores, the VM will be counted on each datastore.

Use this metric to know how many VMs have at least one VMDK on a specific datastore.

The number of VMs should be within your Concentration Risk policy.

You should also expect the datastore to be well used. If only a few VMs are using the datastore, this is not considered a good use.

Key: summary|total_number_vms

Summary|Maximum Number of VMs

Maximum number of virtual machines.

Key: summary|max_number_vms

Summary|Workload Indicator

Workload indicator.

Key: summary|workload_indicator

* Summary|Total Number of Clusters

This metric shows the number of clusters that the datastore is connected to.

Key: summary|total_number_clusters

Template Metrics for Datastores

Metric

Description

Template|Virtual Machine used

Space used by virtual machine files.

Key: template|used

Template|Access Time

Last access time.

Key: template|accessTime