The vCenter adapter provides alert definitions that generate alerts on the virtual machine objects in your environment.

Health/Symptom-Based

These alert definitions have the following impact and criticality information.

Impact

Health

Criticality

Symptom-based

Alert Definition

Symptom

Recommendations

Virtual machine is experiencing memory compression, ballooning or swapping due to memory limit.

  • Virtual machine memory limit is set AND

  • Virtual machine memory demand exceeds configured memory limit AND

  • [Virtual machine memory is compressed OR

  • Virtual machine is using swap OR

  • Virtual machine memory ballooning is at warning/immediate/critical level] AND

  • Recommended virtual machine memory size

Increase the memory limit for the virtual machine to match the recommended memory size. Alternatively, remove memory limit for the virtual machine.

Virtual machine has CPU contention caused by swap wait.

Virtual machine CPU swap wait is at warning/Immediate/Critical level.

  1. Upgrade the host with more memory.

  2. Use vSphere vMotion to migrate this virtual machine to a different host or cluster.

  3. Set memory reservations for the virtual machine to prevent swapping.

Virtual machine has CPU contention caused by IO wait.

Virtual machine CPU I/O wait is at warning/immediate/critical level.

Increase the datastore I/O capacity for the connected data stores to reduce CPU I/O wait on the virtual machine.

Virtual machine has unexpected high CPU workload.

Symptoms include all of the following:

  • Virtual machine CPU demand at warning/immediate/critical level

  • Anomaly is starting to/moderately/critically high

  1. Check the guest applications to determine whether high CPU workload is an expected behavior.

  2. Add more CPU capacity for this virtual machine.

Virtual machine has unexpected high memory workload.

Symptoms include all of the following:

  • Virtual machine memory workload is at Warning/Immediate/Critical level

  • Anomaly is starting to/moderately/critically high

  1. Check the guest applications to determine whether high memory workload is an expected behavior.

  2. Add more memory for this virtual machine.

Virtual machine has memory contention due to swap wait and high disk read latency.

Symptoms include all of the following:

  • Virtual machine CPU swap wait is at warning/immediate/critical level (5/10/15)

  • Virtual machine has

    read latency at warning level

  • Recommended virtual machine memory size

Add more memory for this virtual machine.

Virtual machine has memory contention due to memory compression, ballooning or swapping.

  • ! Virtual machine memory limit is set AND

  • Virtual machine has memory contention at warning/immediate/critical level AN

  • [ Virtual machine memory ballooning at warning/immediate/critical level OR

  • Virtual machine memory is compressed OR

  • Virtual machine is using swap]

  1. Add memory reservations to this virtual machine to prevent ballooning and swapping.

  2. Use vSphere vMotion to migrate this virtual machine to a different host or cluster.

Virtual machine has unexpected high disk I/O workload.

Symptoms include all of the following:

  • Virtual machine disk I/O workload at Warning/Immediate/Critical level (80/90/95)

  • Virtual machine disk I/O workload above DT

  1. Check the applications running on the virtual machine to determine whether high disk I/O workload is an expected behavior.

  2. Use vSphere Storage vMotion to migrate this virtual machine to a different datastore with higher IOPS.

Virtual machine has disk I/O read latency problem.

Symptoms include all of the following:

  • Virtual machine disk read latency at Warning /Immediate/Critical level

  • Virtual machine disk read latency above DT

  • Virtual machine has low co-stop

  • Virtual machine has low CPU swap wait

  1. Check whether you have enabled Storage IO control on the datastores connected to the virtual machine.

  2. Increase IOPS for the datastores connected to the virtual machine.

  3. UsevSphere Storage vMotion to migrate this virtual machine to a different datastore with higher IOPS.

Virtual machine has disk I/O write latency problem.

Symptoms include all of the following:

  • Virtual machine disk write latency at Warning/ Immediate/Critical level

  • Virtual machine disk write latency above DT

  • Virtual machine has low CPU swap wait (< 3 ms)

  1. Check whether you have enabled Storage IO Control on the data stores connected to the datastore.

  2. Increase IOPS for the data stores connected to the virtual machine.

  3. If the virtual machine has multiple snapshots, delete the older snapshots.

  4. Use vSphere Storage vMotion to migrate some virtual machines to a different datastore.

Virtual machine has disk I/O latency problem caused by snapshots.

Symptoms include all of the following:

  • Virtual machine CPU I/O wait is at warning/immediate/critical level

  • Virtual machine has at least one snapshot

  • All child datastores have [ ! Disk command latency at warning level ]

  1. If the virtual machine has multiple snapshots, delete the older snapshots.

  2. Reduce the number of snapshots by consolidating the snapshots into one snapshot. In vSphere Client, select the VM, right-click, select Snapshot, and then Consolidate.

Virtual machine is consuming disk space in a rapid and unexpected manner.

Symptoms include all of the following:

  • Guest file system overall disk space usage reaching warning/immediate/critical limit (80, 90, 95)

  • Virtual machine disk space time remaining high (> 60 days)

  • Guest file system space usage above DT

  • Guest partition disk space usage

  1. Check the application and verify that it is behaving correctly.

  2. Add a new hard disk to the virtual machine and configure the guest file system partition to use the disk.

One or more guest file systems is out of disk space.

One or more guest file systems out of disk space (Fault symptom).

Add a new hard disk to the virtual machine and configured the guest file system partition to use the disk.

Not enough resources for vSphere HA to start the virtual machine.

Not enough resources forvSphere HA to start VM (Fault symptom).

  1. If virtual machine CPU reservation is set, decrease the CPU reservation configuration.

  2. If virtual machine memory reservation is set, decrease the memory reservation configuration.

  3. Add more hosts to cluster.

  4. Bring any failed hosts online or resolve a network partition, if one exists.

  5. If DRS is in manual mode, look for pending recommendations and approve the recommendations so that vSphere HA failover can proceed.

The Fault tolerance state of the virtual machine has changed to "Disabled" state.

VM fault tolerance state changed to disabled (Fault symptom).

Enable the secondary virtual machine indicated in the alert.

vSphere HA failed to restart a network isolated virtual machine.

vSphere HA failed to restart a network isolated virtual machine (Fault symptom).

Manually power on the virtual machine.

The fault tolerance state of the virtual machine has changed to "Needs Secondary" state.

VM Fault Tolerance state changed to needs secondary (Fault symptom).

Keep HA enabled when Fault tolerance (FT) is required to protect virtual machines.

vSphere HA cannot perform a failover operation for the virtual machine

vSphere HA virtual machine failover unsuccessful (Fault symptom)

  1. If the error information reports that a file is locked, the virtual machine might be powered on a host that the vSphere HAmaster agent can no longer monitor by using the management network or heartbeat datastores.

  2. The virtual machine might have been powered on by a user on a host outside of the cluster. If any hosts are declared offline, determine whether a networking or storage problem caused the situation.

  3. If the error information reports that the virtual machine is in an invalid state, an in-progress operation might be preventing access to the virtual machine files. Determine whether any operations are in progress, such as a clone operation that is taking a long time to complete.

  4. You can also try to power on the virtual machine and investigate any returned errors.

Virtual machine is experiencing memory compression, ballooning or swapping due to memory limit.

  • Virtual machine memory limit is set

  • Virtual machine memory demand exceeds configured memory limit

  • [Virtual machine memory is compressed OR

  • Virtual machine is using swap OR

  • Virtual machine memory ballooning is at warning/immediate/critical level]

  • Recommended virtual machine memory size

Increase the memory limit for the virtual machine to match the recommended memory size. Alternatively, remove memory limit for the virtual machine.

Efficiency/Symptom-Based

These alert definitions have the following impact and criticality information.

Impact

Efficiency

Criticality

Symptom-based

Alert Definition

Symptom

Recommendations

Virtual machine has large disk snapshots.

Symptoms include all of the following:

  • Virtual machine has large disk snapshots

  • Reclaimable snapshot waste

  • Datastore space usage reaching warning/immediate/critical limit

If the virtual machine has multiple snapshots, delete the older snapshots.

Efficiency/Warning

These alert definitions have the following impact and criticality information.

Impact

Efficiency

Criticality

Warning

Alert Definition

Symptom

Recommendations

Virtual machine is idle.

Symptoms include all of the following:

  • Virtual machine is idle

  • Virtual machine high ready time on each vCPU

  • ! Virtual machine is powered off

Power off this virtual machine to allow for other virtual machines to use CPU and memory that this virtual machine is wasting.

Risk/Symptom-Based

These alert definitions have the following impact and criticality information.

Impact

Risk

Criticality

Symptom-based

Alert Definition

Symptom

Recommendations

Virtual machine has CPU contention caused by co-stop.

Symptoms include all of the following:

  • Virtual machine CPU co-stop at warning/immediate/critical level

  • ! Virtual machine is powered off

  • Number of vCPUs to remove from virtual machine

Review the symptoms listed and remove the number of vCPUs from the virtual machine as recommended by the symptom.

Virtual machine has chronic high CPU workload leading to CPU stress.

Symptoms include all of the following:

  • Virtual machine CPU stress is at warning/immediate/critical level

  • Recommended number of vCPUs to add

Add more CPU capacity for this virtual machine.

Virtual machine has high CPU co-stop due to snapshots.

Symptoms include all of the following:

  • Virtual machine CPU co-stop is at warning/immediate/critical level

  • Virtual machine has at least one snapshot

To reduce the high co-stop (%CSTP) values and increase virtual machine performance, consolidate any snapshots into the main virtual disk. In the vSphere Client, select the VM, right click, and select Snapshot, and then Consolidate. After consolidation, the %CSTP value is reduced or eliminated and VM performance is improved. If performance is not improved enough, continue researching other potential VM performance issues. See VMware KB: http://kb.vmware.com/kb/2000058

Virtual machine has chronic high memory workload leading to memory stress.

Symptoms include all of the following:

  • Virtual machine memory stress at warning/immediate/critical level

  • Recommended virtual machine memory size > 0

Add more memory for the VM.

Virtual machine is projected to run out of disk space.

Symptoms include all of the following:

  • Virtual machine disk space time remaining low (<= 60 days)

  • ! Guest file system space usage above DT

  • ! Guest file system overall disk space usage reaching warning limit (85%)

  • Guest partition disk space usage

  1. Check the application configuration to determine whether the virtual machine disk capacity will be sufficient.

  2. Add a new hard disk to the virtual machine and configured the guest file system partition to use the disk.

Virtual machine is running out of disk space.

Symptoms include all of the following:

  • Guest file system overall disk space usage reaching warning/immediate/critical limit (80, 90, 95)

  • Virtual machine disk space time remaining low (<= 60 days)

  • ! Guest file system space usage above DT

  • Guest partition disk space usage

  1. Add a new hard disk to the virtual machine and configured the guest file system partition to use the disk.

  2. Reclaim disk space using in-guest disk cleanup mechanisms.

Virtual machine is violating vSphere 5.5 hardening guide.

  • Unrestricted VM-to-VM communication through VMCI OR

  • VMsafe CPU/Memory APIs-port number configured OR

  • Dvfilter network API enabled OR

  • Non-compliant max VMX file size OR

  • Non-compliant max VM log file size OR

  • Allow unauthorized modification of device settings OR

  • Allow unauthorized connect and disconnect of devices OR

  • Tools auto install not disabled OR

  • Non-compliant max number of remote console connections OR

  • Allow VM to obtain detailed information about the physical host OR

  • Non-compliant max VM log file count OR

  • Feature not exposed in vSphere: MemsFss is not disabled OR

  • VMsafe CPU/memory API enabled OR

  • Parallel port connected OR

  • Console drag and drop operation not disabled OR

  • Console copy operation not disabled OR

  • Serial port connected OR

  • Feature not exposed in vSPhere: AutoLogon is not disabled OR

  • Use independent non persistent disk OR

  • Feature not exposed in vSphere: UnityPush is not disabled OR

  • Shrink virtual disk not disabled - diskShrink OR

  • Feature not exposed in vSphere: GetCreds is not disabled OR

  • CD-ROM connected OR

  • Feature not exposed in vSPhere: HGFSServerSet is not disabled OR

  • Console paste operation not disabled OR

  • Feature not exposed in vSPhere: BIOSBBS is not disabled OR

  • Shrink virtual disk not disabled - diskWiper OR

  • USB controller connected OR

  • Feature not exposed in vSPhere: Monitor Control is not disabled OR

  • Floppy drive connected OR

  • Feature not exposed in vSphere: LaunchMenu is not disabled OR

  • Versionget is not disabled OR

  • Feature not exposed in vSphere: Toporequest is not disabled OR

  • Feature not exposed in vSphere: Unity-interlock not disabled OR

  • VM logging is not disabled OR

  • Feature not exposed in vSphere: Unity is not disabled OR

  • Feature not exposed in vSphere: Trashfolderstate is not disabled OR

  • VGA only mode is not enabled OR

  • Feature not exposed in vSphere: Trayicon is not disabled OR

  • Feature not exposed in vSphere: Unity-Taskbar is not disabled OR

  • Feature not exposed in vSphere: Versionset is not disabled OR

  • VM console access via VNC protocol is not disabled OR

  • Feature not exposed in vSphere: Protocolhandler is not disabled OR

  • VIX message is not disabled OR

  • Feature not exposed in vSphere: Shellaction is not disabled OR

  • 3D features is not disabled OR

  • Feature not exposed in vSphere: Unity-Windowcontents is not disabled OR

  • Feature not exposed in vSphere: Unity-Unityactive is not disabled

Fix the vSphere 5.5 hardening guide rule violations according to the recommendations in the vSphere Hardening Guide (XLSX).

Risk/Warning

These alert definitions have the following impact and criticality information.

Impact

Risk

Criticality

Warning

Alert Definition

Symptom

Recommendations

Virtual machine is demanding more CPU than the configured limit.

Symptoms include all of the following:

  • Virtual machine CPU limit is set

  • Virtual machine CPU demand exceeds configured limit

  • ! Virtual machine's CPU demand exceeds its provisioned capacity

Increase of remove CPU limits on the VM.