VMware Smart Assurance software provides a utility that can monitor escalation, including statistics for each escalation level in every escalation path configured in the Global Manager. The syntax is as follows:
tdmctl -s <server_name> invoke ICS_AutoActionManager::ICS-AutoActionManager dumpStatistics <string>fileNames
Figure Output of the Dumpstatistics option for escalation shows the sample output for the Dumpstatistics option for escalation. The following values are included as a total for all escalation paths and for each level in each path:
Number of executed jobs. A job is a scheduled notification that matched a policy-path and that repeats for each level in that path. The number of executed jobs is indicated for all levels in all paths (overall) and for each level.
Total, maximum, and average execution times. Execution time indicates how long the tool actions defined for a job took in seconds. If the maximum execution time at a level far exceeds the average, it may indicate an issue for a particular tool.
Total, maximum, and average queue delays for all the escalation paths is also displayed. These delays are not related to execution time. Delay is difference between when a job was scheduled to execute and when the job was actually executed. A positive value for the delay indicates that the system is not keeping up with scheduled escalation. Delay may be caused by the overall load on the server, number of notifications, policies, paths, or levels. When a burst of notifications make it into a path, all jobs are scheduled to execute at the same time in a future time, so delay may increase due to these bursts.
Increase worker threads for escalation provides information on improving escalation performance.