vRealize Log Insight sends two types of email notifications, system notifications and user-defined notifications.

Administrators can configure vRealize Log Insight to send email notifications when certain events occur in the system. The from address of system notification emails is configured by the Administrator user on the SMTP configuration page of the Administration UI, in the Sender text box. See Configure the SMTP Server for vRealize Log Insight.

Administrator users can also configure vRealize Log Insight to send notification emails when the storage capacity drops below a defined threshold.

Every vRealize Log Insight user can create alert queries to receive email notifications from vRealize Log Insight when certain criteria are met.

Administrator users can disable all user-defined notifications.

The following table lists vRealize Log Insight system notifications that you can use.

Type

notification Name

Description

System

Oldest Data Will Be Unsearchable Soon

This notification notifies you when vRealize Log Insight is expected to start decommissioning old data from the virtual appliance storage and what the expected size of searchable data is at the current ingest rate. Data that has been rotated out is archived if you have configured archiving, or deleted if you have not.

The notification is sent after each restart of the vRealize Log Insight service.

System

Repository Retention Time

This notification notifies you about the amount of searchable data that vRealize Log Insight can store at the current ingest rates and in the storage space that is available on the virtual appliance. Admin users can define the storage notification threshold. See Configure vRealize Log Insight System Notifications to Send Email Messages.

System

Dropped Events

This notification notifies you that vRealize Log Insight failed to ingest all incoming log messages.

  • In case of any TCP Message drops, as tracked by vRealize Log Insight server, a system notification is sent in both cases as follows:

    • Once a day

    • Each time the vRealize Log Insight service is restarted, manually or automatically.

  • The email contains the number of messages dropped since last notification email was sent and total message drops since the last restart of vRealize Log Insight.

Note:

The time in the sent line is controlled by the email client, and is in the local time zone, while the email body displays UTC time.

System

Corrupt Index Buckets

This notification notifies you that part of the on-disk index is corrupt. A corrupt index usually indicates serious issues with the underlying storage system. The corrupt part of the index will be excluded from serving queries. A corrupt index affects the ingestion of new data. vRealize Log Insight checks the integrity of the index upon service start-up. In case of detected corruption, vRealize Log Insight sends a system notification as follows:

  • Once a day

  • Each time the vRealize Log Insight service is restarted, manually or automatically.

System

Out Of Disk

This notification notifies you that vRealize Log Insight is running out of allocated disk space. This notification signals that vRealize Log Insight has most probably run into a storage-related issue.

System

Archive Space Will Be Full

This notification notifies you that the disk space on the NFS server used for archiving vRealize Log Insight data will be used up soon.

System

Archive Failure

This notification notifies you that an operation of archiving vRealize Log Insight data to the NFS server has failed. This usually means that vRealize Log Insight is having trouble connecting to or writing to the NFS server.

System

Total Disk Space Change

This notification notifies you that the total size of the partition for vRealize Log Insight data storage has decreased. This usually signals a serious issue in the underlying storage system. When vRealize Log Insight detects the condition it sends this notification as follows:

  • Immediately

  • Once a day

System

Pending Archivings

This notification notifies you that vRealize Log Insight cannot archive data as expected. The notification usually indicates problems with the NFS storage that you configured for data archiving.

System

License is about to be expired

This notification notifies you that the license for vRealize Log Insight is about to expire.

System

License is expired

This notification notifies you that the license for vRealize Log Insight has expired.

System

Unable to connect to AD server

This notification notifies you that vRealize Log Insight is unable to connect to the configured Active Directory server.

System

Cannot take over High Availability IP address [IP Address] as it is already held by another machine

This notification notifies you that the vRealize Log Insight cluster was unable to take over the configured IP Address for the Integrated Load Balancer (ILB). The most common reason for this notification is that another host within the same network holds the IP address, and therefore the IP address is not available to be taken over by the Log Insight cluster.

You can resolve this conflict by either releasing the IP address from the host that currently holds it, or configuring Log Insight Integrated Load Balancer with a Static IP address that is available in the network. When changing the ILB IP address, remember to reconfigure all clients to send logs to the new IP address, or to a FQDsN/URL that resolves to this IP address. You must also unconfigure and reconfigure every vCenter Server integrated with Log Insight from the vSphere integration page.

System

High Availability IP address [IP Address] is unavailable due to too many node failures

This notification notifies you that the IP Address configured for the Integrated Load Balancer (ILB) is unavailable. This means that clients trying to send logs to a Log Insight cluster via the ILB IP address or a FQDN/URL that resolves to this IP address will see it as unavailable. The most common reason for this notification is that a majority of the nodes in the Log Insight cluster are unhealthy, unavailable, or unreachable from the master node. Another common reason is that NTP time synchronization has not been enabled, or the configured NTP servers have significant time drift between each other. You can confirm that the problem is still ongoing by trying to ping (if allowed) the IP address to verify that it is not reachable. You can resolve this problem by ensuring a majority of your cluster nodes are healthy and reachable, and enabling NTP time synchronization to accurate NTP servers.

System

Too many migrations of High Availability IP address [your IP Address] between vRealize Log Insight nodes

This notification notifies you that the IP address configured for the Integrated Load Balancer (ILB) has migrated too many times within the last 10 minutes. Under normal operation, the IP address rarely moves between Log Insight cluster nodes. However, the IP address might move if the current owner node is restarted or put in maintenance. The other reason can be lack of time synchronization between Log Insight cluster nodes, which is essential for proper cluster functioning. In case of latter, you can fix the problem by enabling NTP time synchronization to accurate NTP servers.

System

SSL Certificate Error

This notification notifies you that a syslog source has initiated a connection to vRealize Log Insight over SSL but ended the connection abruptly. This may indicate that the syslog source was unable to confirm the SSL certificate's validity. In order for vRealize Log Insight to accept syslog messages over SSL, a certificate that is validated by the client is required and the clocks of the systems must be synchronized. There may be an issue with the SSL Certificate or with the Network Time Service.

You can validate that the SSL Certificate is trusted by your syslog source, reconfigure the source not to use SSL, or reinstall the SSL Certificate. See Configure the vRealize Log Insight Agent SSL Parameters and Install a Custom SSL Certificate.

System

vCenter collection failed

This notification notifies you that vRealize Log Insight is unable to collect vCenter events, tasks, and alarms. To look for the exact error that caused the collection failure and to see if collection is working currently, look in the /storage/var/loginsight/plugins/vsphere/li-vsphere.log file.

System

Event Forwarder Events Dropped

This system notification is sent when a forwarder drops events because of connection or overload issues.

Example:

Log Insight Admin Alert: Event Forwarder Events Dropped 
This alert is about your Log Insight installation on https://<your_url>

Event Forwarder Events Dropped triggered at 2016-08-02T18:41:06.972Z

Log Insight just dropped 670 events for forwarder target 'Test',
reason: Pending queue is full.

System

Alert Queries Behind Schedule

This notification notifies you that vRealize Log Insight was unable to run a user alert at its configured time. The reason for the delay may be because of one or more inefficient user alerts or because the system is not properly sized for the ingestion and query load.

System

Auto Disabled Alert

If an alert has run at least ten times and its average run time is more than one hour, then the alert is deemed to be inefficient and is disabled to prevent impacting other user alerts.

System

Inefficient Alert Query

If an alert takes more than one hour to complete, then the alert is deemed to be inefficient.

User Defined

Alert Queries

This alert notifies you that a query returned results that match the criteria that you have set for the alert. Every user can define alert queries that send email notifications when certain criteria are met.

See Add an Alert Query in Log Insight to Send Email Notifications.