Data archiving preserves old logs that might otherwise be removed from the vRealize Log Insight virtual appliance due to storage constraints. vRealize Log Insight can store archived data to NFS mounts.

About this task

vRealize Log Insight collects and stores logs on disk in a series of 1 GB buckets. A bucket consists of compressed log files and an index. A bucket contains everything necessary to perform queries for a specific time range. When the size of the bucket exceeds 1 GB in size, vRealize Log Insight stops writing, closes all files in the bucket and seals the bucket.

When data archiving is enabled, vRealize Log Insight copies the raw compressed log files from the bucket to an NFS mount when the bucket is sealed. Buckets that have been sealed prior to enabling data archiving are not retroactively archived.

The path created within an archive export is in the form year/month/day/hour/bucketuuid/data.blob, using the timestamp at which the bucket was originally created in UTC. For example, 1970/12/01/15/12345678-90ab-cdef-1234-567890abcdef/data.blob.

Note:

vRealize Log Insight does not manage the NFS mount used for archiving purposes. If system notifications are enabled, vRealize Log Insight sends an email when the NFS mount is about to run out of space or is unavailable. If the NFS mount does not have enough free space or is unavailable for a period of time greater than the retention period of the virtual appliance, vRealize Log Insight stops ingesting new data until the NFS mount has enough free space, becomes available, or archiving is disabled.

Prerequisites

  • Verify that you have access to an NFS partition that meets the following requirements.

    • The NFS partition must allow reading and writing operations for guest accounts.

    • The mount must not require authentication.

    • The NFS server must support NFS v3.

    • If using a Windows NFS server, allow unmapped user UNIX access (by UID/GID).

  • Verify that you are logged in to the vRealize Log Insight Web user interface as a user with the Edit Admin permission. The URL format is https://log-insight-host, where log-insight-host is the IP address or host name of the vRealize Log Insight virtual appliance.

Procedure

  1. Click the configuration drop-down menu icon and select Administration.
  2. Under Configuration, click Archiving.
  3. Select Enable Data Archiving and enter the path to an NFS partition where logs will be archived in the form nfs://servername/sharename.
  4. Click Test to verify the connection.
  5. Click Save.

Results

Note:

Data archiving preserves log events that have since been removed from the vRealize Log Insight virtual appliance due to storage constraints. Log events that have been removed from the vRealize Log Insight virtual appliance, but have been archived are no longer searchable. If you want to search archived logs, you must import them into a vRealize Log Insight instance. For more information about importing archived log files, see Import a vRealize Log Insight Archive into vRealize Log Insight.

What to do next

After vRealize Log Insight restarts, verify that syslog feeds from ESXi continue to arrive in vRealize Log Insight. For troubleshooting, see ESXi Logs Stop Arriving in Log Insight.