vRealize Log Insight provides real-time log management and log analysis with machine learning-based intelligent grouping, high-performance searching, and troubleshooting across physical, virtual, and cloud environments.

Overview

vRealize Log Insight collects data from ESXi hosts using the syslog protocol. It connects to other VMware products, like vCenter Server, to collect events, tasks, and alarms data, and integrates with vRealize Operations Manager to send notification events and enable launch in context. vRealize Log Insight also functions as a collection and analysis point for any system capable of sending syslog data. In addition to syslog data an ingestion agent can be installed on Linux or Windows servers or may come pre-installed on certain VMware products to collect logs. This agent approach is especially useful for custom application logs and operating systems that don't natively support the syslog protocol, such as Windows.

Installation Models

You can deploy vRealize Log Insight as a virtual appliance in one of the following configurations:

  • Standalone node

  • Highly available cluster of one master and at least two worker nodes using an integrated load balancer (ILB)

The compute and storage resources of the vRealize Log Insight instances can scale-up as growth demands.

Cluster Nodes

For high availability and scalability, you can deploy several vRealize Log Insight instances in a cluster where they can have either of the following roles:

Master Node

Required initial node in the cluster. In standalone mode, the master node is initially responsible for all activities, including queries and log ingestion; however, after additional nodes have been deployed and configured in a vRealize Log Insight cluster for high availability (HA), these activities are delegated to all available nodes. After HA has been configured for a cluster, the master node still retains responsibility is for the lifecycle of a cluster, which includes performing upgrades, as well as adding and removing of Worker nodes. The master node stores logs locally. If master node is down, the logs on the master becomes unavailable.

Worker Node

Enables scale-out in larger environments. A worker node is responsible for queries and log ingestion. A worker node stores logs locally. If a worker node is down, the logs on that worker becomes unavailable. You need at least two worker nodes to form a cluster with the master node.

Integrated Load Balancer (ILB)

Provides high availability. The ILB runs on one of the cluster nodes. If the node that hosts the ILB Virtual IP (VIP) address stops responding, the VIP address is failed over to another node in the cluster.  All queries against data are directed to the ILB, in which the query request is delegated to a query master for the duration of the query, and in turn queries all nodes (both master and workers) for data, which then aggregates that data for handoff back to the client. The Web User Interface of the ILB serves as a single pane of glass, presenting data from multiple sources in the cluster in a unified display; while individual nodes can be accessed via their Web User Interfaces, unless you are performing specific administrative activities on the individual nodes, it is advised to use the ILB's Interface.

Architecture of a Cluster

The architecture of vRealize Log Insight enables several channels for HA collection of log messages.

Figure 1. Cluster Architecture of vRealize Log Insight


vRealize Log Insight interacts with the interface, with vRO, and with vRLI clients. Inside the vRLI cluster are content packs and the master and worker nodes

vRealize Log Insight clients connect to ILB VIP address, and use the Web user interface and ingestion (by using syslog or the Ingestion API) to send logs to vRealize Log Insight.

By default, the vRealize Log Insight collects data from vCenter Server systems and ESXi hosts. For forwarding logs from NSX for vSphere, and vRealize Automation, use content packs which contain extensions or provide integration with other systems in the SDDC.

Authentication Models

You can configure vRealize Log Insight user authentication to utilize one or more of the following authentication models:

  • Microsoft Active Directory

  • Local Accounts

  • VMware Identity Manager

Integration with vRealize Operations Manager

The integration with vRealize Operations Manager provides data from multiple sources to a central place for monitoring the SDDC. vRealize Log Insight sends notification events to vRealize Operations Manager. Once integrated, vRealize Operations Manager is able to provide the inventory map of any vSphere objects to vRealize Log Insight, allowing for you to perform launching of log messages from vRealize Log Insight to the vRealize Operations Manager Web user interface, taking you either directly to the object itself or the location of the object within the environment.

Integration with vSphere

The integration with vSphere provides data from multiple sources to a central place for monitoring the SDDC. vRealize Log Insight connects to vCenter Server in two-minute intervals, and collect events, alarms, and tasks data from these vCenter Server systems. Further, this integration can be used to configure the managed ESXi hosts within the vCenter Server to send their logs to the vRealize Log Insight instance.

Archiving

vRealize Log Insight supports data archiving on NFS shared storage that each vRealize Log Insight node can access. 

Backup

You back up each vRealize Log Insight cluster using traditional virtual machine backup solutions that use vSphere Storage APIs for Data Protection (VADP) compatible backup software such as vSphere Data Protection.

Multi-Region vRealize Log Insight Deployment

Using vRealize Log Insight in a multi-region design can provide a syslog infrastructure in all regions of the SDDC. Using vRealize Log Insight across multiple regions requires deploying a cluster in each region. vRealize Log Insight supports event forwarding to other vRealize Log Insight deployments across regions in the SDDC. Implementing failover by using vSphere Replication or disaster recovery by using Site Recovery Manager is not necessary. The event forwarding feature adds tags to log message that identify the source region and event filtering prevents looping messages between the regions.

Tagging

vRealize Log Insight provides a capability called tags. When logs are forwarded to a regional hub, each log can be tagged with metadata that allows you to track its origin. With the VMware Validated Design, these tags are used as location identifiers. Configuring these tags as part of the forwarder allows SLA dashboards to be created for high level monitoring and trend analysis at each site, identifying locations with unreliable connectivity or high site resource consumption.