vRealize Log Insight provides scalable log aggregation and indexing for the vCloud Suite, including all editions of vSphere, with near real-time search and analytics capabilities.

vRealize Log Insight collects, imports, and analyzes logs to provide real-time answers to problems related to systems, services, and applications, and derive important insights.

High Performance Ingestion

vRealize Log Insight can process any type of log-generated or machine-generated data. It supports very high throughput rates and low latency and accepts data through syslog and the Ingestion API.

Scalability

vRealize Log Insight can scale out by using multiple virtual appliance instances. This enables linear scaling of the ingestion throughput, increases query performance and allows for ingestion high availability. In cluster mode, vRealize Log Insight provides master and worker nodes. Both master and worker nodes are responsible for a subset of data. Master nodes can query all subsets of data and aggregate the results.

Near Real-Time Search

Data ingested by vRealize Log Insight is available for search within seconds. Also, historical data can be searched from the same interface with the same low latency.

vRealize Log Insight supports complete keyword queries. Keywords are defined as any alphanumeric, hyphen, or underscore characters. In addition to the complete keyword queries, vRealize Log Insight supports glob queries (for example, erro?, vm*) and field-based filtering (for example, hostname does NOT match test*, IP contains "10.64"). Furthermore, log message fields that contain numeric values can be used to define selection filters (for example, CPU>80, 10<threads<100, and so on).

Search results are presented as individual events. Each event comes from a single source, but search results may come from multiple sources. You can use vRealize Log Insight to correlate the data on one or multiple dimensions (for example, time and request identifiers) providing a coherent view across the stack. This way, root cause analysis becomes much easier.

Windows and Linux Agents

vRealize Log Insight includes agents that collect events and files on Linux and Windows machines.

Intelligent Grouping

vRealize Log Insight uses a new machine learning technology. Intelligent Grouping scans incoming unstructured data and quickly groups messages together by problem type in order to give you the ability to rapidly understand issues that may span your physical, virtual, and hybrid cloud environments.

Aggregation

Fields that are extracted from log data can be used for aggregation. This is similar to the functionality that GROUP-BY queries provide in a relational database or pivot-tables in Microsoft Excel. The difference is that there is no need for extract, transform, and load (ETL) processes and vRealize Log Insight scales to any size of data.

You can generate aggregate views of the data and identify specific events or errors without having to to access multiple systems an applications between systems and applications. For example, while viewing an important system metric, for example the number of errors per minute, you can drill down to a specific time-range of events and examine the errors that occurred in the environment.

Runtime Field Extraction

Raw log data is not always easy to understand, and you might need to process some data to identify the fields that are important for searching and aggregation. vRealize Log Insight provides runtime field extraction to address this problem. You can dynamically extract any field from the data by providing a regular expression. The extracted fields can be used for selection, projection, and aggregation, similar to how the fields that are extracted at parse time are used.

Dashboards

You can create dashboards of useful metrics that you want to monitor closely. Any query can be turned into a dashboard widget and summarized for any range in time. You can check the performance of your system for the last five minutes, hour, or day. You can view a breakdown of errors by hour and observe the trends in log events.

Security Considerations

IT decision makers, architects, administrators, and others who must familiarize themselves with the security components of vRealize Log Insight must read the security topics in Administering vRealize Log Insight.

These topics provide concise references to the security features of vRealize Log Insight. Topics include the product external interfaces, ports, authentication mechanisms, and options for configuration and management of security features.