You can configure the vRealize Log Insight Windows agent to collect log events from one or more log files.

Field names are restricted. The following names are reserved and cannot be used as field names.

  • event_type
  • hostname
  • source
  • text

You can have up to three destinations for agent information and filter the information before it is sent. See Forwarding Logs from a vRealize Log Insight Agent.

Note:
  • Monitoring a large number of files, such as a thousand or more, leads to higher resource utilization by the agent and impacts the overall performance of the host machine. To prevent this, configure the agent to monitor only the necessary files using patterns and globs, or archive the old log files. If monitoring a large number of files is a requirement, consider increasing the host parameters such as CPU and RAM.
  • The agent can collect from encrypted directories, but only if it is run by the user who encrypted the directory.
  • The agent supports only static directory structures. If the directories have been renamed or added, you must restart the agent to start monitoring these directories, provided the configuration covers the directories.

Prerequisites

Log in to the Windows machine on which you installed the vRealize Log Insight Windows agent and start the Services manager to verify that the vRealize Log Insight agent service is installed.

Procedure

  1. Navigate to the program data directory of the vRealize Log Insight Windows agent.
    %ProgramData%\VMware\Log Insight Agent
  2. Open the liagent.ini file in any text editor.
  3. Locate the [server|<dest_id>] section of the file. Add configuration parameters and set the values for your environment. 
    [filelog|section_name]
directory=path_to_log_directory
include=glob_pattern
...
    Parameter Description
    [filelog|section_name] A unique name for the configuration section.
    directory=full-path-to-log-file The full path to the log file directory. Glob patterns are supported. Example configurations:
    • To collect from all sub-directories of D:\Logs\new_test_logsdirectory, use directory=D:\Logs\new_test_logs\*
    • If your sub-directories have their own sub-directories, use the following configuration to monitor all sub-directories directory=D:\Logs\new_test_logs\*\*
    Note: To limit the number of files and directories and avoid high resource consumption, you cannot define a directory glob for either the first or second level directories such as: directory=c:/tmp/* or directory=c:\Logs\*. The directory path must be at least two levels.

    You can define a path to a non-existing directory, and the agent will collect the log files in that directory once the directory and files are created.

    You can define the same directory under one or more different configuration sections, to collect logs from the same file multiple times. This process makes it possible to apply different tags and filters to the same source of events.
    Note: If you use identical configurations for these sections, duplicated events are observed on the server side.
    include=file_name; ... (Optional) The name of a filename or a file mask (glob pattern) from which to collect data. You can provide values as a semicolon separated list. The default value is *, which means that all files are included. The parameter is case-sensitive.

    A file mask (glob pattern) can be used to group files that follow the same naming convention, as well as within a single filename. For example, filenames that include spaces, such as vRealize Ops Analytics.log and vRealize Ops Collector.log, can be specified with vRealize?Ops?Analytics*.log or vRealize*.log. By using file masks, you can specify filenames that are acceptable for agent configuration under Linux and Windows hosts.

    By default .zip and .gz files are excluded from collection.

    Important: If you are collecting a rotated log file, use the include and exclude parameters to specify a glob pattern that matches both the primary and the rotated file. If the glob pattern matches only the primary log file, the vRealize Log Insight agents might miss events during rotation. The vRealize Log Insight agents automatically determine the correct order of rotated files and sends events to the vRealize Log Insight server in the right order. For example, if your primary log file is named myapp.log and rotated logs are myapp.log.1 and myapp.log.2 and so on, you can use the following include pattern:

    include= myapp.log;myapp.log.*

    exclude=regular_expression (Optional) A filename or file mask (glob pattern) to exclude from collection. You can provide values as a semicolon separated list. The default value is empty, which means that no file is excluded.
    event_marker=regular_expression (Optional) A regular expression that denotes the start of an event in the log file. If omitted defaults to newline. The expressions you type must use the Perl regular expressions syntax.
    Note: Symbols, for example quotation marks ( " "), are not treated as wrappers for regular expressions. They are treated as part of the pattern.

    Since the vRealize Log Insight agent is optimized for real-time collection, partial log messages written with an internal delay may be split into multiple events. If log file appending stops for more than 200 ms without a new observed event_marker, the partial event is treated as complete, parsed, and delivered. This timing logic is non-configurable and has priority over the event_marker setting. Log file appenders should flush full events.

    enabled=yes|no (Optional) A parameter to activate or deactivate the configuration section. The possible values are yes or no. The default value is yes.
    charset=char-encoding-type (Optional) The character encoding of the log files that the agent monitors. Possible values are:
    • UTF-8
    • UTF-16LE
    • UTF-16BE
    The default value is UTF-8.
    tags={"tag-name" : "tag-value", ...}

    (Optional) A parameter to add custom tags to the fields of collected events. Define tags using JSON notation. Tag names can contain letters, numbers, and underscores. A tag name can only begin with a letter or an underscore and cannot exceed 64 characters. Tag names are not case-sensitive. For example, if you use tags={"tag_name1" : "tag value 1", "Tag_Name1" : "tag value 2" }, Tag_Name1 is ignored as a duplicate. You cannot use event_type and timestamp as tag names. Any duplicates within the same declaration are ignored.

    If the destination is a syslog server, tags can override the APP-NAME field. For example, tags={"appname":"VROPS"}.

    exclude_fields (Optional) A parameter to exclude individual fields from collection. You can provide multiple values as a semicolon- or comma-separated list. For example,
    • exclude_fields=hostname; filepath
    • exclude_fields=type; size
    • exclude_fields=type, size
    raw_syslog=Yes|No For agents that use the syslog protocol, this option allows the agent to collect and send raw syslog events. The default is No, which means collected events are transformed with user-specified syslog attributes. Activate this option to collect events without any syslog transformations.

Example: Configurations

[filelog|vCenterMain]
directory=C:\ProgramData\VMware\VMware VirtualCenter\Logs
include=vpxd-*.log
exclude=vpxd-alert-*.log;vpxd-profiler-*.log
event_marker=^\d{4}-\d{2}-\d{2}[A-Z]\d{2}:\d{2}:\d{2}\.\d{3} 
[filelog|ApacheAccessLogs]
enabled=yes
directory=C:\Program Files (x86)\Apache Software Foundation\Apache2.2\logs
include=*.log
exclude=*_old.log
tags={"Provider" : "Apache"}
[filelog|MSSQL]
directory=C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\Log
charset=UTF-16LE 
event_marker=^[^\s]