This section explains the vRealize Log Insight of the Avi Load Balancer.
Sending Data to vRealize Log Insight
Events are used throughout the Avi Load Balancer Controller to provide a history of relevant changes. Events are a permanent record and can be used to generate alerts for appropriate action to be taken. For more information regarding events, see Events.
Virtual services and pools can log client-to-application interactions for TCP and UDP connections, providing valuable insights into both client requests and server responses. These virtual service client logs can be indexed, viewed, and filtered locally within the Avi Load Balancer Controller.
Both types of data can be crucial for troubleshooting a deployment, along with a general sense of what is occurring within your deployment. Although all this data is available from the Avi Load Balancer Controller UI, there is an option to send this data externally to a centralized platform. This document provides some examples of how to send this data into vRealize Log Insight.
Sending Events to Log Insight
The events can be sent from the Avi Load Balancer Controller as Syslog messages. In the example diagram provided, a Log Insight Log Forwarder is being used, and this Log Forwarder is configured as a Syslog destination within the Avi Load Balancer Controller configuration.
For more information, see Notifications.
vRealize Log Insight Content Pack
For events that are sent into Log Insight, the VMware Avi Load Balancer (by VMware) content pack, available on the VMware Marketplace, will perform by key extractions. The content pack is used only for events.
Along with the key extraction, the content pack provides pre-built dashboards to visually display the event data. The dashboard queries use the extracted key: values.
Below is an example of Key Extraction.
Sending Virtual Service Client Logs to Log Insight
Avi Load Balancer supports the ability to stream the Virtual Service Client Logs directly from the Service Engines to an external server. For more information, see Streaming NSX Advanced Load Balancer Client Logs to an External Server.
Although these logs can be configured to be sent directly from the Service Engine to a Log Insight Log Forwarder like the events design, this direct workflow is not ideal. The limitation here is that the client logs are sent in JSON format. If it is sent directly to the Log Forwarder, there will be no key extraction, thus making querying the client logs more difficult. It is recommended to send the logs through an intermediate proxy that will perform JSON parsing.
Below are two examples of sending the client logs into Log Insight with JSON parsing. The first example uses the Log Insight Agent, and the second uses Fluent Bit.
Using the Log Insight Agent with JSON Parsing
The Log Insight Agent provides the ability to read local log files, parse the JSON data and then forward them onto a Log Forwarder. Running a Log Insight Agent on a Syslog server, the agent is configured for the log files to read and parse.
In this example, the Service Engines are configured to stream the virtual service client logs to an external server running Syslog, in this case, rsyslog is used. The Log Insight Agent is installed on this external Syslog server as well, configured to parse JSON with specific files and forward them to the Log Forwarder.
The following is the configuration file examples for both rsyslog and the Log Insight Agent.
The following changes can be added to the /etc/rsyslog.conf.
# provides UDP syslog reception module(load="imudp") input(type="imudp" port="514") ## rules for process the remote logs # $template noTimestampFormat,"%msg%\n" $template RemoteLogs,"/var/log/avi/%HOSTNAME%.log" *.* ?RemoteLogs;noTimestampFormat
Example liagent.conf Configuration:
[server] hostname=<loginsight_forwarder> ssl=no [filelog|avi] directory=/var/log/avi/ include=*.log exclude=liagent parser=json_parser [parser|json_parser] base_parser=json fields=*
Using the Fluent Bit with JSON Parsing
The Fluent Bit provides the ability to receive logs on a specified port, parse the JSON data and then forward them onto a Log Forwarder.
In this example, the SEs have been configured to stream the virtual service client logs to an external server running Fluent Bit. The Fluent Bit configuration includes adding a new entry to the parsers.conf file. The Fluent Bit will receive the virtual service client logs on the specified port, parse the JSON and then forward it to the Log Forwarder.
The following configuration can be added to /etc/td-agent-bit/parsers.conf.
[PARSER]
Name avi
Format json
Time_Keep Off
Example /etc/td-agent-bit/td-agent-bit.conf configuration:
[INPUT]
Name syslog
Listen 0.0.0.0
Port 514
Mode udp
Parser avi
Buffer_Chunk_Size 1024000
Buffer_Max_Size 2048000
Tag avi.logs
[OUTPUT]
Name http
Match *
Host <loginsight_forwarder>
Port 9000
URI /log-forwarder/ingest
Format json
