By using the right syslog settings, you can achieve proactive monitoring of your environment, reduce downtime and take preventive action on servers.
While setting up syslog, you need to consider several parameters that affect log file retention, syslog transmission, transmission length, error handling, and the set up of SSL certificates for secure syslog message transmission. What follows are recommendations for fine-tuning your syslog parameters. You can see a description of all available parameters at ESXi Syslog Options.
How to Specify Log File Retention
Manage Settings that Affect the Virtual Machine Log File
You can configure some settings that affect the virtual machine log file, vmware.log, either in the vmx file or in the /etc/vmware/config file. You must power off a virtual machine to edit the vmx file and edits take effect only on that virtual machine. If you use the /etc/vmware/config file, you must add the prefix "vmx" to the setting, for example
vmx.log.keepOld = "20", and edits affect all virtual machines on the ESXi host.
Disables all virtual machine logging.
|The default value is
logging = "TRUE"
To disable virtual machine logging:
|Do not use this setting, because disabling virtual machine logging makes it extremely difficult or impossible to get support for virtual machine problems. If you need to use this setting for some reason, you can only place it in the vmx file of a virtual machine.
Controls when a log file throttles. Log file throttling occurs when writes to the vmware.log exceed the specified rate for a significant amount of time. This occurs when code within the VMX process, which controls a virtual machine, creates excessive log messages. The default value for this setting is 1 KB/sec.
In case of log thottling, you see <<< Log Throttled >>> in the vmware.log file.
To disable log throttling, use
|Log file thottling might obscure information necessary to diagnose problems with the affected virtual machine. If you need to disable log throttling, place the line in the example in the vmx file of the affected virtual machine. Remove the line after the debugging session ends.
Controls the number of older vmware.log file to retain.
log.keepOld = "20"
|Do not put the value of this setting below the default value (10). If virtual machines are frequently modified or moved, consider raising this setting to 20 or more.
Controls the maximum size of a vmware.log file in bytes.
To disable limiting the maximum size of a vmware.log file, use
|A value of this setting below 100000 can cause a loss of critical log messages and affect virtual machine performance. In ESXi 7.x and earlier, the default value of this setting places no limit on the size of a vmware.log file. In ESXi 8.x and later, the default value of this setting is 2048000.
Controls the name and location of virtual machine log files.
This setting changes the name of the virtual machine log files from vmware.log to myVMlog.
This setting directs virtual machine log files to a directory on a different VMFS volume (vol1) by using myVM for a file name.
|Do not place a log file outside the virtual machine directory to make sure that the collection of host support bundles picks up the log file, which can be critical to debug virtual machine problems.
Controls the minimum level at which messages are written to vmware.log. Every log message has a level associated with it. Levels below the specified setting are not added to a virtual machine log file. The virtual machine message log levels (from most to least restricted) are:
log.fileLevel = "debug1"
Do not set the level to a more restrictive level than "info" to avoid filtering out messages that are necessary for debugging virtual machine problems. Lower the level below "info" only upon request by licensed support. Restore the setting to "info" after debugging ends.
Controls the output of specialized debugging messages.
log.filter.minLogLevel.disklib = "debug5"
|Use this setting only upon request by licensed support, who should provide one or more <groupName> parameters. Remove the setting after debugging ends.
Enables the sending of virtual machine log messages to the system logger of an ESXi host, such as the syslog.
log.syslogID = "vmx"
Use "vmx" as value for this setting to allow the ESXi syslog daemon, vmsyslogd, to send these messages to a separate log file.
Controls the minimum level at which messages are output to the system logger of an ESXi host, such as the syslog.
log.syslogLevel = "debug"
|The levels and functioning of this setting are identical to those for the log.fileLevel setting.
How to Specify Message Transmission to Remote Hosts
Use the Syslog.global.logHost setting to define remote host specifications. Separate multiple remote host specifications with a comma (,). After setting Syslog.global.logHost, ESXi hosts open and maintain connections to the syslog collectors, and the transmission of messages begins immediately. When ESXi generates a syslog message, it writes it to the appropriate log file on the ESXi host and also forwards it to all configured syslog collectors.
What follows is the syntax for Syslog.global.logHost remote host specifications:
|Specifies the networking protocol. Valid values are udp, tcp, and ssl.
|The ssl protocol specifies that transmission of syslog messages is encrypted. The tcp and udp protocols do not encrypt the transmission.
Note: If capturing syslog messages or audit messages is critical to your system, avoid using the udp protocol because the networking infrastructure external to ESXi might drop UDP messages.
Specifies the remote host. You can use either an IPV4 or IPV6 address, or a host name.
|When you use an IPV6 address, you must embed it in square brackets [xxx], where xxx is the IPV6 address.
|(Optional) Specifies the remote host port to use. If you use UDP or TCP, the default port is 1514. If you use SSL, the default port is 514. If you choose to use different ports from 514 or 1514, you must adjust the ESXi firewall to open the port.
|For details how to open the ESXi firewall for the port specified in each remote host specification, see Configuring the ESXi Firewall.
|Syslog.global.logHost string example
|Transmits syslog messages to 10.176.130.7 using TCP/IP and port 12345.
|Transmits syslog messages to an IPV6 address using port 1514.
|Transmit syslog messages to an IPV6 address using port 54321.
|Transmits syslog messages to company.com using UDP and port 514.
|Transmits syslog messages to two remote hosts. The first remote host uses UDP to communicate with company.com using port 514. The second remote host uses TCP to communicate with the IPV4 address 10.20.30.40 using port 1050.
|Transmits syslog messages to company.com using SSL (TLS) and port 514.
Maximum Message Transmission Length
If you use UDP, the maximum syslog message transmission length is 480 bytes for IPV4 and 1180 bytes for IPV6.
Considerations for Certificates When Configuring SSL Transmissions to Remote Hosts
Additional SSL Transmission Parameters
An ESXi system complying with security certification requirements might require enabling of X509 CRL checks. You turn on the advanced settings Syslog.global.certificate.strictX509Compliance and Syslog.global.certificate.checkCRL by changing the default value of false to true. Due to implementation limitations, if you enable CRL checks by using the setting Syslog.global.certificate.checkSSLCerts, then all certificates in a certificate chain must provide a CRL link. By default, the setting is enabled. You can deactivate SSL certificate checks by changing the setting the false, but this is not a best practice. You might need to turn off SSL certificate checks when troubleshooting communications with a remote host, but only do so for a limited time.
Where to Find Syslog Daemon Error and Status Information
The ESXi syslog daemon uses the log file /var/run/log/vmsyslogd.log to store status and error information, including dropped messages. If audit record transmission is enabled, the syslog daemon also emits audit records related to its operation, such as daemon start, stop, and error conditions, which allows you to verify that the syslog daemon runs properly.
How to Change the Default Syslog Log File Storage Area
The default syslog log file storage area is /var/run/log, local to each ESXi host. Use the Syslog.global.logDir syslog configuration variable to change the default syslog log file storage area, as long as the location resides on persistent storage. If Syslog.global.logDir is configured to a persistent store shared by multiple ESXi hosts to store their syslog log files, change the Syslog.global.logDirUnique setting to true to prevent mixing logs. The Syslog.global.logDirUnique setting makes sure that each ESXi machine gets a unique name added to the Syslog.global.logDir path, separating the log files from other hosts.
Syslog Message Queueing for Remote Hosts and Message Drops
Once syslog emissions start, they never stop except for ESXi reboots and failures, or a syslog reconfiguration to stop.
ESXi uses a queueing mechanism when sending syslog messages to remote hosts. This helps avoid message drops if network connectivity issues are encountered and resolved. However, if the connectivity issues last longer than the queueing mechanism can tolerate, syslog messages are dropped. You can see statistics about message drops in the syslog daemon log file.
You can see dropped messages at /var/run/log/vmsyslogd-dropped.log. This log file has retention settings specific to it, similar to those for the program-specific retention parameters. The dropped messages log file retention parameters are: Syslog.global.droppedMsgs.fileRotate and Syslog.global.droppedMsgs.fileSize.