NSX components write to log files in the directory /var/log. On NSX appliances, NSX syslog messages conform with RFC 5424. On ESXi hosts, syslog messages conform with RFC 3164.
Viewing Logs
On NSX appliances syslog messages are in /var/log/syslog.
get log-file <auth.log | controller | controller-error | http.log | kern.log | manager.log | node-mgmt.log | syslog> [follow]
| Name | Description |
|---|---|
| auth.log | Authorization log |
| controller | Controller log |
| controller-error | Controller error log |
| http.log | HTTP service log |
| kern.log | Kernel log |
| manager.log | Manager service log |
| node-mgmt.log | Node management log |
| nsx-audit-write.log | NSX audit write log |
| nsx-audit.log | NSX audit log |
| syslog | System log |
NSX also sends audit (audispd) logs directly to the remote syslog server. These messages are logged in /var/log/audit/audit.log but not in the local syslog file. Audit logs record security-related events in the operating system.
On hypervisors, you can use Linux commands such as tac, tail, grep, and more to view the logs.
Each syslog message has the component (comp) and sub-component (subcomp) information to help identify the source of the message.
NSX produces logs with facility local6, which has a numerical value of 22.
The audit log is part of syslog. An audit log message can be identified by the string audit="true" in the structured-data field. You can configure an external log server to receive log messages. You can also access audit logs using the API /api/v1/administration/audit-logs. The file nsx-audit.log contains syslog messages with audit="true" in the structured-data field. The file nsx-audit-write.log contains syslog messages with both audit="true" and update="true" in the structured-data field.
<182>1 2020-05-05T00:29:02.900Z nsx-manager1 NSX 147324 - [nsx@6876 audit="true" comp="nsx-manager" level="INFO" reqId="fe75651d-c3e7-4680-8753-9ae9d92d7f0c" subcomp="policy" username="admin"] UserName="admin", ModuleName="AAA", Operation="GetCurrentUserInfo", Operation status="success"
2023-01-25T21:58:42.173Z w4-mydevice-220.eng.vmware.com NSX 1463241016 - [nsx@6876 comp="nsx-controller" level="INFO" subcomp="corfu-cluster"] Received maintenance mode status: {da3a0242-bf6d-ee98-224c-b9799cb5e29f=MAINTENANCE_MODE_OFF}
- An entity ID parameter entId to identify the object of the API.
- A request ID parameter req-id to identify a specific API call.
- An external request ID parameter ereqId if the API call contains the header X-NSX-EREQID:<string>.
- An external user parameter euser if the API call contains the header X-NSX-EUSER:<string>.
- An update flag that shows whether the API operation is a read (GET) or write (PUT/POST/DELETE/...) operation.
- An operation name field that shows the name of the API operation.
- An operation status field that shows whether the API operation succeeded or failed.
- A new value field that shows all parameter values of the API request.
NSX does not have the concept of a privileged mode. API calls from all sources and users are audited.
2020-07-07T16:33:20.339Z svc.nsxmanager NSX 1513 SYSTEM [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="http"] UserName="[email protected]", ModuleName="ACCESS_CONTROL", Operation="LOGIN", Operation status="success" 2020-07-07T16:33:58.779Z svc.nsxmanager NSX 1513 SYSTEM [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="http"] UserName="admin", ModuleName="ACCESS_CONTROL", Operation="LOGOUT", Operation status="success" 2020-07-07T16:50:21.301Z svc.nsxmanager NSX 1513 SYSTEM [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="http"] UserName="[email protected]", ModuleName="ACCESS_CONTROL", Operation="LOGIN", Operation status="success" 2020-07-07T16:43:20.339Z svc.nsxmanager NSX 1513 SYSTEM [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="http"] UserName="[email protected]", ModuleName="ACCESS_CONTROL", Operation="LOGIN", Operation status="failure"
<182>1 2020-07-06T18:09:14.210Z svc.nsxmanager NSX 2326 FABRIC [nsx@6876 audit="true" comp="nsx-manager" entId="68d5a9d0-4691-4c9c-94ed-64fd1c96150f" level="INFO" reqId="4c2335aa-c973-4f74-983f-331a4f7041ca" subcomp="manager" update="true" username="admin"] UserName="admin", ModuleName="TransportZone", Operation="CreateTransportZone", Operation status="success", New value=[{"transport_type":"OVERLAY","host_switch_name":"nsxvswitch","host_switch_mode":"STANDARD","nested_nsx":false,"is_default":false,"display_name":"1-transportzone-1307","_protection":"UNKNOWN"}]
2020-07-07T16:36:41.783Z svc.nsxmanager NSX 21018 - [nsx@6876 comp="nsx-manager" subcomp="cli" username="admin" level="INFO"] NSX CLI started (Manager, Policy, Controller) for user: admin 2020-07-07T16:36:53.469Z svc.nsxmanager NSX 21018 - [nsx@6876 comp="nsx-manager" subcomp="cli" username="admin" level="INFO"] NSX CLI stopped for user: admin
<182>1 2020-07-22T20:51:49.017Z manager2 NSX 1864 - [nsx@6876 comp="nsx-manager" subcomp="cli" username="admin" level="INFO" audit="true"] CMD: set user admin password-expiration 100 (duration: 2.185s), Operation status: CMD_EXECUTED
<182>1 2020-07-21T21:01:38.803Z manager2 NSX 4690 - [nsx@6876 comp="nsx-manager" subcomp="node-mgmt" username="admin" level="INFO" audit="true"] admin 'GET /api/v1/node/services/syslog/exporters' 200 731 "" "PostmanRuntime/7.26.1" 0.004588
<182>1 2020-07-21T20:54:40.018Z manager2 NSX 16915 - [nsx@6876 comp="nsx-manager" subcomp="cli" username="admin" level="INFO" audit="true"] CMD: set logging-server 1.1.1.1 proto udp level info (duration: 4.356s), Operation status: CMD_EXECUTED
RFC 5424 and RFC 3164 define the following severity levels:
| Severity Level | Description |
|---|---|
| 0 | Emergency: system is unusable |
| 1 | Alert: action must be taken immediately |
| 2 | Critical: critical conditions |
| 3 | Error: error conditions |
| 4 | Warning: warning conditions |
| 5 | Notice: normal but significant condition |
| 6 | Informational: informational messages |
| 7 | Debug: debug-level messages |
All logs with a severity of emergency, alert, critical, or error contain a unique error code in the structured data portion of the log message. The error code consists of a string and a decimal number. The string represents a specific module.
Failure to Access a Log File or Remote Log Server
If NSX fails to access or write messages to a log file, an alarm will be generated. The possible errors are:
- A local log file is missing.
- A local log file's permission or ownership setting prevents NSX from writing to the file.
- NSX is unable to send log messages to a third-party remote log server. Note that an alarm will not be raised if NSX fails to send logs to the Aria Operations for Logs agent.
The alarm can be resolved through the alarm framework.
Log Message Formats
For more information about RFC 5424, see https://tools.ietf.org/html/rfc5424. For more information about RFC 3164, see https://tools.ietf.org/html/rfc3164.
RFC 5424 defines the following format for log messages:
<facility * 8 + severity> version UTC-TZ hostname APP-NAME procid MSGID [structured-data] msg
<187>1 2016-03-15T22:53:00.114Z nsx-manager NSX - SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP4039" subcomp="manager"] Connection verification failed for broker '10.160.108.196'. Marking broker unhealthy.
Error Codes
For a list of error codes, see the knowledge base article 71077 NSX-T Data Center 2.x Error Codes.
