Being able to record and monitor the activities of users is an important part of overall system security. Most organizations have rules governing who is allowed to access and make changes to software and related hardware resources. Maintaining an audit log of significant activities enables the organization to verify compliance with rules, detect any violations, and initiate remediation activities. Some businesses are under external laws and regulations that require ongoing monitoring and verification of access and authorization rules.
An audit log can also be helpful in detecting attempts, whether successful or not, to gain illegitimate access to the system, probe its information, or disrupt its operation. Knowing an attack was attempted and the details of the attempt can help in mitigating the damage and preventing future attacks.
Whether or not it is required, it is part of good security practice to regularly examine logs for suspicious, unusual, or unauthorized activity. Routine log analysis will also help identify system misconfigurations and failures and help ensure adherence to SLAs.
- Diagnostic logs
- Diagnostic logs that are maintained in each cell's log directory. These logs can be useful for problem resolution but are not intended to preserve an audit trail of significant system interactions. Each vCloud Director cell creates several diagnostic log files described in Viewing the vCloud Director Logs in the vCloud Director Administrator's Guide.
- Audit logs
- Audit logs record significant actions, including login and logout. The system audit log is maintained in the vCloud Director database and can be monitored through the Web UI. Each Organization administrator and the system administrator have a view into the log scoped to their specific area of control.
We recommend using the syslog utility to preserve these and other vCloud Director logs. In addition, you should consider use of vRealize Log Insight, which supports remote collection of other logs such as request logs, which are not based on log4j.
Using Syslog with vCloud Director
- Database logs are not retained after 90 days, while logs transmitted via syslog can be retained as long as desired.
- It allows audit logs from all cells to be viewed together in a central location at the same time.
- It protects the audit logs from loss on the local system due to failure, a lack of disk space, compromise, and so on.
- It supports forensics operations in the face of problems like those listed above.
- It is the method by which many log management and Security Information and Event Management (SIEM) systems will integrate with vCloud Director. This allows:
- Correlation of events and activities across vCloud Director, vSphere, NSX, and even the physical hardware layers of the stack.
- Integration of cloud security operations with the rest of the cloud provider's or enterprise's security operations, cutting across physical, virtual, and cloud infrastructures.
- Logging to a remote system other than the system the cell is deployed on inhibits tampering with the logs. A compromise of the cell does not necessarily enable access to or alteration of the audit log information.
If you did not set up a syslog destination for logging at initial install time, you can configure it later by going to each cell, editing the $VCLOUD_HOME/etc/global.properties file, and restarting the cell.
See Network Security Requirements for a list of ports that must remain open from the vCloud Director host to the syslog server. The syslog server configuration details are system specific and outside the scope of this document. It is recommended that the syslog server be configured with redundancy, to ensure essential events are always logged.
The above discussion covers only sending the audit log to a syslog server. Security Operations and IT Operations organizations may also benefit from the centralized aggregation and management of the diagnostic logs mentioned above. There are a variety of methods for collecting those logs including scheduling a job to periodically copy them to a centralized location, setting an additional logger in the log4j.properties file ($VCLOUD_HOME/etc/log4j.properties) to a central syslog server, or using a log-collection utility such as vRealize Log Insight to monitor and copy the log files to a centralized location. The configuration of these options is dependent on which system you prefer to use in your environment and is outside the scope of this document.
We recommend using a TLS-enabled syslog infrastructure. The default (UDP) syslog protocol offers neither in-transit encryption nor transmission control/acknowledgement. The lack of encryption exposes log data to sniffing (the information present in logs could be used to launch further attacks), and the lack of transmission control could enable an attacker to tamper with logging data. For more information, see Section 4 of RFC 5426.
Diagnostic Logging and Log Rollover
The Jetty request log file ($VCLOUD_HOME/logs/yyyy_mm_dd.request.log) is programmatically controlled by the Jetty (HTTP) server, but does not come with a maximum size limit. For this reason, there is a risk of unbounded log file growth. A log entry is added to the current file for each HTTP request served by Jetty. For this reason, we recommend that you use logrotate or similar methods to control the size of logs and the number of old log files to keep.
The other diagnostic log files are limited to 400MB total. Ensure that you have sufficient free disk space to accommodate those files plus the size that you allow the Jetty request logs to consume. As mentioned above, centralized logging will ensure you don't lose valuable diagnostic information as the 400MB log file total is reached and files are rotated and deleted.
NTP and Logs
The vCloud Director Installation and Upgrade Guide identifies NTP as a requirement for all vCloud Director cells. A side benefit of using NTP is that log messages from all cells have synchronized timestamps. Certainly, log management tools and SIEM systems incorporate their own timestamps to help coordinate logs from multiple origins, but those timestamps are the time received by those systems, not the time the event was originally logged.
Other systems connected to and used by vCloud Director create audit logs that should be consolidated into your audit processes. These include logs from NSX Manager, the vCloud Director database, vCenter Server, and vSphere hosts. The details of each system's log files and their purpose is beyond the scope of this document and can be found in documentation related to those products.