Learn how VMware Tanzu Mission Control logs events that occur in your organization.
To help you monitor the activities that occur in your organization, Tanzu Mission Control provides logging of audit events.
This includes audit logging of service-level actions that are performed through Tanzu Mission Control, as well as cluster-level actions that take place on Tanzu Kubernetes clusters that you provisioned through Tanzu Mission Control.
Tanzu Mission Control also logs the prior three days of cluster-level interactions and system state changes.
Those changes may be the creation, deletion, or modification of a cluster, cluster group, or namespace. It also covers actions such as running inspections, applying policies, and more.
Cluster Management Events
Tanzu Mission Control generates events for user activity and system state change, including cluster-level interactions that occur between Tanzu Mission Control and your managed clusters. The Events page in the Tanzu Mission Control console shows the last 72 hours of events across your entire fleet of Kubernetes clusters.
The kinds of events that you'll see on this page include cluster health and lifecycle events, inspections, as well as policy insights. In addition to the event summaries listed on the Events page, you can expand each event to view the body of the event or click the link to go to the object where it occurred. This makes it easier to access audit log events on the Tanzu Mission Control console along with regular cluster events to provide a comprehensive view.
You can filter the list of events by name and type to see only the events you want to see using the rich filter capability in Tanzu Mission Control. In addition, specific audit events are available from the Tanzu Mission Control console, without the need to request the download of an audit log file and subsequent offline processing in a log viewer to see the content.
Tanzu Mission Control Service Audit Logging
As an organization administrator, you can monitor the activities that are initiated through Tanzu Mission Control, using the service audit logging capability.
- what was done
- who performed the action
- when and where the action occurred
Because some actions result in subsequent actions, Tanzu Mission Control logs all pertinent events from the original action performed through Tanzu Mission Control to the resulting actions that are initiated on your clusters.
- Tanzu Mission Control event log entries are retained in the service for 60 days, so you can generate an audit report going back up to 60 days.
- The maximum date range for an audit report is seven days.
- Generated audit reports are retained for 60 days after generation, after which they are deleted.
Audit Logging in Your Provisioned Cluster
When you provision a new Tanzu Kubernetes cluster through Tanzu Mission Control, preconfigured audit logging is enabled for the cluster. Although the audit logging policy is not configurable, logging is set to commonly useful levels for certain operations to capture important information without inflating the size of the logs. The logging levels are listed below with the types of operations that use each one.
- kube-proxy watching endpoint resource
- kubelet getting nodes and node status
- Controller manager, scheduler and endpoint controller reading the endpoints
- API server getting namespaces and namespace finalizers
- Controller manager getting metrics
- Read-only APIs like /swagger, /version, and /healthz
- All getting events
- Garbage collector getting resources
- Tanzu Mission Control getting resources
- Config maps, secrets, and token reviews - because they can contain sensitive data
- kubelet updating nodes and node status
- All read-only operations - log the request, but not the data returned back
- All other operations, including write operations like create, update, and delete
- Audit log entries are written to audit.log files in the /var/log/kubernetes/ directory on the control plane nodes for your cluster. You can aggregate the logs, if desired, using a log collector and a log aggregation service.
- The maximum size for a log file is 100 MB. When this limit is reached, a new log file is started. The rotated files are also stored in the /var/log/kubernetes/ directory.
- The maximum number of log files stored on a node is 20 (for a total max size of 2 GB for all stored logs). When this limit is reached, a new log file replaces the oldest log file.
- A log file is stored for a maximum of 7 days, after which it is deleted. This limit is not often hit on a typical cluster, as clusters running workloads tend to churn through the maximum number of log files more quickly.
Import Logs into Your Preferred Logging System
You can can ingest Tanzu Mission Control events, including audit logs, via an event stream API, a secure and scalable solution. For instructions on how to set up automated ingestion of events using the streaming API, visit this VMware developer resource.
Best Practices for Naming Resource Fields
Users are advised to not enter any Personally Identifying Information (PII) into any of the resource name fields such as cluster name, cluster group name, workspace name, or namespace name. These fields can be customized as text fields by users for easy reference but are not meant to include any PII information such as user name, email address, or any such identifying information.
It is strongly recommended as a best practice to avoid any accidental exposure of any PII entered into these non-PII fields by the users who have access to Tanzu Mission Control in their organization. These fields are handled as per rules applicable to non-PII fields and any PII info entered into these non-PII fields is at the sole discretion of the user understanding the potential risk involved.
