Understanding how VMware Aria Operations for Logs processes messages and events is key to using VMware Aria Operations for Logs effectively.

The life cycle of a log message or event has multiple stages including reading, parsing, ingestion, indexing, alerting, query application, archiving, and deletion.

Events and messages transition through the following stages.

  1. It is generated on a device (outside of VMware Aria Operations for Logs).
  2. It is picked up and sent to VMware Aria Operations for Logs in one of the following ways:
    • By a VMware Aria Operations for Logs agent using ingestion API or syslog
    • Through a third-party agent such as rsyslog, syslog-ng, or log4j using syslog
    • By custom writing to ingestion API (such as log4j appender)
    • By custom writing to syslog (such as log4j appender)
  3. VMware Aria Operations for Logs receives the event.
    • If you are using the integrated load balancer (ILB), the event is directed to a single node that is responsible for processing the event.
    • If the event is declined, the client handles declines with UDP drops, TCP with protocol settings, or CFAPI with a disk-backed queue.
    • If the event is accepted, the client is notified.
  4. The event is passed through the VMware Aria Operations for Logs ingestion pipeline, from which the following steps occur:
    • A keyword index is created or updated. The index is stored in a proprietary format on a local disk.
    • Machine learning is applied to cluster events.
    • The event is stored in a compressed proprietary format on the local disk in a bucket.
  5. The event is queried.
    • Keyword and glob queries are matched against the keyword index.
    • Regex is matched against compressed events.
  6. The event is moved to a bucket and archived.
    • A bucket is sealed and archived when it reaches 0.5 GB.
  7. The event is deleted.
    • Buckets are deleted in FIFO order.

For More Information

For more information, see the VMware Technical Publications video,