As an event ages, there are key aspects of event storage and management during the event life cycle to be aware of.

Event Storage

Each event is stored in a single on-disk bucket. When working with buckets, be aware of the following behaviors and characteristics.

  • Buckets can reach a maximum size of .5GB. When a bucket reaches .5GB, it is sealed and can no longer be written to and is marked as to be archived. After a sealed bucket is archived, it is marked as archived. This means an event may be retained locally and in the archives at the same time.

  • Buckets are not replicated across vRealize Log Insight nodes. If you lose a node then you lose the data on that node.

  • All buckets are stored on the /storage/core partition.

  • vRealize Log Insight deletes old buckets when available space on the /storage/core partition is less than 3%. Deletion is done using a FIFO model.

    Note:

    A near-full /storage/core partition is usual and expected. That partition should never reach 100% because vRealize Log Insight manages that partition. However, you should not attempt to store data on that partition as it may interfere with the old bucket delete process.

Event Management

Be aware of the following characteristics and behaviors of vRealize Log Insight events and event management.

  • After an event is deleted locally it can no longer be queried unless it is imported from the archive using the command-line interface.

  • After all events for a machine learning cluster are deleted from vRealize Log Insight, the cluster is removed.

  • vRealize Log Insight automatically rebalances all incoming events fairly across nodes in the cluster. For example, even if a node is explicitly sent to an event, it may not be the node to ingest the event.

  • Event metadata is stored in a proprietary format on a single vRealize Log Insight node and not in a database.

  • An event can exist locally on a node as well as on the archive.