To accommodate all log data from the products in the SDDC, you must size the compute resources and storage for the Log Insight nodes properly.

By default, the vRealize Log Insight virtual appliance uses the predefined values for small configurations, which have 4 vCPUs, 8 GB of virtual memory, and 530.5 GB of disk space provisioned. vRealize Log Insight uses 100 GB of the disk space to store raw data, index, metadata, and other information.

Sizing Nodes

Select a size for the vRealize Log Insight nodes so as to collect and store log data from the SDDC management components and tenant workloads according to the objectives of this design.

Table 1. Compute Resources for a vRealize Log Insight Small-Size Node

Attribute

Specification

Appliance size

Small

Number of CPUs

4

Memory

8 GB

Disk Capacity

530.5 GB (490 GB for event storage)

IOPS

500 IOPS

Amount of processed log data when using log ingestion

30 GB/day of processing per node

Number of processed log messages

2,000 event/second of processing per node

Environment

Up to 100 syslog connections per node

Sizing Storage

Sizing is usually based on IT organization requirements. However, this design provides calculations that are based on a single-region implementation, and is then implemented on a per-region basis. This sizing is calculated according to the following node configuration per region:

Table 2. Management Systems Whose Log Data Is Stored by vRealize Log Insight

Category

Logging Sources

Quantity

Consolidated cluster

vCenter Server with Embedded Platform Services Controller

1

ESXi Hosts

64

NSX for vSphere for the consolidated cluster

NSX Manager

1

NSX Controller Instances

3

NSX Edge services gateway instances:

  • Two ESGs for north-south routing

  • Distributed logical router

3

vRealize Automation

vRealize Proxy Agents

2

vRealize Business

Data Collector

1

vRealize Operations

Remote Collector nodes

2

These components provide approximately 77 syslog and vRealize Log Insight Agent sources.

If you want to retain 7 days of data, apply the following calculation:

vRealize Log Insight receives approximately 150 MB to 190 MB of log data per day per source as follows.

  • The rate of 150 MB of logs per day is valid for Linux where 170 bytes per message is the default message size.

  • The rate of 190 MB of logs per day is valid for Windows where 220 bytes per message is the default message size.

170 bytes per message * 10 messages per second * 86400 seconds per day = 150 MB of logs per day per source (Linux)
220 bytes per message * 10 messages per second * 86400 seconds per day = 190 MB of logs per day per source (Windows)

In this validated design, to simplify calculation, all calculations have been done using the large 220 byte size which results in 190 MB of log data expected per-day per-source.

For 77 logging sources, at a basal rate of approximately 190 MB of logs that are ingested per-day per-source over 7 days, you need the following storage space:

Calculate the storage space required for a single day for log data using the following calculation:

77 sources * 190 MB of logs per day per source * 1e-9 GB per byte ≈ 15 GB disk space per day

Based on the amount of data stored in a day, to size the appliance for 7 days of log retention, use the following calculation:

(15 GB * 7 days) / 3 аppliances ≈ 100 GB log data per vRealize Log Insight node

100 GB * 1.7 indexing overhead ≈ 170 GB log data per vRealize Log Insight node

Based on this example, the storage space that is allocated per small-size vRealize Log Insight virtual appliance is enough to monitor the SDDC.

Consider the following approaches when you must increase the Log Insight capacity:

  • If you must maintain a log data retention for more than 7 days in your SDDC, you might add more storage per node by adding a new virtual hard disk. vRealize Log Insight supports virtual hard disks of up to 2 TB. If you must add more than 2 TB to a virtual appliance, add another virtual hard disk.

    When you add storage to increase the retention period, extend the storage for all virtual appliances.

    When you add storage so that you can increase the retention period, extend the storage for all virtual appliances. To increase the storage, add new virtual hard disks only. Do not extend existing retention virtual disks. To avoid data loss, once provisioned, do not reduce the size or remove virtual disks .

  • If you must monitor more components by using log ingestion and exceed the number of syslog connections or ingestion limits defined in this design, you can do the following:

    • Increase the size of the vRealize Log Insight node, to a medium or large deployment size as defined in the vRealize Log Insight documentation.

    • Deploy more vRealize Log Insight virtual appliances to scale out your environment. vRealize Log Insight can scale up to 12 nodes in an HA cluster.

Table 3. Design Decisions on the Compute Resources for the vRealize Log Insight Nodes

Decision ID

Design Decision

Design Justification

Design Implication

ROBO-OPS-LOG-003

Deploy vRealize Log Insight nodes of small size.

Accommodates the number of expected syslog and vRealize Log Insight Agent connections from the following sources:

  • Consolidated vCenter Server and embedded Platform Services Controller

  • ESXi hosts in the consolidated cluster

  • NSX for vSphere components in the consolidated cluster

  • vRealize Automation components

  • vRealize Business components

  • vRealize Operations Manager components

These components approximately generate about 85 syslog and vRealize Log Insight Agent sources.

Using a small-size appliance ensures that the storage space for the vRealize Log Insight cluster is sufficient for 7 days of data retention.

You must increase the size of the nodes if you configure Log Insight to monitor additional syslog sources.