To accommodate all log data from the products in the SDDC, you must size the compute resources and storage for the Log Insight nodes properly.

By default, the vRealize Log Insight virtual appliance uses the predefined values for small configurations, which have 4 vCPUs, 8 GB of virtual memory, and 510 GB of disk space provisioned. vRealize Log Insight uses 100 GB of the disk space to store raw data, index, metadata, and other information.

Sizing Nodes

Select a size for the vRealize Log Insight nodes to collect and store log data from the SDDC management components and tenant workloads according to the objectives of this design.

Table 1. Compute Resources for a vRealize Log Insight Medium-Size Node

Attribute

Specification

Appliance size

Medium

Number of CPUs

8

Memory

16 GB

Disk Capacity

510 GB (490 GB for event storage)

IOPS

1,000 IOPS

Amount of processed log data when using log ingestion

75 GB/day of processing per node

Number of processed log messages

5,000 event/second of processing per node

Environment

Up to 250 syslog connections per node

Sizing Storage

Sizing is based on IT organization requirements, but this design provides calculations according based on a single region implementation, and is implemented on a per-region basis. This sizing is calculated according to the following node configuration per region:

  • Management vCenter Server

    • Platform Services Controller

    • vCenter Server

  • Compute vCenter Server

    • Platform Services Controller

    • vCenter Server

  • Management, shared edge and compute ESXi hosts

  • NSX for vSphere for the management cluster and for the shared compute and edge cluster

    • NSX Manager

    • NSX Controller instances

    • NSX Edge instances

  • Event forwarding configured between vRealize Log Insight clusters

These components aggregate to approximately 210 syslog and vRealize Log Insight Agent sources. Assuming that you want to retain 7 days of data, use the following calculations:

For 210 syslog sources at a basal rate of 150 MB of logs ingested per-day per-source over 7 days, you need the following storage space:

210 sources * 150 MB of log data ≈ 31.5 GB log data per-day

31.5 GB * 7 days ≈ 220.5 GB log data per vRealize Log Insight node

220.5 GB * 1.7 indexing overhead ≈ 375 GB

Based on this example, the storage space that is allocated per medium-size vRealize Log Insight virtual appliance is enough to monitor the SDDC.

Consider the following approaches when you must increase the Log Insight capacity:

  • If you must maintain a log data retention for more than 7 days in your SDDC, you might add more storage per node by adding a new virtual hard disk. vRealize Log Insight supports virtual hard disks of up to 2 TB. If you must add more than 2 TB to a virtual appliance, add another virtual hard disk.

    When you add storage to increase the retention period, extend the storage for all virtual appliances.

    Note:

    Do not extend existing retention virtual disks. Once provisioned, do not reduce the size or remove virtual disks to avoid data loss.

  • If you must monitor more components by using log ingestion and exceed the number of syslog connections or ingestion limits defined in this design, you can deploy more vRealize Log Insight virtual appliances to scale out your environment. vRealize Log Insight can scale up to 12 nodes in an HA cluster.

Table 2. Compute Resources for the vRealize Log Insight Nodes Design Decisions

Decision ID

Design Decision

Design Justification

Design Implication

SDDC-OPS-LOG-003

Deploy vRealize Log Insight nodes of medium size.

Accommodates the number of expected Syslog and vRealize Log Insight Agent connections from the following. This is approximately 210 syslog and vRealize Log Insight Agent sources.

  • Management & Compute vCenter Server, Platform Services Controller

  • Management, shared edge and compute ESXi hosts, the

  • Management and compute components for NSX for vSphere

  • Cross-vRealize Log Insight cluster event forwarding.

This ensure the storage space for the vRealize Log Insight cluster is sufficient for 7 days of data retention.

You must increase the size of the nodes if you configure Log Insight to monitor additional syslog sources.