In 3.3.0 release, the SD-WAN Orchestrator stores only flow statistics with high resolution to provide visibility and troubleshooting capability. Starting from 3.3.2 release, SD-WAN Orchestrator supports retention of flow stats for upto one year by rolling up flow stats for every edge on a daily basis. Currently, the daily flow stats rollups is supported only for on-prem customers.

Aggregating Flow Statistics

The SD-WAN Orchestrator currently aggregates flow statistics from a higher resolution (every 5 minutes) to a ready-to-use form at a low resolution (every 24 hours). The following tables summarize the flow stats rollup and retention support information.

Table 1. Flow Stats Rollup Support
Resolution Rollup Pre 3.3.0 Rollup Post 3.3.0 Rollup Post 3.3.2
High 5 minutes 5 minutes 5 minutes
Medium 2 hours Deprecated Not Supported
Low 8 hours Deprecated 24 hours
Table 2. Flow Stats Retention Support
Resolution Retention Pre 3.3.0 Retention Post 3.3.0 for OnPrem Retention Post 3.3.0 for Hosted Retention Post 3.3.2 for OnPrem Retention Post 3.3.2 for Hosted
High 6-10 weeks 14 days (Default), 31 days (Maximum) 14 days 14 days 14 days
Medium 10 -14 weeks Deprecated Deprecated Deprecated Deprecated
Low Upto 1 year Deprecated Deprecated Upto 1 year Deprecated

Frequently Asked Questions

  • How to enable flow stats daily rollups post a 3.3.2 upgrade?

    To enable flow stats daily rollups, set the flowStats.daily.rollup.enabled system property to true.

  • What is the maximum number of flows that are rolled up per edge per day?

    By default, a maximum of one million flows are rolled up per edge per day. This averages out to approximately 3500 flows per 5-minute push. You can modify the number of flows that are rolled up per edge per day, by using the flowStats.daily.rollup.flowLimit system property.

  • Are hub flows rolled up?

    By default, rolling up of hub flows is disabled. You can enable hub flows by using the flowStats.daily.rollup.edgeflowLimit system property, which takes a key-value pair of <edgeId>:<numFlows>. You can view high resolution hub flows upto 15 days only.

  • Is the flow stats retention policy configurable?

    The retention policy for rolled up stats is configurable on the SD-WAN Orchestrator using the retentionWeeks.flowStats.daily system property. The rolled-up flow stats retention can be configured to persist anywhere between 1 and 52 weeks.

  • Will the UI be able to query flowstats for more than 15 days after enabling rollups?

    No. Rolling up flowstats for longer retention is separated from actually being able to query those flowstats. You can set the number of days you want to query the flows by using the session.options.maxFlowstatsRetentionDays system property.

  • Will there be side effects from a data perspective after turning on this feature?

    Although, no side effects are observed on aggregated results, time-series graphs on the SD-WAN Orchestrator UI would have a loss in fidelity due to displaying of rolled up series stats.

  • What are the side effects from a system load perspective?

    Since rolling up daily flowstats aggregates results from the full resolution table and stores it separately, the system load (CPU/load average) is bound to increase due to the additional processing required by MySQL for aggregating the results.

  • What would be the impact to storage requirements for on-premise deployments?

    Since rolling up daily flowstats aggregates results from the high resolution table and stores it separately, VMware anticipates the need for the on-premise customers to plan their storage requirements to accommodate rolled up stats. On an average, rolled up flows will consume 1/8th of the space required for high resolution stats; however, this is strongly dependent on the uniqueness of daily flows sent by the edge. In any case, the storage space consumption growth of rolled up flows will be at a much lower rate than the high resolution statistics. For customers that start off with smaller volume drive, VMware recommends using logical volumes so the storage capacity can be grown as the Edges increase.

Changing the Retention Period

High resolution flow stats retention can be configured to persist anywhere between 1 and 31 days. With the 3.3.0 release, the configuration granularity of the high-resolution flow stats retention has been changed from months to days. Operators can change the retention period by creating a system property. To create a system property to change the retention period, follow the steps below.

  1. From the SD-WAN Orchestrator navigation panel, click System Properties.
  2. In the System Properties screen, click the New System Properties button.
  3. In the New System Property dialog box:
    1. In the Name text field, type retention.flowstats.days.
    2. In the Data Type drop-down menu, choose Number.
    3. In the Value text field, enter the retention period in days.

      new-system-property-flow-stats

  4. Click Save.