VMware Telco Cloud Operations streaming platform refers to the component within the VMware Telco Cloud Operations platform that provides capabilities for data streaming, processing, and computation.
The capabilities of this component include Key Performance Indicator (KPI) calculation, windowed and non-windowed threshold detection, and creation of data "streams" for processing of metric data collected by VMware Telco Cloud Operations collectors. KPI and Threshold streams use the concept of "source" and "sink", to receive data from and to send results to. Sources receive events and supplies events in a common format after transforming them from their original source format. Sinks make the results available on the output using format appropriate for the downstream consumption. The stream is the entity that connects sources of input data to a KPI or Threshold definition and the output of that KPI or threshold crossing evaluation to a destination. Message brokers and databases are examples of sources and sinks. Input is expected to arrive as an unbounded stream of events with values. The KPI or threshold crossing is computed over all the values arriving into the system, using windowed or non-windowed operations. Filtering of the data is also possible to allow for selecting only the events that are required for computation. VMware Telco Cloud Operations Streaming component accepts input data in different formats and can generate the output in different formats as well.
Concepts
VMware Telco Cloud Operations Streaming is used whenever streaming/unbounded data must be processed in real time (or near-real time) for calculating KPIs or recognizing threshold crossings. VMware Telco Cloud Operations streaming does not collect data from available sources or devices, this is left to the data collector mechanism within the platform. VMware Telco Cloud Operations Streaming is NOT used for reporting or persistence of the generated output. Separate downstream systems are responsible for these roles. VMware Telco Cloud Operations Streaming communicates with other VMware Telco Cloud Operations components using Kafka as the underlying data broker.
KPI Value or KPI
Also referred as KPI, is the entity computed by the KPI Engine. It is defined by a KPI definition (described further) that consists of its calculation expression and other relevant properties. A KPI is computed over a stream of input values or metrics, arriving within a time window.
Composite KPI
A composite KPI is a KPI where the formula for calculating the KPI depends on the values of other KPIs. Composite KPIs can refer to other KPIs in the same definition or in other definitions within the system.
Threshold
A threshold is a condition identified for a given metric or KPI by establishing limits or ranges for the value and associating a label to that condition. A threshold "crossing" or "breach" can denote the occurrence of such a condition. A threshold event represents the occurrence of such condition. Thresholds can be associated with a KPI, where the result of KPI calculation is evaluated to determine a threshold condition. Alternatively, thresholds can be associated with a raw metric, in which case there is no windowing component and evaluation of threshold condition occurs in the real time as events are consumed from sources.
Metric
A named measurement, collected from a source. Usually (but not necessarily) a numeric value. Metrics are collected by the different collectors within the VMware Telco Cloud Operations platform.
Metric Catalog
A structured list of metrics associated with some managed entity. The metric catalog is a JSON file that users can add to the system by invoking one of the Streaming REST Service API calls so that these metrics are available in the KPI Designer when creating KPI and Threshold definitions.
Calculation
A computation over a metric or set of metrics, for example, computing a KPI namely their average. A calculation mentions a list of metric names, the language used to define the expression (Spring Expression Language (SpEL)), and the computation expression containing arithmetic operators "and "or "or" functions applied over the metrics.
Window
The time window over which a calculation is performed. Windows can be sliding (that is, overlapping) or tumbling (that is, disjoint). The calculation is performed over all the metric values arriving within the window to generate a KPI Value. The user can select to use the event’s own timestamp generated at its source or the system time when processing the event. This specification also includes the periodicity or frequency with which a KPI Value is to be calculated. The configuration of window attributes is described in greater detail as part of the KPI Definition creation.
Group
Key for grouping input metrics - for example, by the device to which the metric applies. Computed KPI Values are grouped by the same key.
KPI Definition
A uniquely named group of properties that describes one or more KPIs. It includes each KPI’s calculation and settings common to all KPIs namely a window definition, group, and metric filters.
Threshold Definition
A uniquely named group of properties that describes one or more thresholds. This includes each thresholds' range/value configuration and labeling for each of the expected conditions.
Formatter
Converts between the external and internal VMware Telco Cloud Operations Streaming representations of a metric or KPI Value. Input Formatters parse incoming metrics and Output Formatters format the computed KPI Values. Out-of-box formatters are provided for JSON and CSV formats. The Formatter is specified by its name.
Input
Set of attributes of the source of metrics which are ingested into the KPI Engine. Attributes can include URL or IP address and port number of the source service. In the VMware Telco Cloud Operations platform, this is one or more topics on one or more Kafka brokers. Also mention a formatter used to convert from the external representation of metrics to their internal format.
Output
Set of attributes of the destination of computed KPI Values output by the KPI Engine. Destination can be a service such as a message broker that publishes KPI Values or a database that stores them for future use. In the VMware Telco Cloud Operations platform, this is a topic on a Kafka broker. Also mention a formatter used to convert from the internal representation of values to their external format
KPI Stream
A uniquely named association that connects one or more KPI Definitions with one or more inputs and one or more outputs. When the KPI Stream is deployed, the KPI Engine processes the incoming metric streams as defined by the inputs, computes the KPI Values as specified in the KPI Definitions and outputs the KPI Values as specified to the outputs.
Threshold Stream
A uniquely named association that connects one or more Threshold Definitions with one or more inputs and one or more outputs. When the Threshold Stream is deployed, the KPI Engine processes the incoming metric streams as defined by the inputs, determines whether the thresholds' condition has been met (and which one), and outputs the threshold events to the outputs.
Job
A running instance of a KPI or Threshold Stream. Created in the KPI Engine when the KPI Stream is deployed.
Service
A component that provides specific function (stream processing, storage, messaging, configuration, and so on) and enables interoperability and integration options. All VMware Telco Cloud Operations Streaming components are considered to be services with properties that include identifiers, access points, and status. These details are available through its REST API.
