Enrichment stream GUI visualizes enrichment stream management functions, helps the user to create, read, update, delete, deploy, and undeploy enrichment streams. The following sections introduce the functionalities of the enrichment stream GUI.

Enrichment Stream List Page

When users click the Enrichment Stream tab under the KPI Designer, the browser navigates to the enrichment stream list page, click Enrichment Stream. Where the previous created enrichment stream displays:

There are four pre-deployed enrichment streams:

  1. event-stream: an enrichment stream for events.
  2. mnr-gateway-stream: an enrichment stream for mnr gateway metrics.
  3. metric-stream: an enrichment stream for metrics.
  4. topology stream: an enrichment stream for topology.
Each row in the list shows a stream name and its description. At the beginning of each row, there is an action menu (three vertical dots), which has the following menu items:
  • Edit: Allows the page to navigate to the enrichment stream creating/editing page to edit the selected stream.
  • Delete: Deletes the selected enrichment stream definition.
  • Deploy: Allows the users to deploy the selected enrichment stream. The menu item shows when the stream is not deployed.
  • Undeploy: Allows the users to undeploy the selected enrichment stream. The menu item shows when the stream is deployed.
  • Export: Exports the stream configuration DSL (Domain-Specific Language) in the JSON format.

Create an Enrichment Stream

Click the Create button and the browser navigates to the stream create and edit page.

Following are the two fields in the form on the top of the page::
  • Stream Name: The name of the enrichment stream.
  • Description: The description about the enrichment stream
On the left navigation menu following four categories of stream components:
  • Input: Allows you to select the input components that receive events, such as Kafka Input.
  • Transformer: Allows you to select the transformation components that convert event content, such as Formatter.
  • Enricher: Allows you to select the enrichment components that enrich event content, such as Data Enrichment.
  • Output: Allows you to select the output components that publish events, such as Kafka Output.

The following section introduces the components in more detail.

Input Components

TCOps Kafka and External Kafka are two input components supported in this release.

TCOps Kafka: Omega stream internal Kafka broker. Used for the metric event input.

Drag and drop the Default Kafka to the workspace. The component is added to the enrichment stream.

Click the edit icon on the component. The edit panel allows you to configure:
  • Description: Description of the component, which makes the stream more readable.
  • Topic: Kafka topic for the metric event stream input.

After the fields are configured, click APPLY to save the configuration to memory. A validation error in the edit panel fields can disable the APPLY button.

External Kafka input, which uses to enrich the event stream.

Dragging and dropping the External Kafka input to the workspace, the component is added to the enrichment stream.

Click the editing icon on the component, the edit panel allows you to configure the external data input:
  • Description: The description of the component.
  • Bootstrap servers: Configures Kafka broker address. Hostname:port or IP-address:port in a comma-separated format, such as, kafka-edge:9093,kafka-edge1:9093.
  • Topic: Kafka topic for the external data stream.

After the fields are configured, click APPLY to save the configuration to the memory. Note: validation error in the edit panel fields will disable the APPLY button.

Transformation Components

One transformation component is supported in this release. More transformation components are added where use cases apply.
  • External Data Formatter: formats the external data to the internal data model for the stream enrichment.
  • Event Formatter: formats VMware Telco Cloud Operations event data.
  • Topology Formatter: formats VMware Telco Cloud Operations topology data.
  • Metric Formatter: formats VMware Telco Cloud Operations metric data.

Drag and drop the component to the workspace, and click the editing icon on the External Data Formatter component, the edit panel appears. You can configure the description for this component:. After the fields are configured, click APPLY to save the configurations to memory. A validation error in the edit panel fields can disable the APPLY button.

Enrichment Components

One enrichment component is supported in this release. More enrichment components are added where use cases apply.
  • Data Enricher: Enriches the event stream with external data. The data enricher enriches the internal event tags section by using the tags section in the external reference data.

By dragging and dropping the component to the workspace, the component is added to the enrichment stream.

By clicking the editing icon on the Data Enricher, the edit panel shows to configure the data enricher. The following fields are configured through the edit panel for the Data Enricher.

  • Name: The name of this data enricher.
  • Description: The description of the component.
  • Event Key: The key using the properties in the internal data model, which is used to match with the corresponding external data key. Note: double quote is not allowed in the Event Key field. Type '\' for content assist. The properties in the drop-down list support the following data models:

    1. Metric properties: for VMware Telco Cloud Operations & MnR gateway metric data model.

    2. Event properties: for event datam model.

    3. Topology properties: for topology data model.

  • Partition Type: Configures how the stream events and external data events are partitioned. The two selections are:
    • Replicated: The external data is replicated across the cluster so that a copy is present on each node.
    • Partitioned: The external data is spread out across the cluster and must be partitioned in the same way as the event. In this mode, users can configure:
      • Event Partition Key: Configures how the metric events are partitioned.
      • Data Partition Key: Configures how the external data events are partitioned.

After the fields are configured, click APPLY to save the configuration to the memory. A validation error in the edit panel fields will disable the APPLY button.

Output Components

One output component is supported in this release.
  • TCOps Kafka: Omega stream internal Kafka broker. Used to publish the metric events.

By dragging and dropping the component to the workspace, the component is added to the enrichment stream.

By clicking the editing icon on TCOps Kafka, the edit panel shows to configure TCOps Kafka The following fields are configured through the edit panel:
  • Description: The description of the component.
  • Topic: Kafka output topic for the metric event stream.

After the fields are configured, click APPLY to save the configuration to the memory. A validation error in the edit panel fields will disable the APPLY button.

Connect the Components

After the components are added and configured, the users must connect the components to indicate the data flow directions.
  • To connect two components, press and hold 'Alt'/'Option' key, and left-click on the first components, then left-click on the second component. An arrowed line is added from the first component to the second component.
  • To remove a connection between the two components, repeat the operation as connecting the two components.

After the stream is configured, click SAVE to save the configuration, so that the stream configuration can be submitted and saved to the backend server.

After the stream is saved, the users can deploy the stream to internal streaming engine by clicking the DEPLOY button.

Edit an Enrichment Stream

After the stream is saved. The saved stream is shown in the enrichment stream list page, for example: the Location enrichment is newly added enrichment stream in the following diagram:

To navigate to the stream editing page, Click the Edit in the action menu, where you can edit the existing stream configuration. For example, the following diagram shows that a new enricher and corresponding external data input are added to a stream:

For example, the following diagram shows a new enricher, and corresponding external data inputs are added to the stream.