Enrichment stream GUI visualizes enrichment stream management functions, helps the user to create/read/update/delete/deploy/undeploy enrichment streams.

Enrichment Stream List Page

To navigates to the enrichment stream list page, click Enrichment Stream. Where the previous created enrichment stream displays:

Each row in the list shows a stream name and its description. At the beginning of each row, there is an action menu (three vertical dots), which has the following menu items:
  • Edit: Allows the page to navigate to the enrichment stream creating/editing page to edit the selected stream.
  • Delete: Deletes the selected enrichment stream definition.
  • Deploy: Allows the users to deploy the selected enrichment stream. The menu item shows when the stream is not deployed.
  • Undeploy: Allows the users to undeploy the selected enrichment stream. The menu item shows when the stream is deployed.
  • Export: Exports the stream configuration DSL (Domain-Specific Language) in the JSON format.

Create an Enrichment Stream

To navigate to the stream creating or editing page, click CREATE.

Following are the two text boxes in the Create an Enrichment Stream:
  • Stream Name: The name of the enrichment stream.
  • Description: The description about the enrichment stream
On the left, following four categories of stream components are present:
  • Input: Allows you to select the input components that receive events, such as Kafka Input.
  • Transformer: Allows you to select the transformation components that convert event content, such as Formatter.
  • Enricher: Allows you to select the enrichment components that enrich event content, such as Data Enrichment.
  • Output: Allows you to select the output components that publish events, such as Kafka Output.

The following section introduces the components in more detail.

Input Components

There are two input components supported in this release, Default Kafka and Kafka External Data Input.

Default Kafka: Omega stream internal Kafka broker. Used for the metric event input.

Drag the Default Kafka to the workspace, the component is added to the enrichment stream.

Click the edit icon on the component, the edit panel allows you to configure:
  • Description: Description of the component, which makes the stream more readable.
  • Topic: Kafka topic for the metric event stream input.

To save the configuration to the memory, click SAVE.

Kafka External Data Input: external data Kafka input, which uses to enrich the event stream.

Drag the Kafka External Data Input to the workspace, the component is added to the enrichment stream.

Click the editing icon on the component, the edit panel allows you to configure the external data input:
  • Description: The description of the component.
  • Group ID: Kafka client group ID.
  • Bootstrap servers: Configures Kafka broker address. Hostname:port or IP-address:port in a comma-separated format, such as, localhost: 9092
  • Topic: Kafka topic for the external data stream.

Transformation Components

One transformation component is supported in this release. More transformation components are added where use cases apply.
  • External Data Formatter: formats the external data to the internal data model for the stream enrichment.

Drag the component to the workspace, and click the editing icon on the External Data Formatter component, the edit panel appears.

You can configure the description for this component:

Enrichment Components

One enrichment component is supported in this release. More enrichment components are added where use cases apply.
  • Data Enricher: Enriches the event stream with external data.

To add the component in the enrichment stream, drag the component to the workspace.

To configure the data enricher, click the editing icon on the Data Enricher. The following text boxes are configured through the edit panel:
  • Name: The name of this data enricher.
  • Description: The description of the component.
  • Event Key: The key in the metric event, used to match the event with its corresponding external data event.
  • Event Map: The map in the metric event where the enrichment data is added.
  • Data Key: The key in the external data event, used to match the stream event.
  • Data Map: The map in the external data where the enrichment data is retrieved.
  • Partition Type: Configures how the stream events and external data events are partitioned. The two selections are:
    • Replicated: The external data is replicated across the cluster so that a copy is present on each node.
    • Partitioned: The external data is spread out across the cluster and must be partitioned in the same way as the event. In this mode, users can configure:
      • Event Partition Key: Configures how the metric events are partitioned.
      • Data Partition Key: Configures how the external data events are partitioned.

Output Components

One output component is supported in this release.
  • Default Kafka: Omega stream internal Kafka broker. Used to publish the metric events.

To add the component in the enrichment stream, drag the component to the workspace. The component is added to the enrichment stream.

To configure the Default Kafka, click the editing icon on the Default Kafka, the edit panel appears. The following text boxes are configured through the edit panel:
  • Description: The description of the component.
  • Topic: Kafka output topic for the metric event stream.

Connect the Components

After the components are added and configured, the users must connect the components to indicate the data flow directions.
  • To connect two components, press and hold 'Alt'/'Option' key, and left-click on the first components, then left-click on the second component. An arrowed line is added from the first component to the second component.
  • To remove a connection between the two components, repeat the operation as connecting the two components.

After the stream is configured, click SAVE to save the configuration, so that the stream configuration can be submitted and saved to the backend server.

After the stream is saved, the users can deploy the stream to internal streaming engine by clicking the DEPLOY button.

Edit an Enrichment Stream

After the stream is saved. The saved stream is shown in the enrichment stream list page, for example: the Location enrichment is newly added enrichment stream in the following diagram:

To navigate to the stream editing page, Click the Edit in the action menu, where you can edit the existing stream configuration. For example, the following diagram shows that a new enricher and corresponding external data input are added to a stream: