Input | Kafka Mapper | Output |
---|---|---|
{ "timestamp":"2021-06-08T12:17:27Z", "labels":{ "instance":"100.96.2.131:8080", "amf_nas_non_delivery_indication":" value", "app":"amf", "kubernetes_namespace":"meta323", "job":"kubernetes-namespace-pods", "pod_template_hash":"549dc5ff4d", "kubernetes_pod_name":"fusion-5g-c-ae701-696lz-5g-core-amf-549dc5ff4d-jxcrb", "release":"fusion-5g-c-ae701-696lz", "__name__":"nf_counter_stat" }, "name":"nf_counter_stat", "value":"0" } |
{ "metricType":"5G-Core", "instance":"Metaswitch-5g-core-metrics", "properties.entityName":"$.labels.app", "properties.entityType":"app", "processedTimestamp": "$.timestamp.transform_date(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")", "metrics.$['name']":"$.value", "type":"Kafka-Collector", "properties.dataSource":"$.labels.instance", "properties.deviceName":"$.labels.kubernetes_pod_name", "timestamp":"$.timestamp.transform_date(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")", "properties.deviceType":"$.labels.job", "tags":"$.labels" } |
{ "instance":"Metaswitch-5g-core-metrics", "metricType":"5G-Core", "timestamp":1623179847000, "processedTimestamp":1623179847000, "type":"Kafka-Collector", "metrics":{ "nf_counter_stat":0.0 }, "properties":{ "deviceType":"kubernetes-namespace-pods", "entityName":"amf", "entityType":"app", "dataSource":"100.96.2.131:8080", "deviceName":"fusion-5g-c-ae701-696lz-5g-core-amf-549dc5ff4d-jxcrb" }, "tags":{ "app":"amf", "kubernetes_namespace":"meta323", "instance":"100.96.2.131:8080", "__name__":"nf_counter_stat", "release":"fusion-5g-c-ae701-696lz", "amf_nas_non_delivery_indication":" value", "pod_template_hash":"549dc5ff4d", "job":"kubernetes-namespace-pods", "kubernetes_pod_name":"fusion-5g-c-ae701-696lz-5g-core-amf-549dc5ff4d-jxcrb" } } |
The explanation for all the lines in the Kafka Mapper is covered in Example 1. There is one line which is not covered in example 1 and can be highlighted:
"timestamp":"$.timestamp.transform_date(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")"
The above line demonstrates the use of a custom function for identifying the format of the timestamp in the incoming data. The
$.timestamp
identifies the field in the incoming data that contains the timestamp, and the transform_date is the function call that takes the timestamp format as argument, which is a
string. Java
timestamp library format are accepted.
{ "timestamp":"$.timestamp.transform_date(\"yyyy-MM-dd'T'HH:mm:ss'Z'\")" } would resolve to the output { "timestamp":1623179847000 }