Configure Connections to Pipeline Reports

You can access VMware Telco Cloud Automation Pipeline reports using Edge Kafka with the procedure described in this section.

For information about how to view the VMware Telco Cloud Automation reports, see View Pipeline Reports in VMware Telco Cloud Service Assurance User Guide.

Prerequisites

1. Ensure that you configure VMware Telco Cloud Service Assurance Edge Kafka credentials in the Airflow. To configure Edge Kafka, navigate to Airflow > Variables and configure kafka_brokers variable.

   

   Note: Airflow used here is a third-party platform.
   Example of kafka_brokers:

   [{
       "bootstrap_servers": ["kafka-edge:32092 "],
       "topic": "tca-stages",
       "security_protocol": "SASL_SSL",
       "additional_properties" : {
           "sasl_mechanism": "SCRAM-SHA-512",
           "sasl_plain_username": "kafka-scram-sha-512-client-credentials",
           "sasl_plain_password" : "RDFYek9vSTJCNGRYCg=="
       },
       "password": "Y2EucGFzc3dvcmQK",
       "ssl_cafile": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZMVENDQXhXZ0F3SUJBZ0lVVlcrT2lEYVlUM3QyeWh6T3lidHJEelFrcXB3d0RRWUpLb1pJaHZjTkFRRU4KQlFBd0xURVRNQkVHQTFVRUNnd0thVzh1YzNSeWFXMTZhVEVXTUJRR0ExVUVBd3dOWTJ4MWMzUmxjaTFqWVNCMgpNREFlRncweU1qQTVNVE14TWpJMU5UVmFGdzB5TXpBNU1UTXhNakkxTlRWYU1DMHhFekFSQmdOVkJBb01DbWx2CkxuTjBjbWx0ZW1reEZqQVVCZ05WQkFNTURXTnNkWE4wWlhJdFkyRWdkakF3Z2dJaU1BMEdDU3FHU0liM0RRRUIKQVFVQUE0SUNEd0F3Z2dJS0FvSUNBUUMyR2l6cTl0aEs2aHhqaS9PeDRlNHprVVdDK3hWQzVNTW5ybGlzSWUrWApyMXFVWllnNTJxYWxJc0NaeGlER1FiM3BMb3R4bTJGNEhOQm10NlJuVWQ3QkJwaHFkcys3MjFSanpRZ1BZbmYyCkVtU0doOHMrai8wWlMzMmQ0ZUtKKzMwMVlSa0ljSkNQa3dTOWN5SkNVMU95OWoyTG9FZ21PNnlSVmplVm9YaFEKYjJ3ZmpqY2d1bTFkYmNYanNSY1R1cHRxclk4MlJVbWVoL0pPbkh1d016TUFFUElBYU9WZEpvVytrRW5oTFF0Zwp3cEUvMHltNncxelhWZm5iSjlXZUNkcnUwaEdJZ2ZkaWZ1MUdJTnM4UDVCcmU2TGZhcjRvZmlYMXJLNENHMHZWCkxKTTJoS29DYm1zRGhpNXc3azgxZ1V4b3hicDBKMTZWQWs5UWtCdnZwVG5zZnBhTDh5ZWMwMTBRS1ZtS0VUQWgKMHFVZFhxRy9XaUE5bVZlaU1lY1owSlp1RTViY2hsZnVrdTFoT0dBRlVvdVVDaDJuV1lYcHg5azdueW01VlN5ZQpEb3puSU8rSkpLMWREVlZwbUVJaUJSbzRrZTBySVhBZ0ZMc3F1U2tLNStlWEZBUkJLVEdISUlEWWYzeFZKdVJ5CkVpWkQ4NGdrK0RpSC9kTThGWkFrTXpjRGdCaUdKTjA0QysralZNYXZqd0pNRzlVV2FhaGgzV3UraHBlY2g3VnMKT0VtaXRHRGxTWUszWHdMNktYSnUrRjMwZkNyYlBGc2xIaTFmTjR5OVRGWWo0UmFOSmREUG0rQmRtMDEvR0taUQpjakhNbWU0VzhTa3J4TlhxLzhwWTJhdVE4cEVYWmlWdlNPRHhDRmMwbEQzVisxUXo0WVVrZXhQanBuNTVUQ1RtCjd3SURBUUFCbzBVd1F6QWRCZ05WSFE0RUZnUVVsR0traFpZTTNUN0FyQmM1SGRLT2w1SWJlSDh3RWdZRFZSMFQKQVFIL0JBZ3dCZ0VCL3dJQkFEQU9CZ05WSFE4QkFmOEVCQU1DQVFZd0RRWUpLb1pJaHZjTkFRRU5CUUFEZ2dJQgpBSG9MM2I5V1JWZUc5dElPQzROcCtZanVWclBiaEtoaTY0UU8vVEw0OVdJVjNRYkxsV0tlRkhacUtQNkJKV29qCi9EYkFtUVNaeWxmbkNzejZubll0RDNJcnFjN2w5WENGNjAwZEZaRCtGNkp1dnpCdlFxR3VJYlAza0czOHlDdU4KQWdyWHBiaXlRQXhRMzhXSnRFSlc4UHFzVzZtdnFSSEY2YUt4VXg4eDF4dWZKSHY4aWcxcG9PT1VqNWZ1Y044QwpQUWY1NGEzWm5YajA2K0QxM1Q2dnh6dzRlMVQyVllYWEtCSEFDL0hIZ095akZpbUhJeDg3eU9Dczduc2Q3c0NwCnU4RE9aT21URWRlUEkxcW95R2w5VmNBdjdVdERjdHI2dmNmdHFHZ2dMNUtEaTVFd09odDdjL1VuNEd0OFQwQTUKNDRWejJvdnZvcDFkU1JuNWplVHFocmxxUTRCVENrb0tSUFIxeEduMmFid1oxOWJFMEVzOXhjMkhWVEk4cDBDYwpJT085SW1mcVptVlBqQjc3cmZSZVpNWmRtTmpqOVVmeHBJMGZvcXZKN21kTTFYNzR1S2tCbTVkbG1sdUFRRE5rCk9BUFdhMnVvVXJEOGhkSUgrT0MrTzFueE02SW9Odm85aU5sdDFUaGl1RC9Mb2EvMUw0YWljQk9pK2dKbjk0UWEKclpQVlo5d2RxNjYvT0VoNlpyZmtadU1iMmVaRUZmZDJzN1dPYW54V0s1YXlNTTZKbmJLUUhjQ25iUWJZUG1lVQpNenF5c090SWxqeWQ1VnVQbVlzLyt1RUd5S1V6U2RIN1NQYTloeXFGWWJKUmJTY09yVFdhNTlTazlCZ1dwREtqClJ0ZkdHdzRieUZsd2FNbW5kYk11L25hU0M2V005eVVsSWJFdmJzWk9hTDUxCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K"
   }]

   

   Get the passwords and certificate by executing the following commands from the VMware Telco Cloud Service Assurance deployer VM:

   ssl_cafile:
   export CLUSTER_NAME=edge
   EDGENS=kafka-edge
   kubectl get secret -n $EDGENS $CLUSTER_NAME-cluster-ca-cert -o jsonpath='{.data.ca\.crt}'
   
   password:
   kubectl get secret -n $EDGENS $CLUSTER_NAME-cluster-ca-cert -o jsonpath='{.data.ca\.password}' > ca.password
   
   sasl_plain_password:
   export USER_NAME=kafka-scram-sha-512-client-credentials
   export SCRAM_PASSWORD_FILE_PATH=user-scram.password
   kubectl get secret -n $EDGENS $USER_NAME -o jsonpath='{.data.password}' > $SCRAM_PASSWORD_FILE_PATH
   export SCRAM_PASSWORD=`cat $SCRAM_PASSWORD_FILE_PATH`

2. Run the following statefulset commands:

   kubectl get statefulset -n airflow 
   NAME                 READY   AGE
   airflow-postgresql   1/1     78d
   airflow-redis        1/1     78d
   airflow-worker       3/3     78d
   capv@airflow-cnc-control-plane-vjbx5 [ ~ ]$ 

3. You can edit and update the following worker statefulset section:

   dnsPolicy: ClusterFirst 
   hostAliases:
   - hostnames:
    - kafka-edge
    ip: 10.198.96.9
   initContainers:
   - args:
    - airflow

Procedure

1. Configure Kafka Mapper for Pipeline reports. For information about configuring Kafka Mapper, see the Add Kafka Mapping section.
   VMware Telco Cloud Automation Pipeline reports, you can use the following Kafka Mapper file content:

   {
     "instance": "tca-stages",
     "tags.error_msg": "$.error_msg",
     "tags.event": "$.additional_data.[?(@.event)].event",
     "properties.entityType": "$.fsm_state",
     "processedTimestamp": "$.timestamp.transform_date(\"yyyy-MM-dd HH:mm:ss.SSSSSS\")",
     "tags.logs": "$.additional_data.[?(@.logs)].logs",
     "tags.task": "$.additional_data.[?(@.task)].task",
     "type": "KafkaCollector",
     "tags.site_id": "$.site_id",
     "properties.deviceName": "$.fsm_state",
     "metrics.preVal": "1",
     "tags.clustername": "$.additional_data.[?(@.clustername)].clustername",
     "metricType": "tca-stages",
     "properties.entityName": "$.fsm_state",
     "tags.fsm_status": "$.fsm_status",
     "tags.fsm_state": "$.fsm_state",
     "properties.dataSource": "$.site_id",
     "timestamp": "$.timestamp.transform_date(\"yyyy-MM-dd HH:mm:ss.SSSSSS\")",
     "properties.deviceType": "$.fsm_state",
     "tags.dagrunid": "$.additional_data.[?(@.dagrunid)].dagrunid"
   }

   

2. Configure Kafka Collector. For more information about configuring Kafka Collector, see Configuring the Kafka Collector topic.