You can access VMware Telco Cloud Automation Pipeline reports using Edge Kafka with the procedure described in this section.

For information about how to view the VMware Telco Cloud Automation reports, see View Pipeline Reports in VMware Telco Cloud Service Assurance User Guide.

Prerequisites

  1. Ensure that you configure VMware Telco Cloud Service Assurance Edge Kafka credentials in the Airflow. To configure Edge Kafka, navigate to Airflow > Variables and configure kafka_brokers variable.

    Note: Airflow used here is a third-party platform.
    Example of kafka_brokers:
    [{
        "bootstrap_servers": ["kafka-edge:32092 "],
        "topic": "tca-stages",
        "security_protocol": "SASL_SSL",
        "additional_properties" : {
            "sasl_mechanism": "SCRAM-SHA-512",
            "sasl_plain_username": "kafka-scram-sha-512-client-credentials",
            "sasl_plain_password" : "RDFYek9vSTJCNGRYCg=="
        },
        "password": "Y2EucGFzc3dvcmQK",
        "ssl_cafile": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZMVENDQXhXZ0F3SUJBZ0lVVlcrT2lEYVlUM3QyeWh6T3lidHJEelFrcXB3d0RRWUpLb1pJaHZjTkFRRU4KQlFBd0xURVRNQkVHQTFVRUNnd0thVzh1YzNSeWFXMTZhVEVXTUJRR0ExVUVBd3dOWTJ4MWMzUmxjaTFqWVNCMgpNREFlRncweU1qQTVNVE14TWpJMU5UVmFGdzB5TXpBNU1UTXhNakkxTlRWYU1DMHhFekFSQmdOVkJBb01DbWx2CkxuTjBjbWx0ZW1reEZqQVVCZ05WQkFNTURXTnNkWE4wWlhJdFkyRWdkakF3Z2dJaU1BMEdDU3FHU0liM0RRRUIKQVFVQUE0SUNEd0F3Z2dJS0FvSUNBUUMyR2l6cTl0aEs2aHhqaS9PeDRlNHprVVdDK3hWQzVNTW5ybGlzSWUrWApyMXFVWllnNTJxYWxJc0NaeGlER1FiM3BMb3R4bTJGNEhOQm10NlJuVWQ3QkJwaHFkcys3MjFSanpRZ1BZbmYyCkVtU0doOHMrai8wWlMzMmQ0ZUtKKzMwMVlSa0ljSkNQa3dTOWN5SkNVMU95OWoyTG9FZ21PNnlSVmplVm9YaFEKYjJ3ZmpqY2d1bTFkYmNYanNSY1R1cHRxclk4MlJVbWVoL0pPbkh1d016TUFFUElBYU9WZEpvVytrRW5oTFF0Zwp3cEUvMHltNncxelhWZm5iSjlXZUNkcnUwaEdJZ2ZkaWZ1MUdJTnM4UDVCcmU2TGZhcjRvZmlYMXJLNENHMHZWCkxKTTJoS29DYm1zRGhpNXc3azgxZ1V4b3hicDBKMTZWQWs5UWtCdnZwVG5zZnBhTDh5ZWMwMTBRS1ZtS0VUQWgKMHFVZFhxRy9XaUE5bVZlaU1lY1owSlp1RTViY2hsZnVrdTFoT0dBRlVvdVVDaDJuV1lYcHg5azdueW01VlN5ZQpEb3puSU8rSkpLMWREVlZwbUVJaUJSbzRrZTBySVhBZ0ZMc3F1U2tLNStlWEZBUkJLVEdISUlEWWYzeFZKdVJ5CkVpWkQ4NGdrK0RpSC9kTThGWkFrTXpjRGdCaUdKTjA0QysralZNYXZqd0pNRzlVV2FhaGgzV3UraHBlY2g3VnMKT0VtaXRHRGxTWUszWHdMNktYSnUrRjMwZkNyYlBGc2xIaTFmTjR5OVRGWWo0UmFOSmREUG0rQmRtMDEvR0taUQpjakhNbWU0VzhTa3J4TlhxLzhwWTJhdVE4cEVYWmlWdlNPRHhDRmMwbEQzVisxUXo0WVVrZXhQanBuNTVUQ1RtCjd3SURBUUFCbzBVd1F6QWRCZ05WSFE0RUZnUVVsR0traFpZTTNUN0FyQmM1SGRLT2w1SWJlSDh3RWdZRFZSMFQKQVFIL0JBZ3dCZ0VCL3dJQkFEQU9CZ05WSFE4QkFmOEVCQU1DQVFZd0RRWUpLb1pJaHZjTkFRRU5CUUFEZ2dJQgpBSG9MM2I5V1JWZUc5dElPQzROcCtZanVWclBiaEtoaTY0UU8vVEw0OVdJVjNRYkxsV0tlRkhacUtQNkJKV29qCi9EYkFtUVNaeWxmbkNzejZubll0RDNJcnFjN2w5WENGNjAwZEZaRCtGNkp1dnpCdlFxR3VJYlAza0czOHlDdU4KQWdyWHBiaXlRQXhRMzhXSnRFSlc4UHFzVzZtdnFSSEY2YUt4VXg4eDF4dWZKSHY4aWcxcG9PT1VqNWZ1Y044QwpQUWY1NGEzWm5YajA2K0QxM1Q2dnh6dzRlMVQyVllYWEtCSEFDL0hIZ095akZpbUhJeDg3eU9Dczduc2Q3c0NwCnU4RE9aT21URWRlUEkxcW95R2w5VmNBdjdVdERjdHI2dmNmdHFHZ2dMNUtEaTVFd09odDdjL1VuNEd0OFQwQTUKNDRWejJvdnZvcDFkU1JuNWplVHFocmxxUTRCVENrb0tSUFIxeEduMmFid1oxOWJFMEVzOXhjMkhWVEk4cDBDYwpJT085SW1mcVptVlBqQjc3cmZSZVpNWmRtTmpqOVVmeHBJMGZvcXZKN21kTTFYNzR1S2tCbTVkbG1sdUFRRE5rCk9BUFdhMnVvVXJEOGhkSUgrT0MrTzFueE02SW9Odm85aU5sdDFUaGl1RC9Mb2EvMUw0YWljQk9pK2dKbjk0UWEKclpQVlo5d2RxNjYvT0VoNlpyZmtadU1iMmVaRUZmZDJzN1dPYW54V0s1YXlNTTZKbmJLUUhjQ25iUWJZUG1lVQpNenF5c090SWxqeWQ1VnVQbVlzLyt1RUd5S1V6U2RIN1NQYTloeXFGWWJKUmJTY09yVFdhNTlTazlCZ1dwREtqClJ0ZkdHdzRieUZsd2FNbW5kYk11L25hU0M2V005eVVsSWJFdmJzWk9hTDUxCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K"
    }]

    Get the passwords and certificate by executing the following commands from the VMware Telco Cloud Service Assurance deployer VM:
    ssl_cafile:
    export CLUSTER_NAME=edge
    EDGENS=kafka-edge
    kubectl get secret -n $EDGENS $CLUSTER_NAME-cluster-ca-cert -o jsonpath='{.data.ca\.crt}'
    
    password:
    kubectl get secret -n $EDGENS $CLUSTER_NAME-cluster-ca-cert -o jsonpath='{.data.ca\.password}' > ca.password
    
    sasl_plain_password:
    export USER_NAME=kafka-scram-sha-512-client-credentials
    export SCRAM_PASSWORD_FILE_PATH=user-scram.password
    kubectl get secret -n $EDGENS $USER_NAME -o jsonpath='{.data.password}' > $SCRAM_PASSWORD_FILE_PATH
    export SCRAM_PASSWORD=`cat $SCRAM_PASSWORD_FILE_PATH`
  2. Run the following statefulset commands:
    kubectl get statefulset -n airflow 
    NAME                 READY   AGE
    airflow-postgresql   1/1     78d
    airflow-redis        1/1     78d
    airflow-worker       3/3     78d
    capv@airflow-cnc-control-plane-vjbx5 [ ~ ]$ 
  3. You can edit and update the following worker statefulset section:
    dnsPolicy: ClusterFirst 
    hostAliases:
    - hostnames:
     - kafka-edge
     ip: 10.198.96.9
    initContainers:
    - args:
     - airflow

Procedure

  1. Configure Kafka Mapper for Pipeline reports. For information about configuring Kafka Mapper, see the Add Kafka Mapping section.
    VMware Telco Cloud Automation Pipeline reports, you can use the following Kafka Mapper file content:
    {
      "instance": "tca-stages",
      "tags.error_msg": "$.error_msg",
      "tags.event": "$.additional_data.[?(@.event)].event",
      "properties.entityType": "$.fsm_state",
      "processedTimestamp": "$.timestamp.transform_date(\"yyyy-MM-dd HH:mm:ss.SSSSSS\")",
      "tags.logs": "$.additional_data.[?(@.logs)].logs",
      "tags.task": "$.additional_data.[?(@.task)].task",
      "type": "KafkaCollector",
      "tags.site_id": "$.site_id",
      "properties.deviceName": "$.fsm_state",
      "metrics.preVal": "1",
      "tags.clustername": "$.additional_data.[?(@.clustername)].clustername",
      "metricType": "tca-stages",
      "properties.entityName": "$.fsm_state",
      "tags.fsm_status": "$.fsm_status",
      "tags.fsm_state": "$.fsm_state",
      "properties.dataSource": "$.site_id",
      "timestamp": "$.timestamp.transform_date(\"yyyy-MM-dd HH:mm:ss.SSSSSS\")",
      "properties.deviceType": "$.fsm_state",
      "tags.dagrunid": "$.additional_data.[?(@.dagrunid)].dagrunid"
    }

  2. Configure Kafka Collector. For more information about configuring Kafka Collector, see Configuring the Kafka Collector topic.