Configuration

SP related configuration files can be located in following directories:

<DCF-Install>/Collecting/Collector-Manager/<SP-Instance>(clearwater)/conf/

<DCF-Install>/ Collecting /SNMP -Collector/<SP-Instance>/conf/

<DCF-Install>/ Collecting /Kafka-Connector/<SP-Instance>/conf/

1. Collector Manager Configuration

   A Collecting Manager Configuration file can be found in below location:

   File: <DCF-Install>/ Collecting/Collector-Manager /<INSTANCE>(clearwater)/conf/collecting.xml

   Sample contents :

   <?xml version="1.0" encoding="UTF-8"?>
   <config xmlns="http://www.watch4net.com/APG/Collecting" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.watch4net.com/APG/Collecting collecting.xsd ">
       <connectors>
           <connector enabled="false" name="File" type="File-Connector"   config="conf/file-connector.xml" />
           <connector enabled="true" name="Kafka" type="Kafka-Connector"   config="Kafka-Connector/clearwater/conf/kafka-connector.xml" />
       </connectors>
       <filters>
           <filter enabled="true" name="Param-CLEANUP" next="Kafka" config="Variable-Handling-Filter/clearwater/conf/vhf-param-cleanup.xml"/>
       </filters>
           <collectors>
           <collector enabled="true" name="CLEARWATER-SNMP" next="Param-CLEANUP" config="SNMP-Collector/clearwater/conf/snmpcollector.xml" />
           </collectors>
     </config>
   

   This is basic configuration of the SP which deals with how the processing should start.

2. SNMP Metrics Collecting Configuration Files

   To configure the SNMP collector, the following configuration files are required:

   1. Snmpcollector.xml: This file specifies the file path for other configuration files.

      <?xml version="1.0" encoding="UTF-8"?>
      <config>
          <slave-snmp-poller-file >conf/slave-snmp-poller.xml</slave-snmp-poller-file>
          <snmp-masks-file >conf/snmp-masks.xml</snmp-masks-file>
          <snmp-polling-distribution-file >conf/snmp-polling-distribution.xml</snmp-polling-distribution-file>
          <translations-file>conf/translations.xml</translations-file>
          <mib-files-directory >mibs</mib-files-directory>
      </config>
      

   2. slave-snmp-poller.xml: This file associates SNMP Agents to SnmpMasks in PollingGroups.

      <?xml version="1.0" encoding="UTF-8"?><slave-snmp-poller name="ClearwaterCollector" pollerPrefix="SNMP-CLEARWATER" groupName="group" oidsThreads="100" tablesThreads="100" alwaysPushProperties="true" handleCounterWrapAround="true" usePollerPrefixInVariable="false" oidAsPropertyInRawValues="false" configVersion="6.6" xmlns:xi="http://www.w3.org/2001/XInclude">
        <local-communication-settings hostIpAddress="localhost" communicationPort="52006" communicationInterfaceIPAddress="0.0.0.0" maxConcurrentSessions="98">
          <out-interface-ip-addresses>
            <out-interface-ip-address>0.0.0.0</out-interface-ip-address>
          </out-interface-ip-addresses>
          <accept-commands-from-list/>
        </local-communication-settings>
        <xi:include  href="polling-groups.xml"/>
      </slave-snmp-poller>
      

      polling-groups.xml:

      <?xml version="1.0" encoding="UTF-8"?><polling-groups>
        <polling-group name="PG_CLEARWATER-BONO" pollingPeriod="300" tableDiscoveryTime="00:42:00" groupName="group" uniformTimestamps="false" startAtDefinedPeriod="false" enabled="true" startTimeOffset="0">
          <polling-masks>
            <polling-mask-name>CLEARWATER-BONO</polling-mask-name>
          </polling-masks>
          <polled-snmp-agents>
            <snmp-agents-explicit-group-name>PG_CLEARWATER-BONO</snmp-agents-explicit-group-name>
          </polled-snmp-agents>
        </polling-group>
        <polling-group name="PG_CLEARWATER-SPROUT" pollingPeriod="300" tableDiscoveryTime="00:42:00" groupName="group" uniformTimestamps="false" startAtDefinedPeriod="false" enabled="true" startTimeOffset="0">
          <polling-masks>
            <polling-mask-name>CLEARWATER-SPROUT</polling-mask-name>
          </polling-masks>
          <polled-snmp-agents>
            <snmp-agents-explicit-group-name>PG_CLEARWATER-SPROUT</snmp-agents-explicit-group-name>
          </polled-snmp-agents>
        </polling-group>
      </polling-groups>
      

      Polling Group attributes are:

      1. name: The polling group’s name.
      2. pollingPeriod: The polling group’s polling period, specified in seconds. This value indicate the time interval between each polling.
      3. groupName: This attribute overrides the retention group name specified in the slave-snmp-poller element for this polling group and all of the ones defined under it.
      4. polling-masks: This element set contains polling-mask-name elements. At least one pollingmask- name element must be provided per polling-group.
      5. polled-snmp-agents: This element set contains at least one snmp-agent-ip-address, snmpagents- ip-group-name or snmp-agents-explicit-group-name element. These elements actually indicate which SNMP agent to poll in this polling group.

   3. Snmp-masks.xml: SNMP collecting process defined in the snmp-masks.xml file. Clearwater uses SNMP-table polling method.

      Sample snmp-masks file:

      <snmp-mask name="CLEARWATER-SPROUT" doNotEdit="true" version="1.0.0.0">
          <snmp-properties>
            <snmp-property id="vendor" name="vendor" value="Clearwater"/>
            <snmp-property id="type" name="type" value="Sprout"/>
            <snmp-property id="datagrp" name="datagrp" value="PM"/>
          </snmp-properties>
          <snmp-tables>
            <snmp-table id="ICSCFSessionEstablishmentNetworkTable" name="ICSCFSessionEstablishmentNetworkTable" oid=".1.2.826.0.1.1578918.9.3.37" tableIndex=".1.1" tableColNameOid=".1.1" indexEntryMask="oid._._.1" useSnmpV2cMultipleRowPolling="true" cacheAllSnmpReferenceTables="false" sortOnOid="false" matchNFirstOidSuffixOnly="0">
              <columns-and-filters>
                <snmp-column name="ICSCFSessionEstablishmentNetworkAttempts" appendOid=".1.2" unit="integer" refreshOnPropertyChange="false"/>
                <snmp-column name="ICSCFSessionEstablishmentNetworkSuccesses" appendOid=".1.3" unit="integer" refreshOnPropertyChange="false"/>
                <snmp-column name="ICSCFSessionEstablishmentNetworkFailures" appendOid=".1.4" unit="integer" refreshOnPropertyChange="false"/>
                <snmp-column name="ICSCFSessionEstablishmentNetworkSuccessPercent" appendOid=".1.5" unit="integer" refreshOnPropertyChange="false"/>
              </columns-and-filters>
            </snmp-table>
           </snmp-tables>
          <mib-files>
            <mib-file-name>INET-ADDRESS-MIB.txt</mib-file-name>
            <mib-file-name>PROJECT-CLEARWATER-MIB</mib-file-name>
          </mib-files>
      

      Snmp-masks attributes are:

      1. id: The table’s id, which must be unique among all snmp-oid and snmp-table elements.
      2. name: The table’s name.
      3. oid: The table’s base oid. i.e.: ifTable is .1.3.6.1.2.1.2.2.
      4. tableIndex: The table’s index columns. The form MUST BE AS FOLLOWS: “.1.1 .1.2” This indicates column 1 and 2 of the table.
      5. tableColNameOid: Specifies which column values must be used, in the table, to set the part property for all polled columns. i.e.: tableColNameOid=“.1.1 .1.2” to use column 1 and 2 values.
      6. indexEntryMask: This describes the table’s rows oid structure and is used to generate index pseudo-columns when the table does not have explicit index columns.
      7. partType: Specifies the raw value’s property parttype. This is user defined.
      8. name: the oid name. It may or may not be the same as the oid name specified in a MIB. This is the name property in the resulting rawValue.
      9. appendOid: This specifies the table’s column to poll. It’s format is ALWAYS “.1.X”, where X is the column number.
      10. MIB Files: Mib files are used to get textual values from integer enumerated values. The textual value will be set as the property value if polled as a property.

          <mib-files>
                <mib-file-name>INET-ADDRESS-MIB.txt</mib-file-name>
                <mib-file-name>PROJECT-CLEARWATER-MIB</mib-file-name>
              </mib-files>
          

   4. agents-groups.xml

      <?xml version="1.0" encoding="UTF-8"?><agents-groups>
        <snmp-agent ipAddress="localhost" transportProtocol="udp" snmpPort="161" timeout="30000" retries="2" snmpVersion="v2c" snmpCommunityOrUserV3Name="{5563E4FF395EB0585419E36F21AA940ECB70A8EEBEE91682A2C57A9CA4C4E0F31F51CD48E0EBC0B099F148A88002B4AF}" maxBulkSize="50" devtype="vIMS" doNotUseGetBulkRequests="false" onFailTryWith="v1"/>
        <snmp-agents-explicit-group name="PG_CLEARWATER-BONO">
          <agent-ip-address>localhost:161</agent-ip-address>
        </snmp-agents-explicit-group>
        <snmp-agents-explicit-group name="PG_CLEARWATER-SPROUT">
          <agent-ip-address>localhost:161</agent-ip-address>
        </snmp-agents-explicit-group>
      </agents-groups>
      

      Agents-groups attributes are:

      1. ipAddress: This is the ipAddress of the SNMP agent (Clearwater).
      2. transportProtocol: udp is by default.
      3. snmpPort: This is the SNMP port to use when polling the SNMP agent. Port 161 is set by default.
      4. snmpVersion: the SNMP version to use when polling the agent. Defaults to v2c for clearwater.
      5. snmpCommunityOrUserV3Name: SNMP v2c community string.
      6. snmp-agents-explicit-group: Specifies SNMP agent group name.
      7. agent-ip-address: SNMP agent IP address and its port.

3. Kafka Connector Configuration: Kafka server configuration used to publish SNMP performance metrics from clearwater to Kafka is below:

   File : <DCF-Install>/Collecting/Kafka-Connector/<INSTANCE>(clearwater)/conf/kafka-connector.xml)

   <?xml version="1.0" encoding="UTF-8"?>
   <kafka-connector-config xmlns="http://www.watch4net.com/KafkaConnector" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.watch4net.com/KafkaConnector ../kafka-connector.xsd ">
           <kafka-producer-settings>
               <!-- Servers information -->
                   <server host="localhost" port="9092"/>
               <!-- Producer (and topic) information -->
               <producer topic-name="clearwater-snmp"
                   acks="1"
                   retries="0"
                   linger-ms="0ms"
                   buffer-memory="33554432"
                   compression-type="none"
                   batch-size="16384"
                   max-block-ms="5s"
                   max-in-flight-requests-per-connection="5"
                   max-request-size="1048576"/>
   
               <!-- All connection parameters will be attributes except "server" -->
               <connection request-timeout-ms="5s"
                   connections-max-idle-ms="9m"
                   retry-backoff-ms="100ms"
                   reconnect-backoff-ms="50ms"/>
   
               <!-- Additional properties -->
               <additional-parameters key="metadata.max.age.ms">1000</additional-parameters>
               <additional-parameters key="receive.buffer.bytes">32768</additional-parameters>
               <additional-parameters key="send.buffer.bytes">131072</additional-parameters>
   
                   <additional-parameters key="sasl.mechanism">PLAIN</additional-parameters>
           <additional-parameters key="security.protocol">SASL_PLAINTEXT</additional-parameters>
           <additional-parameters key="sasl.jaas.config">org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="{BCB52C90E273915E2727C2D77EDFFED922A6608165C43784D1FB5B24FBB18A2434C8260D30DF673B67D17BA5870177A2}";</additional-parameters>
   
           </kafka-producer-settings>
   
           <connector-component-behavior outputJson="true" flush-every-n-rawvalues="5000"/>
   
   </kafka-connector-config>
   
   

   Kafka Connector parameters

   Parameters	Description
   server	This tag must occur at least once. host: The address of one of the kafka bootstrap servers port: The port that the bootstrap server is listening on.
   producer	This tag is optional, but may be used for templating kafka producers. Please refer to the schema for more information.
   connection	This tag is optional, but may be used for templating kafka connections. Please refer to the schema for more information.
   additional-parameters	This tag is optional, but may be used for configuring kafka options outside of the purview of the previous tags. These options include, but are not limited to SSL connection parameters.

   Configuring Kafka Connector:

   Parameters	Description
   kafka-producer-settings	This tag may be used if we want to customize how the writer writes to kafka. Please refer to the schema for more information.
   connector-component-behavior	This tag may be used if we want to control how often we flush to kafka.

   Note: Few raw metrics from kafka are directly read by vROps management pack, and few other raw metrics are read by K4M and are processed back to Kafka, which in-turn read by vROps. Refer vRealize Operations Management Pack for Smart Assurance Adapter Guide and VMware Smart Assurance K4M KPI Designer User Guide for more information.