JSON Parser

You can customize the JSON parser configuration to selectively parse the JSON log.

You can configure comma-separate value (CSV) parsers for both FileLog and WinLog collectors. Only valid JSON logs are parsed with the Log Insight agent JSON parser. Invalid JSON log parsers return empty results.

The default JSON parser configuration extracts all fields from the JSON log by the Log Insight agent. When the JSON log represents itself as a complex JSON object, which can contain JSON objects as well, the parser uses an underscore (_) characcter to concatenate names of nested and higher tiered JSON objects. This produces an informative field name for the corresponding elements. If the JSON log also contains an array, the member element names contain the array name followed by the element's index in the array.

The JSON parser also provides a specific option, known as fields.

JSON Parser 'fields' Option

You can use the fields option to specify which fields are parsed in the configuration. The purpose of this option is to enable selective parsing of the JSON log.

Note: For selective parsing, you must specify the path to the desired JSON element. JSON objects from different tiers must be separated with a dot (.) character.

The following list provides example configurations that enable you to selective parse the JSON log as desired.

To parse more than one element from the JSON log, the desired elements must be listed as parameters for the fields option and separated by commas. See example below:

{"operation" : {"timestamp" :
        "2018-11-22T15:28:58.094000", "thread_id" : "0x05673", "initiator" : "connector",
        "log_severity" : "info", "log_message" : "Requested connection to the server."},
        "operation_result" : "success"}

To parse only the most inner JSON objects, such as timestamp, log_severity, and log_message see the example below. This example configuration produces the following field results: operation_timestamp ="2018-11-22T15:28:58.094000" and operation_log_severity ="info"
```
[parser|json_parser]
base_parser=json
fields=operation.timestamp,operation.log_severity, operation.log_message
```

To parse the entire JSON object, include the path to the object followed by an asterisk (*) character.

{"product_name" : "LI Agent",
        "operation" : {"timestamp" : "2018-11-22T15:28:58.094000", "thread_id" :
        "0x05673", "initiator" : "connector", "log_severity" : "info", "log_message" :
        "Requested connection to the server."}, "operation_result" :
      "success"}

To parse only the operation object, use the following configuration:
```
[parser|json_parser]
base_parser=json
fields=operation.*
```

If the JSON log contains an array and you want to parse only specific elements of the array, use the array's element index in the configuration, as seen in this example configuration:

{           
      "Records": [{                                 
             "object":{                                               
                "key": "/events/mykey",                                               
                "size": 502,                                               
                "eTag": "091820398091823",                                               
                "sequencer": "1123123"                                   
            }                       
       },                       
      {                                  
            "object":{                                               
                  "key": "/events/user_key",                                               
                  "size": 128,                                               
                  "eTag": "09182039000001",                                               
                  "sequencer": "1123231"                                   
            }                        
        },                        
        {                                   
           "object":{                                               
                 "key": "/events/admin_key",                                               
                 "size": 1024,                                               
                 "eTag": "09182039547241",                                               
                 "sequencer": "1123213"                                   
            }                      
        }           
     ] 
}

To only parse the key and size elements from the same log, use the following configuration to produce the following fields:
records0_object_key="/events/mykey"
records0_object_size=502
records2_object_key="/events/admin_key"
records2_object_size=1024
```
[parser|json_parser]
base_parser=json
fields = Records0.object.key Records0.object.size, Records2.object.key, Records2.object.size
```
To parse the key field for all array elements, use the following configuration:
```
[parser|json_parser]
base_parser=json
fields=Records.#.object.key
```
To parse all fields, use the fields option with an asterisk (*) character. This configuraiton is equivalent to the default JSON parser configuration.
```
[parser|json_parser]
base_parser=json
fields=*
```