You can configure Comma-Separated Value (CSV) parsers for both FileLog and WinLog collectors.
The available options for the csv parser are fields and delimiter.
Comma-Separated Value Parser Options
Note the following information about the structure of the csv parser.
| Option | Description |
|---|---|
fields |
The The
Field names must be separated by commas, for example
fields = field_name1, field_name2, field_name3, field_name4This definition assumes that the names field_name1, field_name2, field_name3 and field_name4 are assigned sequentially to the extracted fields. If some fields must be omitted by the CSV parser, their names can be omitted from the list. For example, fields = field_name1, , field_name3, field_name4 In this case, the parser extracts only the first, third and fourth fields from the event and subsequently assigns the names field_name1, field_name3 and field_name4 to them. If the fields option does not specify a complete list of the fields in your logs, the parser returns an empty list. For example, if the log file contains You cannot use |
delimiter |
The For example, The csv parser supports any set of characters as delimiters that are enclosed in quotes, for example "||" or "asd". The field values' separators in the logs should exactly match the pattern defined by the delimiter parameter, otherwise the parser will fail. Special characters such as a space or a tab can be defined for as a delimiter for the csv parser as long as the escape character precedes the special character for (\", \s, \t). For example, The |
CSV Log Parser Configuration
To parse logs collected from either winlog or filelog sources, use the following configuration.
[filelog|some_csv_logs]
directory=D:\Logs
include=*.txt;*.txt.*
parser=myparser
[parser|myparser]
base_parser = csv
fields = timestamp,field_name1, field_name2, field_name3
delimiter = ";"
field_decoder={"timestamp": "tsp_parser"}
[parser|tsp_parser]
; timestamp is a built-in parser
base_parser=timestamp
; "format" is an option of timestamp parser
format=%Y-%m-%d %H:%M:%S
With this configuration, logs collected from some_csv_logs source (for example, from the directory=D:\Logs directory) are parsed by myparser. If the collected logs contain three values that are separated by a semicolon, the parsed events sequentially receive the field_name1, field_name2 and field_name3 names.
To parse the following CSV log:
"United States","USA","North America","High income: OECD","Fiscal year end: September 30; reporting period for national accounts data: CY."
[parser|csv_log_parser] base_parser=csv fields=country_name, country_code, region, income_group, special_notesThe CSV parser returns the following fields:
country_name=United States country_code=USA region=North America income_group=High income: OECD special_notes=Fiscal year end: September 30; reporting period for national accounts data: CY.