The Labeled Tab-separated Values (LTSV) format is a variant of Tab-separated Values (TSV).
Each record in a LTSV file is represented as a single line. Each field is separated by <TAB>
and has a label and a value. The label and the value are separated by :
. With the LTSV format, you can parse each line by splitting the line with <TAB>
(the same as the TSV format) and extend any fields with unique labels in no particular order. For more information about the LTSV definition and format, see http://ltsv.org/.
LTSV Parser Configuration
ltsv
parser name in the configuration.
[parser|myltsv] base_parser=ltsvAn LTSV file must be a byte sequence that matches the LTSV production in the ABNF format.
ltsv = *(record NL) [record] record = [field *(TAB field)] field = label ":" field-value label = 1*lbyte field-value = *fbyte TAB = %x09 NL = [%x0D] %x0A lbyte = %x30-39 / %x41-5A / %x61-7A / "_" / "." / "-" ;; [0-9A-Za-z_.-] fbyte = %x01-08 / %x0B / %x0C / %x0E-FF
host:127.0.0.1<TAB>ident:-<TAB>user:frank<TAB>time:[10/Oct/2000:13:55:36 -0700]<TAB>req:GET /apache_pb.gif HTTP/1.0<TAB>status:200<TAB>size:2326<TAB>referer:http://www.example.com/start.html<TAB>ua:Mozilla/4.08 [en] (Win98; I ;Nav)
host=127.0.0.1 ident=- user=frank time=[10/Oct/2000:13:55:36 -0700] req=GET /apache_pb.gif HTTP/1.0 status=200 size=2326 referer=http://www.example.com/start.html ua=Mozilla/4.08 [en] (Win98; I ;Nav)
Debug Configuration
Additional debugging is also available for the LTSV parser. By default, LTSV debugging is deactivated. To turn on LTSV debugging, enter debug=yes
.
[parser|myltsv] base_parser=ltsv debug=yes
When debugging is turned on, the LTSV parser extracts values of all valid labels from the log. The LTSV parser requires that label names consist only of alpha-numeric characters, the underscore ('_"), dot ('.') and dash ('-') characters. If at least one invalid label name exists in the log, its parsing will fail. Even if the label name is valid, the agent will check the field name. If invalid names exist, the label name should be corrected to a valid field name.
Configuring the LTSV Parser from the filelog
Section
You can also configure the LTSV parser from the filelog
section directly.
[filelog|simple_logs] directory=/var/log include=* parser=ltsv