Parser operators are used to isolate values from a string. There are two classes of parsers, simple and complex.
Simple parsers perform a very specific operation where the result is assigned to a predetermined field in the log data model.
For example, the time_parser
will parse a timestamp and assign it to the a log record's Timestamp
field.
List of simple parsers:
Complex parsers differ from simple parsers in several ways.
- Parsing produces multiple key/value pairs, where the values may be strings, ints, arrays, or objects.
- By default, these key/value pairs will be added to log entry's
attributes
field. In the case of collisions between keys, the newly parsed value will override the existing value. Alternately, the field to which the object is written can be configured using theparse_to
setting. - The configuration may "embed" certain followup operations. Generally, these operations correlate to the simple parsers listed above. Embedded operations are applied immediately after the primary parsing operation, and only if the primary operation was successful. Each embedded operation is executed independently of the others. (e.g. failure to parse a timestamp will not prevent an attempt to parse severity.)
The following examples illustrate how a json_parser
may embed timestamp and severity parsers.
Consider a simple json log: {"message":"foo", "ts":"2022-08-10", "sev":"INFO"}
# Standalone json parser
- type: json_parser
# Regex parser with embedded timestamp and severity parsers
- type: json_parser
timestamp:
parse_from: attributes.ts
layout_type: strptime
layout: '%Y-%m-%d'
severity:
parse_from: attributes.sev
Note that when configuring embedded operations, it is typically necessary to reference a field that was set by the primary operation. In the above example, the values specified in the parse_from
fields have taken into account that the json_parser
will write key/value pairs to attributes
.
List of complex parsers:
List of embeddable operations:
timestamp
severity
trace
scope_name
body
: A field that should be assigned to a the log body.