Basic Rules

Rule Chain (rules): The rule chain is the core of the Event Log Processor. It basically evaluates a list of rules as described in Architectural Overview. It does not require any specific configuration, but accepts name and override parameters as all rules do. A rule chain always forwards the result of the last evaluated rule.

<rules>
<rule name="Rule 1" />
<rule name="Rule 2" />
<rule name="Rule 3" />
</rules>

Include Rule (include): The include rule allows the administrator to break a large config file into multiple smaller files. This rule allows the use of patterns, which is a quick way to include many files. When a pattern selects multiple files, they are included in alphabetical order. The path to the included files is relative to the configuration file that references them.

<include name="Includes XML files in rules directory" dir="rules" includes="*.xml" />

Switch Rule (switch): The switch rule is a quick way to redirect the evaluation flow to a specific rule chain based on a field value. It tests the value of a field against different cases and evaluates the rules for the matching one. If a default is specified and the field value does not match any of the cases, the default rules are evaluated. By default, the comparison performed is strict , meaning that no type conversion is attempted (which means that 42 : 0 6 = 42 6 = "42" ). However, with the strict attribute set to false , the rule will take care of converting the data before performing the comparison. The strict can be set separately on each case and/or globally for the whole switch.

<switch name="Switches based on Type field value" field="Type">
<case value="ERROR" type="STRING">
...
</case>
<default>
...
</default>
</switch>

Forward Rule (forward): The forward rule allows the propagation of events to other Processing Elements. The name of the output stream is defined by the stream attribute. Declaring this rule also allows the Event Processing Manager to link the given output stream name to another Processing Element . Any attempt to use an output stream that has not been declared in a forward rule will result in an error. The forward rule always returns success.

<forward name="Forwards events on the data stream" stream="data" />

Noop Rule (noop): The noop rule does nothing (no operation). However, its result is configurable, which allows its use as a DROP or ACCEPT rule. The rule result is configured with the result attribute.

<noop name="This rule does nothing" />
<noop name="This rule drops every incoming event" result="failure" />

Log Rule (log): The log rule prints events in a log file. This is mainly a troubleshooting rule that allows the administrator to understand what's going on during the evaluation. It always returns continue.

This rule uses the standard Java Logging API. The logger's name used by this rule can be configured with the logger attribute. If you modify your Event Processing Manager logging configuration ( conf/logging.properties ), this allows you, for example, to redirect logs from a specific log rule to different files. See http:// download.oracle.com/javase/6/docs/technotes/guides/logging/overview.html for more information.

<log name="Log these events" />
<log name="Log these events in another logger" logger="another.logger" />

Count Rule (count): The count rule is a specific instance of a log rule which periodically evaluates the current event rate. It uses the same logger attribute to change the logger and the period attribute to determine how often (in milliseconds) the event rate should be logged. This rule always returns continue.

<count name="Count these events and log every 5s" period="5000" /> 

Copy Rule (copy): The copy rule simply copies a field into another. This is useful to rename or reuse some fields. The field to be copied is defined with the field attribute and the destination field with the to attribute. Furthermore, the new copy can be converted in a specific output type with the type attribute. If this attribute is omitted, the original field type will be used. This rule returns continue if the field exists or else failure .

<copy name="Copy a to b" field="a" to="b" />

Set Rule (set): The set rule allows the definition of constant fields. It will set the value of the value attribute of the field defined in the to attribute. It optionally uses the datatype defined in type in order to not use the default string type. This rule always returns continue. If value is omitted, the resulting field will be null.

<set name="Sets a string field F containing V" to="F" type="STRING" value="V" />
<set name="Sets an integer field F containing 42" to="F" type="INT" value="42" />

Tagging Rules

Timestamp Rule (timestamp): The timestamp rule sets a field to the current time in milliseconds. The destination field is defined with the to attribute. To allow maximum performance, this rule accepts an accuracy attribute which defines how accurate the current time should be. For example, if accuracy is set to 0, the current time will be as accurate as possible, but if it is set to 1000, the current time will only be reliable up to the second. This rule always returns continue.

<timestamp name="Sets the current time in F" to="F" accuracy="100" />
<timestamp name="Sets the current time in F with the maximum accuracy" to="F" accuracy="
0" />

UUID Rule (uuid): The UUID rule sets a field to a newly random UUID (universal unique identifier). The destination field is defined with the to attribute. This rule always returns continue .

<uuid name="Generates a random UUID in F" to="F" />

Sequence Rule (seq): The sequence rule generates a sequence and sets the current value to the given field. The last attributed sequence number is not persisted, so you should not use it as an id. The destination field is defined with the to attribute. The initial sequence value can be set with the start attribute. This rule always returns continue .

<seq name="Generates sequence in F starting at 42" to="F" start="42" />

Id Generator Rule (id): The id generator rule is a combination of timestamp and sequence that allows the generation of a unique id (since the system clock is always increasing). It uses the same attributes used by the sequence rule and by the timestamp rule. This rule always returns continue.

<id name="Generates an id in Id" to="Id" />

Link Rule (link): The link rule is used to create a link between successive events. It is done by setting a field in each event that corresponds to the previous event id. The event id is set with the id attribute and the destination field with the to attribute. If the id is missing, the rule returns failure or else it returns continue .

<link name="Copies Id to pId in the next event" id="Id" to="pId" />

MAC Rule (mac): The MAC rule allow the creation of a Message Authentication Code for each event to protect their integrity. Combined with the link rule, this is a powerful tool to detect missing or altered events. A MAC rule uses a key and a list of fields to create the code. The code is added to the event as a binary field, whose name is defined by the to attribute. The source fields are set with the nested field elements. The MAC algorithm can be customized using the algorithm attribute. Each algorithm uses a key, which is specified using the key attribute. The key attribute format is base64:<base 64 representation of the key> and can optionally be encrypted using DCF crypt-password command.

<mac name="Creates a MAC of the Id,pId,Message of each events" key="base64:a2V5Cg==" to=
"mac" algorithm="HMac-MD5">
<field name="Id" type="LONG" />
<field name="pId" type="LONG" />
<field name="Message" type="STRING" encoding="ASCII" />
</mac>
Format Rule (format): The format rule allows the combination of multiple fields to create a new one using a format string. The destination field is set with the to attribute and the format string with the pattern attribute. The rule support two types of format string (set with the type attribute):
  • message is the default format and uses a Java MessageFormat . For more information on the format string syntax, refer http://download.oracle. com/javase/6/docs/api/java/text/MessageFormat.html.

  • printf uses a Java Formatter which format string syntax is similar the the wellknown C printf . For more information on the format string syntax, refer http://download.oracle.com/javase/6/docs/api/java/util/Formatter.html. The format string usually contains placeholders to insert some field values (for example, %s ). Each placeholders must be attribute its value with a nested field element. By default, each field is handled as a string , however, some placeholders might expect different data types (for example, %d would expect an integer ). You can force a type conversion with the type attribute (one of INT, SHORT, FLOAT, DOUBLE, LONG, NUMERIC, STRING, BOOLEAN, BYTE, BYTES or OBJECT is expected).

    <format name="Prints Id=Type into IdType" pattern="{0}={1}" to="IdType" type="message">
    <field name="Id" type="LONG" />
    <field name="Type" type="STRING" />
    </format>
    

Matching Rules

Exists Rule (exists): This rule returns continue if a specific field (designated by the field attribute) exists. If not, it returns failure . The allow-null attribute determines the behavior if the field exists but is null. The default is false which treats null as if the field does not exist.

<exists name="Match messages with a field f" field="f" allow-null="false" />

Starts With Rule (starts-with): This rule returns continue if a specific field (designated by the field attribute) start with the value defined in the prefix attribute. If not, it returns failure.

<starts-with name="Match messages starting with ERROR" field="Message" prefix="ERROR" /> 

Contains Rule (contains): This rule returns continue if a specific field (designated by the field attribute) contains the value defined in the value attribute. If not, it returns failure.

<contains name="Match messages containing ERROR" field="Message" value="ERROR" /> 

Equals Rule (equals): This rule returns continue if a specific field (designated by the field attribute) is equal to the value defined in the value attribute. If not, it returns failure. By default, the comparison performed is strict , meaning that no type conversion is attempted (which means that 42 : 0 6 = 42 6 = "42" ). However, with the strict attribute set to false , the rule will take care of converting the data before performing the comparison.

<equals name="Match messages which type is ERROR" field="type" value="ERROR" type="
STRING" />
<equals name="Match severities of 0, 0.0, ..." field="severity" value="0" type="INT"
strict="false" />

Not Equals Rule (not-equals): This rule is the exact opposite of the Equals Rule ( equals ) . It returns failure if a specific field (designated by the field attribute) is equal to the value defined in the value attribute. If not, it returns continue . This rule also support the strict attribute of the Equals Rule ( equals ).

<not-equals name="Match messages which type is not DEBUG" field="type" value="DEBUG"
type="STRING" />

Regex Rule (regex): The regex rule has two main purposes: match a field value against a regular expression and optionally extract some data from it. If the regular expression given in the pattern is found in the field designated by the field attribute, this rule will try to extract some data from the value and return continue . If the value doesn't match, the rule will return failure . The data extraction is configured with a nested extraction element whose name attribute is the destination field and value attribute the extraction string. The extraction string can refer to the group matched in the field value using the $ <ref> syntax. If it is missing, a tail extraction is performed, meaning that everything after the match is extracted. For complete documentation of the supported regular expression dialect, refer the Java documentation.

<regex name="Extracts a type from a message" field="Message" pattern="^(ERROR|INFO): ">
<extraction name="Type" value="$1" />
<extraction name="Message" />
</regex>
<regex name="Match messages starting with ERROR" field="Message" pattern="^ERROR: ">

Parsing Rules

Date Parser Rule (parse-date): This rule converts the string representation of a date into a timestamp in milliseconds. The date format can be configured using the pattern and locale attributes. When using the default parser, the pattern must be compatible with the standard Java data parser. When the type attribute is set to rfc3164 , the pattern is not required and the parser will interpret RFC 3164 TIMESTAMP type (which imply guessing the year as it is not included in a RFC 3164 TIMESTAMP). The locale must be a RFC 3066 compliant locale id. It is used to decode some locale dependent elements of the date (e.g. the month name). Optionally, a time-zone can be specified with the timezone attribute. This is the time-zone of the date to parse when it is not specified in the date itself. The field to parse is designated by the attribute field and the destination field by the attribute to . If to is omitted, field is used (this overwrites the previous value). This rule returns continue if the date field has been successfully parsed, failure if not.

<parse-date name="Parses a date" field="Date" to="Timestamp" pattern="MMM dd yyyy 
HH:mm:ss" />
<parse-date name="Parses a date in GMT" field="Date" to="Timestamp" timezone="GMT"
pattern="MMM dd yyyy HH:mm:ss" />

Duration Parser Rule (parse-duration): This rule converts a duration in the format [[hh:]mm:]ss into a number of milliseconds. The field to parse is designated by the attribute field and the destination field by the attribute to . If to is omitted, field is used (this overwrites the previous value). The separator can be changed with the attribute separator . This rule returns con- tinue if the duration field has been successfully parsed, failure if not.

<parse-duration name="Parses a duration" field="Duration" to="DurationMsec" />

Syslog Priority Parser Rule (parse-pri): This rule converts a Syslog Priority field of a syslog record into its corresponding Severity and Facility as described in the RFC 5424 . The field to parse is designated by the attribute field and the destination field by the attributes facility and severity. This rule returns continue if the Priority field has been successfully parsed, failure if not.

<parse-pri name="Parses a Syslog Priority" field="Pri" facility="Facility" severity="
Severity" />

IPv4 To Integer Rule (ipv4-to-int): This rule converts the string representation of an IP address into a compact 4 byte representation. This is especially useful for IP addresses stored in a database, as it allows more compact data and faster queries. The field to parse is designated by the attribute field and the destination field by the attribute to . If to is omitted, field is used (this overwrites the previous value). This rule returns continue if the IP address has been successfully converted, failure if not.

<ipv4-to-int name="Parses an IPv4" field="Address" />

Integer To IPv4 Rule (int-to-ipv4): This rule converts the integer representation of an IP address into a standard dot notation. This is especially useful for IP addresses coming from a Flow Listener. The field to parse is designated by the attribute field and the destination field by the attribute to . If to is omited, field is used (this overwrites the previous value). This rule returns continue if the IP address has been successfully converted, failure if not.

<int-to-ipv4 name="Integer to IPv4" field="Address" />

Advanced Rules

Tokenizer Rule (tokenize): The tokenize rule generates token events by splitting a field into tokens. This rule can be used to create a token stream in order to build a keyword index and make the search more user-friendly. The tokenized field can be set with the field attribute. and the tokenizer type with the type attribute. Currently, the tokenize rule supports two tokenizer types:
  • simple uses some stop characters (set with the pattern attribute) to split the field into tokens.

  • lucene uses the well known Lucene tokenizer components to split the message. The underlying Lucene tokenizer is designated by the class attribute. A tokenize rule acts as a rule chain. Token events generated by the tokenizer rule will be submitted to the nested rule chain. A typical use case is to forward them to a specific output stream in order to process them separately like in the following examples (note the on-success="continue" to make sure that all tokens are forwarded and that evaluation continues after the tokenize rule):
    <tokenize field="Message" to="Token" type="simple" pattern="[ \r\n\t]">
    <forward name="Forwards tokens on the token stream" on-success="continue" stream="
    token" />
    </tokenize>
    <tokenize field="Message" to="Token" type="lucene" class="org.apache.lucene.analysis.
    SimpleAnalyzer">
    <forward name="Forwards tokens on the token stream" on-success="continue" stream="
    token" />
    </tokenize>
    

Custom Rule (custom): The custom rule is an extension point for Java developers that would like to create their own rules. The only required attribute is class pointing to the custom rule class name. Every other attribute or element is private to the custom rule. A custom rule must extend the com.watch4net.events.processing.processor.rules.CustomRule class from event-log-processor.jar to be recognized by the Event Log Processor.

<custom name="A custom rule" class="com.company.MyRule" setting="value">
<parameter>value</parameter>
</custom>