This topic describes how to use wildcards when building Data Forwarder custom query filters.

Data Forwarder custom query filters supports two wildcard characters:

  • * matches 0 or more characters.

    For example, process_path:*\\powershell.exe returns any process path that ends in \powershell.exe.

  • ? matches a single character.

    For example, process_path:*temp?comand* returns any process path that contains temp and command having a single character in between. This example matches Windows (\temp\command\) and macOS or Linux (/temp/command), but can also include unexpected directories (/stempocommandcenter/).

Tokenization

A tokenized field is a field that can be searched by using a partial phrase.

A tokenized field enables full text search on any phrase that occurs anywhere in the field. This phrase is an alphanumeric substring that exists between special characters.

To determine which fields are tokenized, see the Data Forwarder Schema (external link).

Both Carbon Black Cloud Data Forwarder custom query filters and the Carbon Black Cloud console Investigate search engine use Lucene syntax and wildcarding. However, the field tokenization differs between the two instances.

For example, using Carbon Black Cloud console Investigate search, the following fields are tokenized:

process_username:SVCSL1*MONPrd

In this case, the syntax matches and returns a process_username of CARLYLEDC\SVCSL11MONPrd.

When using Carbon Black Cloud Data Forwarder custom query filters, fewer fields are tokenized. When filtering, most fields (that is, those that are not tokenized) require leading wildcards to achieve a similar result.

For example, process_username:*SVCSL1*MONPrd is the best way to match a process_username of CARLYLEDC\SVCSL11MONPrd.

Primary Wildcard Use Cases

The following examples show some of the most common use cases for using wildcards in Data Forwarder custom query filters.

  • Example 1: Existence testing

    alert_id:*

    Data Forwarder does not emit empty data fields. Therefore, to forward only those events that have an associated alert_id, use an INCLUDE filter that specifies alert_id:*. Only events that include an alert_id value are forwarded.

    You can use this field:* filtering approach with any FILTERABLE field.

  • Example 2: Substring matching

    process_path:*\\powershell.exe

    To construct filters that have complicated strings and wildcards to match a narrow subset of records from the Data Forwarder, we recommend that you use substring matching.

    For example, setting the example filter ensures that you only receive events that have process_path ending in \powershell.exe.

Wildcards Usage with EXCLUDE versus INCLUDE Filters

Data Forwarder custom query results that use EXCLUDE versus INCLUDE filters are opposite to one another.

Using a field:value query in an INCLUDE filter ensures that only those matching records are emitted; using the same field:value query in an EXCLUDE filter omits those matching records.

Using the example of process_path:*\\powershell.exe, you can extrapolate this effect for each type of filter.

  • Include: Yields any process path that ends in \powershell.exe.
  • Exclude: Yields any process path that does not end in \powershell.exe.

Wildcard Anti-Patterns

Lucene query syntax is a powerful tool for filtering Data Forwarder records. However, in a streaming context, certain types of queries are ineffective or redundant.

  • Example 1: Wildcards on ENUM fields

    type:*load

    When using wildcards on fields that contain ENUM values such as action or process_publisher_state, you can accidentally construct a filter that is more permissive or restrictive than intended. type:*load would yield results for events of type endpoint.event.moduleload and endpoint.event.scriptload, but also would yield events of type endpoint.event.fileless_scriptload. To receive events that have particular ENUM field values, query specifically for the combination of those types: for example, type:endpoint.event.scriptload OR type:endpoint.event.fileless_scriptload .

  • Example 2: Wildcard INCLUDE filters on everything

    org_key:*,process_username:*,event_origin:*

    This filter is redundant. The Carbon Black Cloud Data Forwarder assumes "send everything" by default, and does not require INCLUDE filters. To simply forward everything except a handful of specific queries, specify EXCLUDE filters and specify no INCLUDE filters.