What is the search operator in APL

The search operator in APL is a special operator that scans full-text search over the specified datasets. Instead of specifying a particular field or condition, the search operator allows for a broad search across multiple fields for a given text or phrase.

Importance of the search operator:

  • Versatility: It allows users to find a specific text or term across various fields within a dataset that they choose or select for their search, without the necessity to specify each field.

  • Efficiency: Saves time when you aren’t sure which field or datasets in APL might contain the information you are looking for.

  • User-Friendliness: It’s particularly useful for users or developers unfamiliar with the schema details of a given database.

Search operator syntax

search [kind=<Search Kind>] <Search-Expression>

or

search [kind= CaseSensitivity ] SearchPredicate

Parameters

NameTypeRequiredDescription
DatasetstringThe data to be searched over in a dataset.
CaseSensitivity or Search KindstringA flag that controls the behavior of all string scalar operators, such as has, with respect to case sensitivity. Valid values are default, case_insensitive, case_sensitive. The options default and case_insensitive are synonymous, since the default behavior is case insensitive.
SearchPredicate or SearchExpressionstringA boolean expression to be evaluated for every event in the input. If it returns true, the record is outputted.

Search predicate syntax

The SearchPredicate allows you to search for specific terms in all fields of a dataset. The operator that will be applied to a search term depends on the presence and placement of a wildcard asterisk (*) in the term, as shown in the following table.

LiteralOperator
axiomkhas
*axiomkhassuffix
axiomk*hasprefix
*axiomk*contains
ax*igmatches regex

You can also restrict the search to a specific field, look for an exact match instead of a term match, or search by regular expression. The syntax for each of these cases is shown in the following table.

SyntaxExplanation
FieldName:StringLiteralThis syntax can be used to restrict the search to a specific field. The default behavior is to search all fields.
FieldName==StringLiteralThis syntax can be used to search for exact matches of a field against a string value. The default behavior is to look for a term-match.
Field matches regex StringLiteralThis syntax indicates regular expression matching, in which StringLiteral is the regex pattern.

Use boolean expressions to combine conditions and create more complex searches. For example, "axiom" and b==789 would result in a search for events that have the term axiom in any field and the value 789 in the b field.

Search predicate syntax examples

#SyntaxMeaning (equivalent where)Comments
1search "axiom"where * has "axiom"
2search field:"axiom"where field has "axiom"
3search field=="axiom"where field=="axiom"
4search "axiom*"where * hasprefix "axiom"
5search "*axiom"where * hassuffix "axiom"
6search "*axiom*"where * contains "axiom"
7search "Pad*FG"where * matches regex @"\bPad.*FG\b"
8search *where 0==0
9search field matches regex "..."where field matches regex "..."
10search kind=case_sensitiveAll string comparisons are case-sensitive
11search "axiom" and ("log" or "metric")where * has "axiom" and (* has "log" or * has "metric")
12search "axiom" or (A>a and A<b)where * has "axiom" or (A>a and A<b)
13search "AxI?OM"where * matches regex @"\bAxI.OM\b"? matches a single character
14search "axiom" and not field:"error"where * has "axiom" and not field has "error"Excluding a field from the search

Examples:

Search for a term over the dataset in scope.

['sample-http-logs']
| search "image"

Run in Playground

Search for records that match both terms in the dataset.

['sample-http-logs']
| search "jpeg" and ("GET" or "true")

Run in Playground

Search for events that match both case-sensitive terms in the dataset.

['sample-http-logs']
| search kind=case_sensitive "css"

Run in Playground

Search specific Fields

Search for a term in the method and user_agent fields in the dataset.

['sample-http-logs']
| search method:"GET" or user_agent :"Mozilla"

Run in Playground

Limit search by timestamp

Search for a term over the dataset if the term appears in an event with a date greater than the given date.

['sample-http-logs']
| search "get" and _time > datetime('2022-09-16')

Run in Playground

Using kind=default

By default, the search is case-insensitive and uses the simple search.

['sample-http-logs']
| search kind=default "INDIA"

Run in Playground

Using kind=case_sensitive

Search for logs that contain the term “text” with case sensitivity.

['sample-http-logs']
| search kind=case_sensitive "text"

Run in Playground

Using kind=case_insensitive

Explicitly search for logs that contain the term “CSS” without case sensitivity.

['sample-http-logs']
| search kind=case_insensitive "CSS"

Run in Playground

Search all logs. This would essentially return all rows in the dataset.

['sample-http-logs']
| search *

Run in Playground

Contain any substring

Search for logs that contain any substring of “brazil”.

['sample-http-logs']
| search "*brazil*"

Run in Playground

Search for Multiple Independent Terms

Search the logs for entries that contain either the term “GET” or “covina”, irrespective of their context or the fields they appear in.

['sample-http-logs']
| search "GET" or "covina"

Run in Playground

Use the search operator efficiently

Using non-field-specific filters such as the search operator has an impact on performance, especially when used over a high volume of events in a wide time range. To use the search operator efficiently, follow these guidelines:

  • Use field-specific filters when possible. Field-specific filters narrow your query results to events where a field has a given value. They are more efficient than non-field-specific filters, such as the search operator, that narrow your query results by searching across all fields for a given value. When you know the target field, replace the search operator with where clauses that filter for values in a specific field.
  • After using the search operator in your query, use other operators, such as project statements, to limit the number of returned fields.
  • Use the kind flag when possible. When you know the pattern that string values in your data follow, use the kind flag to specify the case-sensitivity of the search.