extend-valid
Creates calculated columns that are used to filter out rows. If valid it then appends them to the result set.
Creates calculated columns that are used to filter out rows. If valid it then appends them to the result set.
This is a shorthand operator to create a field while also doing basic checking on the validity of the field. In many cases, additional checks are required and it is recommended in those cases a combination of an extend and a where operator are used. The basic checks that are preformed are dependent on the type of the expression. Checks are as follows:
- Dictionaries. The valid is checked to ensure it is not null and has at least one entry in the dictionary.
- Arrays. Arrays should not be null and should have at least one value.
- Strings. Strings should not be empty and have at least one character in them.
- Other types. The same logic as tobool and a check for true
Some situations you’d want to not use extend-valid for including checking to see if a map has a particular key in it. Cases where an empty array is valid. Where returning and empty string could be a useful result. Or when a value of a number being 0 should be included in the new field.
Syntax
| extend-valid alias1 = expression1, alias2 = expression2, alias3 = ...
Arguments
name | type | description |
---|---|---|
Alias | string | The name of the column to add or update. |
Expression | expression | A calculation over the columns of the dataset. |
Returns
Rows in dataset for which Expression is valid with:
- Column names noted by extend that already exist in the input are removed and appended as their new calculated values.
- Column names noted by extend that do not exist in the input are appended as their new calculated values.
Examples
['http-logs']
// if content_type is "" this will filter the row
| extend-valid upper_ct = toupper(content_type)
['http-logs']
// if extract us unable to find the regex, this will filter the row
| extend-valid extension = extract('/([a-z]*)', 1, content_type)