The project-keep operator in APL is a powerful tool for field selection. It allows you to explicitly keep specific fields from a dataset, discarding any others not listed in the operator’s parameters. This is useful when you only need to work with a subset of fields in your query results and want to reduce clutter or improve performance by eliminating unnecessary fields.

You can use project-keep when you need to focus on particular data points, such as in log analysis, security event monitoring, or extracting key fields from traces.

For users of other query languages

If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.

Usage

Syntax

| project-keep FieldName1, FieldName2, ...

Parameters

  • FieldName: The field you want to keep in the result set.

Returns

project-keep returns a dataset with only the specified fields. All other fields are removed from the output. The result contains the same number of rows as the input table.

Use case examples

For log analysis, you might want to keep only the fields that are relevant to investigating HTTP requests.

Query

['sample-http-logs'] 
| project-keep _time, status, uri, method, req_duration_ms

Run in Playground

Output

_timestatusurimethodreq_duration_ms
2024-10-17 10:00:00200/index.htmlGET120
2024-10-17 10:01:00404/non-existent.htmlGET50
2024-10-17 10:02:00500/server-errorPOST300

This query filters the dataset to show only the request timestamp, status, URI, method, and duration, which can help you analyze server performance or errors.

  • project: Use project to explicitly specify the fields you want in your result, while also allowing transformations or calculations on those fields.
  • extend: Use extend to add new fields or modify existing ones without dropping any fields.
  • summarize: Use summarize when you need to perform aggregation operations on your dataset, grouping data as necessary.

Wildcard

Wildcard refers to a special character or a set of characters that can be used to substitute for any other character in a search pattern. Use wildcards to create more flexible queries and perform more powerful searches.

The syntax for wildcard can either be data* or ['data.fo']*.

Here’s how you can use wildcards in project-keep:

['sample-http-logs']
| project-keep resp*, content*,  ['geo.']*

Run in Playground

['github-push-event']
| project-keep size*, repo*, ['commits']*, id*

Run in Playground