project-away
This page explains how to use the project-away operator function in APL.
The project-away operator in APL is used to exclude specific fields from the output of a query. This operator is useful when you want to return a subset of fields from a dataset, without needing to manually specify every field you want to keep. Instead, you specify the fields you want to remove, and the operator returns all remaining fields.
You can use project-away in scenarios where your dataset contains irrelevant or sensitive fields that you do not want in the results. It simplifies queries, especially when dealing with wide datasets, by allowing you to filter out fields without having to explicitly list every field to include.
For users of other query languages
If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.
Usage
Syntax
Parameters
FieldName: The field you want to exclude from the result set.
Returns
The project-away operator returns the input dataset excluding the specified fields. The result contains the same number of rows as the input table.
Use case examples
In log analysis, you might want to exclude unnecessary fields to focus on the relevant fields, such as timestamp, request duration, and user information.
Query
Output
| _time | req_duration_ms | id | geo.city | geo.country |
|---|---|---|---|---|
| 2023-10-17 10:23:00 | 120 | u1 | Seattle | USA |
| 2023-10-17 10:24:00 | 135 | u2 | Berlin | Germany |
The query removes the status, uri, and method fields from the output, keeping the focus on the key fields.
In log analysis, you might want to exclude unnecessary fields to focus on the relevant fields, such as timestamp, request duration, and user information.
Query
Output
| _time | req_duration_ms | id | geo.city | geo.country |
|---|---|---|---|---|
| 2023-10-17 10:23:00 | 120 | u1 | Seattle | USA |
| 2023-10-17 10:24:00 | 135 | u2 | Berlin | Germany |
The query removes the status, uri, and method fields from the output, keeping the focus on the key fields.
When analyzing OpenTelemetry traces, you can remove fields that aren’t necessary for specific trace evaluations, such as span IDs and statuses.
Query
Output
| _time | duration | trace_id | service.name | kind |
|---|---|---|---|---|
| 2023-10-17 11:01:00 | 00:00:03 | t1 | frontend | server |
| 2023-10-17 11:02:00 | 00:00:02 | t2 | checkoutservice | client |
The query removes the span_id and status_code fields, focusing on key service information.
In security log analysis, excluding unnecessary fields such as the HTTP method or URI can help focus on user behavior patterns and request durations.
Query
Output
| _time | req_duration_ms | id | status | geo.city | geo.country |
|---|---|---|---|---|---|
| 2023-10-17 10:25:00 | 95 | u3 | 200 | London | UK |
| 2023-10-17 10:26:00 | 180 | u4 | 404 | Paris | France |
The query excludes the method and uri fields, keeping information like status and geographical details.
Wildcard
Wildcard refers to a special character or a set of characters that can be used to substitute for any other character in a search pattern. Use wildcards to create more flexible queries and perform more powerful searches.
The syntax for wildcard can either be data* or ['data.fo']*.
Here’s how you can use wildcards in project-away:
List of related operators
- project: The
projectoperator lets you select specific fields to include, rather than excluding them. - extend: The
extendoperator is used to add new fields, whereasproject-awayis for removing fields. - summarize: While
project-awayremoves fields,summarizeis useful for aggregating data across multiple fields.