Delete blocks of data

This page explains how to delete blocks of data.

After data ingestion, Axiom stores events in blocks of tens of thousands of events. Currently, you can’t delete individual events after you send them to Axiom, but you can delete entire blocks of data. With the /v2/datasets/_apl/delete endpoint, you can specify an APL query and delete all blocks that contain matching events.

Warning

The /v2/datasets/_apl/delete endpoint deletes all blocks containing events that match your query. For example, even if a block only contains one event that matches your query, the endpoint removes all events in the block. For this reason, using this endpoint may remove more data than your query specifies.

Deleting data using this endpoint is slow and expensive. Only use this endpoint in exceptional cases. For example, when you have sent sensitive data to Axiom by mistake and you need to immediately delete it for regulatory reasons.

Use the block deletion endpoint for deleting less than one million rows in one request. To delete more than one million rows, send a number of requests and delete a limited number of rows in each request. If you delete too many rows in one request, the request might fail.

Delete blocks

To delete all blocks containing events that match an APL query, send a POST request to /v2/datasets/_apl/delete.

To authenticate your request, use an API token with update permissions for datasets. For more information, see API tokens.

In the request body, include a JSON object with the following fields:

apl: An APL query that specifies the events to be deleted. You can only use the operators where and search.
startTime: The start of the time period over which you wantto run the query.
endTime: The end of the time period over which you want to run the query.
commit: A Boolean that specifies whether to actually perform the deletion. Set commit to false to preview the impact of the deletion request without actually deleting any events.

A valid request receives a 200 response with the following fields present in the response body:

dryRun: Returns true if the value of commit specified in the request was false.
rowsMatched: The number of matching events.
rowsDeleted: The number of deleted events. This is the number of all events in all the blocks that contain events matching your query. The number of deleted events can be higher than the number of matching events.
firstMatchedEvent: The timestamp of the first event matching the query.
lastMatchedEvent: The timestamp of the last event matching the query.

When you make a successful request with commit set to true, blocks containing matching events are queued for deletion. The action can’t be reversed. A block queued for deletion is typically deleted within 5 minutes, but it may take longer if your organization makes a large number of deletion requests.

Best practices

Start with a dry run. Set commit to false to safely review the number of events that your request is about to delete.
Deletions are slow, resource-intensive, and costly operations. Use them rarely.
After deleting data, vacuum the fields of the dataset to clean up the dataset completely.

Example request

shell

curl -X 'POST' 'https://api.axiom.co/v2/datasets/_apl/delete' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d  '{
  "apl": "DATASET_NAME | where trace_id == \"b630e23cfc3a6ac9e57fee13542b3e50\"",
  "startTime": "now-6h",
  "endTime": "now",
  "commit": false
  }'

Example response

shell

{
    "dryRun": true,
    "firstMatchedEvent": "2024-08-25 00:18:19 +0000 UTC",
    "lastMatchedEvent": "2024-08-25 00:18:19 +0000 UTC",
    "rowsDeleted": 10249,
    "rowsMatched": 1
}