Pagination in Axiom API

Learn how to use pagination with the Axiom API.

Pagination allows you to retrieve responses in manageable chunks.

You can use pagination for the following endpoints:

Prerequisites

Create an Axiom account.
Create a dataset in Axiom where you send your data.
Create an API token in Axiom with permissions to ingest data to the dataset you have created.

Pagination mechanisms

You can use one of the following pagination mechanisms:

Pagination based on timestamp (stable)
Pagination based on cursor (public preview)

Axiom recommends timestamp-based pagination. Cursor-based pagination is in public preview and may return unexpected query results.

Timestamp-based pagination

The parameters and mechanisms differ between the current and legacy endpoints.

Run Query

To use timestamp-based pagination with the Run Query endpoint:

Include the limit operator in the APL query of your API request. The argument of this operator determines the number of events to display per page.
Use sort by _time asc or sort by _time desc in the APL query. This returns the results in ascending or descending chronological order. For more information, see sort operator.
Specify startTime and endTime in the body of your API request.

Run Query (Legacy)

To use timestamp-based pagination with the legacy Run Query endpoint:

Add the limit parameter to the body of your API request. The value of this parameter determines the number of events to display per page.
Add the order parameter to the body of your API request. In the value of this parameter, order the results by time in either ascending or descending chronological order. For example, [{ "field": "_time", "desc": true }]. For more information, see order operator.
Specify startTime and endTime in the body of your API request.

Page through the result set

Use the timestamps as boundaries to page through the result set.

Queries with descending order

To go to the next page of the result set for queries with descending order (_time desc):

Determine the timestamp of last item on the current page. This is the least recent event.
Optional: Subtract 1 nanosecond from the timestamp.
In your next request, change the value endTime parameter in the body of your API request to the timestamp of the last item (optionally, minus 1 nanosecond).

Repeat this process until the result set is empty.

Queries with ascending order

To go to the next page of the result set for queries with ascending order (_time asc):

Determine the timestamp of last item on the current page. This is the most recent event.
Optional: Add 1 nanosecond to the timestamp.
In your next request, change the value startTime parameter in the body of your API request to the timestamp of the last item (optionally, plus 1 nanosecond).

Repeat this process until the result set is empty.

In the procedures above, the steps about incrementing the timestamp are optional. If you increment the timestamp, there is a risk of duplication. If you don’t increment the timestamp, there is a risk of overlap. Duplicated data is possible for many reasons, such as backfill or natural duplication from external data sources. For these reasons, regardless of the method you choose (increment or not increment the timestamp, sort by descending or ascending order), Axiom recommends you implement some form of deduplication mechanism in your pagination script.

Limits

Both the Run Query and the Run Query (Legacy) endpoints allow request-based limit configuration. This means that the limit they use is the lowest of the following: the query limit, the request limit, and Axiom’s server-side internal limit. Without a query or request limit, Axiom currently defaults to the limit of 1,000 events per page. For the pagination of datasets that are greater than 1,000 events, Axioms recommends specifying the same limit in the request and the APL query to avoid the default value and contradictory limits.

Examples

Example request Run Query

shell

curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "apl": "DATASET_NAME | sort by _time desc | limit 100",
    "startTime": "2024-11-30T00:00:00.000Z",
    "endTime": "2024-11-30T23:59:59.999Z"
  }'

Example request Run Query (Legacy)

shell

curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/query' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "startTime": "2024-11-30T00:00:00.000Z",
    "endTime": "2024-11-30T23:59:59.999Z",
    "limit": 100,
    "order": [{ "field": "_time", "desc": true }]
  }'

Example request to page through the result set

Example request to go to next page for Run Query:

shell

curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "apl": "DATASET_NAME | sort by _time desc | limit 100",
    "startTime": "2024-11-30T00:00:00.000Z",
    "endTime": "2024-11-30T22:59:59.999Z"
  }'

Example request to go to next page for Run Query (Legacy):

shell

curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/query' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "startTime": "2024-11-30T00:00:00.000Z",
    "endTime": "2024-11-30T22:59:59.999Z",
    "limit": 100,
    "order": [{ "field": "_time", "desc": true }]
  }'

Cursor-based pagination

Cursor-based pagination is in public preview and may return unexpected query results. Axiom recommends timestamp-based pagination.

The parameters and mechanisms differ between the current and legacy endpoints.

Run Query

To use cursor-based pagination with the Run Query endpoint:

Include the limit operator in the APL query of your API request. The argument of this operator determines the number of events to display per page.
Use sort by _time asc or sort by _time desc in the APL query. This returns the results in ascending or descending chronological order. For more information, see sort operator.
Specify startTime and endTime in the body of your API request.

Run Query (Legacy)

To use cursor-based pagination with the legacy Run Query endpoint:

Add the limit parameter to the body of your API request. The value of this parameter determines the number of events to display per page.
Add the order parameter to the body of your API request. In the value of this parameter, order the results by time in either ascending or descending chronological order. For example, [{ "field": "_time", "desc": true }]. For more information, see order operator.
Specify startTime and endTime in the body of your API request.

Response format

object

Contains metadata about the response including pagination information.

string

Cursor for the first item in the current page.

string

Cursor for the last item in the current page.

integer

Total number of rows matching the query.

array

Contains the list of returned objects.

Page through the result set

To page through the result set, add the cursor parameter to the body of your API request.

string

Optional. A cursor for use in pagination. Use the cursor string returned in previous responses to fetch the next or previous page of results.

The minCursor and maxCursor fields in the response are boundaries that help you page through the result set.

For queries with descending order (_time desc), use minCursor from the response as the cursor in your next request to go to the next page. You reach the end when your provided cursor matches the minCursor in the response.

For queries with ascending order (_time asc), use maxCursor from the response as the cursor in your next request to go to the next page. You reach the end when your provided cursor matches the maxCursor in the response.

If the query returns fewer results than the specified limit, paging can stop.

Examples

Example request Run Query

shell

curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "apl": "DATASET_NAME | sort by _time desc | limit 100",
    "startTime": "2024-01-01T00:00:00.000Z",
    "endTime": "2024-01-31T23:59:59.999Z"
  }'

Example request Run Query (Legacy)

shell

curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/query' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "startTime": "2024-01-01T00:00:00.000Z",
    "endTime": "2024-01-31T23:59:59.999Z",
    "limit": 100,
    "order": [{ "field": "_time", "desc": true }]
  }'

Example response

JSON

{
  "status": {
    "rowsMatched": 2500,
    "minCursor": "0d3wo7v7e1oii-075a8c41710018b9-0000ecc5",
    "maxCursor": "0d3wo7v7e1oii-075a8c41710018b9-0000faa3"
  },
  "matches": [
    // ... events ...
  ]
}

Example request to page through the result set

To page through the result set, use the appropriate cursor value in your next request. For more information, see Page through the result set.

Example request to go to next page for Run Query:

shell

curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "apl": "DATASET_NAME | sort by _time desc | limit 100",
    "startTime": "2024-01-01T00:00:00.000Z",
    "endTime": "2024-01-31T23:59:59.999Z",
    "cursor": "0d3wo7v7e1oii-075a8c41710018b9-0000ecc5"
  }'

Example request to go to next page for Run Query (Legacy):

shell

curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/query' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "startTime": "2024-01-01T00:00:00.000Z",
    "endTime": "2024-01-31T23:59:59.999Z",
    "limit": 100,
    "order": [{ "field": "_time", "desc": true }],
    "cursor": "0d3wo7v7e1oii-075a8c41710018b9-0000ecc5"
  }'