> ## Documentation Index
> Fetch the complete documentation index at: https://axiom.co/docs/llms.txt
> Use this file to discover all available pages before exploring further.

<AgentInstructions>

## Submitting Feedback

If you encounter incorrect, outdated, or confusing documentation on this page, submit feedback:

POST https://axiom.co/docs/feedback

```json
{
  "path": "/apl/scalar-functions/genai-functions/genai-estimate-tokens",
  "feedback": "Description of the issue"
}
```

Only submit feedback when you have something specific and actionable to report.

</AgentInstructions>

# genai_estimate_tokens

> This page explains how to use the genai_estimate_tokens function in APL.

The `genai_estimate_tokens` function estimates the number of tokens in a text string. This estimation helps you predict API costs, validate input sizes, and monitor token usage before making actual API calls to LLM services.

You can use this function to validate prompt sizes, estimate costs before API calls, monitor content length, or analyze token efficiency across different prompts.

## For users of other query languages

If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.

<AccordionGroup>
  <Accordion title="Splunk SPL users">
    In Splunk SPL, there’s no direct equivalent for token estimation. You would typically use character or word count as a rough approximation.

    <CodeGroup>
      ```sql Splunk example theme={null}
      | eval estimated_tokens=len(text)/4
      ```

      ```kusto APL equivalent theme={null}
      ['ai-logs']
      | extend estimated_tokens = genai_estimate_tokens(text)
      ```
    </CodeGroup>
  </Accordion>

  <Accordion title="ANSI SQL users">
    In ANSI SQL, you would need to use character-based estimations, which are less accurate than proper token counting.

    <CodeGroup>
      ```sql SQL example theme={null}
      SELECT 
        text,
        LENGTH(text) / 4 as estimated_tokens
      FROM prompts
      ```

      ```kusto APL equivalent theme={null}
      ['ai-logs']
      | extend estimated_tokens = genai_estimate_tokens(text)
      ```
    </CodeGroup>
  </Accordion>
</AccordionGroup>

## Usage

### Syntax

```kusto theme={null}
genai_estimate_tokens(text)
```

### Parameters

| Name | Type   | Required | Description                                                     |
| ---- | ------ | -------- | --------------------------------------------------------------- |
| text | string | Yes      | The text string for which you want to estimate the token count. |

### Returns

Returns a long integer representing the estimated number of tokens in the input text.

## Example

Estimate the number of tokens in a GenAI conversation prompt.

**Query**

```kusto theme={null}
['otel-demo-genai']
| extend user_prompt = genai_extract_user_prompt(['attributes.gen_ai.input.messages'])
| extend estimated_tokens = genai_estimate_tokens(user_prompt)
| summarize avg_tokens = avg(estimated_tokens), max_tokens = max(estimated_tokens)
```

[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B%27otel-demo-genai%27%5D%20%7C%20extend%20user_prompt%20%3D%20genai_extract_user_prompt\(%5B%27attributes.gen_ai.input.messages%27%5D\)%20%7C%20extend%20estimated_tokens%20%3D%20genai_estimate_tokens\(user_prompt\)%20%7C%20summarize%20avg_tokens%20%3D%20avg\(estimated_tokens\)%2C%20max_tokens%20%3D%20max\(estimated_tokens\)%22%7D)

**Output**

| avg\_tokens | max\_tokens |
| ----------- | ----------- |
| 245         | 1024        |

This query analyzes prompt token usage patterns, helping you predict costs and validate input sizes.

## List of related functions

* [genai\_cost](/apl/scalar-functions/genai-functions/genai-cost): Calculates the actual cost based on token usage. Use this in combination with token estimates to predict costs.
* [strlen](/apl/scalar-functions/string-functions#strlen): Returns string length in characters. Use this for a simpler character count without token estimation.
* [string\_size](/apl/scalar-functions/string-functions/string-size): Returns string length in characters. Use this when you need character count instead of token count.
* [genai\_input\_cost](/apl/scalar-functions/genai-functions/genai-input-cost): Calculates input token cost. Combine with token estimation to predict prompt costs.