Monitoring and Notifiers

Setup Monitors and Notifiers to be alerted when queries match certain conditions.

This section introduces Threshold Monitors and Notifiers, and explains how you can use them to generate automated alerts from your event data.

Threshold Monitors

A Threshold Monitor is a background task that periodically runs a query, and compares the values returned by this query against a user-defined threshold rule. When the monitor results match the threshold rule, the monitor enters an alert state. The user can configure the monitor to notify external parties when the monitor enters and exits the alert state.

monitors page

Threshold Monitors can be created using the Simple Query Builder or Advanced Query Language.

monitors page

Threshold Monitor Options

In addition to query defining the monitor, the following options are available for controlling monitor behaviour:

Threshold value - The value to compare the results of the query against. This can be any numeric value.
Comparison operator: The rule to apply when comparing the results against the threshold value. Possible values are above, above or equal, below, and below or equal.
Frequency - How often the monitor should run. This is given as a positive integer number of minutes.
Range - How far to look back when running the query. This is given as a positive integer number of minutes. Each run of the monitor will apply a time range to the query of length equal to the Range, and with an end time given by the time the monitor is run.
Your query may return 'no data' if no events matched your filters, and if an aggregation used in the query is undefined in this situation. For example, attempting to take the average of a field not present in any matching events. To trigger the monitor when your query returns no data, you can enable Alert on no data.
You can group by attributes when defining your query. By default, your monitor will enter the alert state if any of the values returned for the grouped by attributes match the threshold rule, and will remain in alert state until none of the values returned match the threshold rule. To instead have the monitor trigger separately for each group that matches the threshold rule, you can enable Notify by group. Note that at most one trigger notification will be generated per monitor run. This option only has an effect if the monitor's query groups by a non-time field.
Notifiers - Connect Notifiers to receive alerts when the monitor changes state (See below).

Notifiers

Notifiers are how monitors alert external parties when they are triggered.

Axiom supports a range of different notifiers as documented below.

Email

Create an email notifier to notify a list of email addresses. An email will be sent to each email address on the list whenever an associated monitor enters or exits alert state.

Email notifier

Slack

Create Slack notifiers to notify specific channels in your Slack organization. This notifier requires setting up an Incoming Webhook in Slack.

To configure a new “Incoming Webhook” in Slack go to Apps & integrations > Manage > Custom integrations > Incoming WebHooks and follow the instructions provided.

slack notifier

PagerDuty

Create a PagerDuty notifier to use all the incident management features of PagerDuty with Axiom. Messages will be sent by Axiom when a monitor enters or exits alert state, and you can configure schedules and alerting rules inside PagerDuty.

To configure a new “Service” in PagerDuty using the Events V2 API, go to Configuration > Services > Add New Service and create a new Service named ‘Axiom’ with all the default settings. Take note of the Integration Key provided and enter this when creating a PagerDuty notifier in Axiom.

You will also need to provide an API Access Key with the right permissions which can be generated at pagerduty.com/api_keys.

Pagerduty notifier

Webhook

Create Webhook notifiers to connect to internal or external services using your own handlers.

The URL will receive a HTTP(s) POST with a content type of application/json and a payload as shown below:

{
    "action":"close",
    "sender":"monitors"
    "event":{
        "id":"f11f8121-c949-4b59-84ba-40ef868f4d54",
        "name":"Queue backlogging",
        "title":"Current value is above threshold value 2500",
        "body":"Triggered with a value of 2782",
        "value":"2782",
        "timestamp":"2021-02-23T14:43:45.34205696Z",
        "source":"monitors.qKKbK6n4xeokNBF9GC.COUNT",
        "priority":0,
        "snoozedUntil":"0001-01-01T00:00:00Z",
        "state":3
    },
}

webhook notifier

Opsgenie

Create an Opsgenie notifier to use all the incident management features of OpsGenie with Axiom. Messages will be sent by Axiom when a monitor enters or exits alert state, and you can configure schedules and alerting rules inside Opsgenie.

To configure a new integration in Opsgenie, go to Teams > Integrations > Add Integration > API Integration.

Opsgenie notifier

Discord

Create a Discord notifier to notify specific channels in your Discord server. You need to provide Axiom with a Discord token and channel identifier to create a Discord notifier.

To get a token, go to discord.com/developers/applications and create a new application.
Click Bot > Add Bot > Reset Token to get your Discord token.
Go to OAuth2 > URL Generator, check the Bot scope and the Send Messages permission, then copy and open the generated URL to add the bot to your server.
To get the channel ID, go to User Settings > Advanced and enable developer mode. Then right-click a channel and click Copy ID.
Confirm if the check box of the Discord Bot has the proper allow channel access permissions from your settings.

Discord Notifier

Discord Webhook

Create a Discord Webhook notifier to notify specific channels in your Discord server via web hook.

To get your webhook URL, go to Server Settings > Integrations > New Webhook > Copy Webhook URL.