
Vector
axiom
sink.
Prerequisites
- Create an Axiom account.
- Create a dataset in Axiom where you send your data.
- Create an API token in Axiom with permissions to update the dataset you have created.
Installation
Follow the quickstart guide in the Vector documentation to install Vector, and to configure sources and sinks.If you use Vector version v0.41.1 (released on September 11, 2024) or earlier, use the
@timestamp
field instead of _time
to specify the timestamp of the events. For more information, see Timestamp in legacy Vector versions.If you upgrade from Vector version v0.41.1 or earlier to a newer version, update your configuration. For more information, see Upgrade from legacy Vector version.Configuration
Send data to Axiom with Vector using thefile
method and the axiom
sink.
The example below configures Vector to read and collect logs from files and send them to Axiom.
-
Create a vector configuration file
vector.toml
with the following content:ReplaceVECTOR_SOURCE_ID
with the Vector source ID. ReplacePATH_TO_LOGS
with the path to the log files. For example,/var/log/**/*.log
. ReplaceSINK_ID
with the sink ID.ReplaceAPI_TOKEN
with the Axiom API token you have generated. For added security, store the API token in an environment variable.ReplaceDATASET_NAME
with the name of the Axiom dataset where you send your data. - Run Vector to send logs to Axiom.
Example with data transformation
The example below deletes a field before sending the data to Axiom:Replace
FIELD_TO_REMOVE
with the field you want to remove.Replace API_TOKEN
with the Axiom API token you have generated. For added security, store the API token in an environment variable.Replace DATASET_NAME
with the name of the Axiom dataset where you send your data.Any changes to Vector’s
file
method can make the code example above outdated. If this happens, please refer to the official Vector documentation on the file
method, and we kindly ask you to inform us of the issue using the feedback tool at the bottom of this page.Send Kubernetes logs to Axiom
Send Kubernetes logs to Axiom using the Kubernetes source.Replace
API_TOKEN
with the Axiom API token you have generated. For added security, store the API token in an environment variable.Replace DATASET_NAME
with the name of the Axiom dataset where you send your data.Send Docker logs to Axiom
To send Docker logs using the Axiom sink, you need to create a configuration file, for example,vector.toml
, with the following content:
Replace
API_TOKEN
with the Axiom API token you have generated. For added security, store the API token in an environment variable.Replace DATASET_NAME
with the name of the Axiom dataset where you send your data.Send AWS S3 logs to Axiom
To send AWS S3 logs using the Axiom sink, create a configuration file, for example,vector.toml
, with the following content:
Replace
API_TOKEN
with the Axiom API token you have generated. For added security, store the API token in an environment variable.Replace DATASET_NAME
with the name of the Axiom dataset where you send your data.vector --config ./vector.toml
. This starts Vector and begins reading logs from the specified S3 bucket and sending them to the specified Axiom dataset.
Send Kafka logs to Axiom
To send Kafka logs using the Axiom sink, you need to create a configuration file, for example,vector.toml
, with the following code:
Replace
API_TOKEN
with the Axiom API token you have generated. For added security, store the API token in an environment variable.Replace DATASET_NAME
with the name of the Axiom dataset where you send your data.vector --config /path/to/your/vector.toml
Send NGINX metrics to Axiom
To send NGINX metrics using Vector to the Axiom sink, first enable NGINX to emit metrics, then use Vector to capture and forward those metrics. Here is a step-by-step guide:Step 1: Enable NGINX Metrics
Configure NGINX to expose metrics. This typically involves enabling thengx_http_stub_status_module
module in your NGINX configuration.
- Open your NGINX configuration file (often located at
/etc/nginx/nginx.conf
) and in yourserver
block, add:
- Restart or reload NGINX to apply the changes:
/metrics
endpoint on your server.
Step 2: Configure Vector
Configure Vector to scrape the NGINX metrics and send them to Axiom. Create a new configuration file (vector.toml
), and add the following:
Replace
API_TOKEN
with the Axiom API token you have generated. For added security, store the API token in an environment variable.Replace DATASET_NAME
with the name of the Axiom dataset where you send your data.vector --config /path/to/your/vector.toml
Send Syslog logs to Axiom
To send Syslog logs using the Axiom sink, you need to create a configuration file, for example,vector.toml
, with the following code:
Replace
API_TOKEN
with the Axiom API token you have generated. For added security, store the API token in an environment variable.Replace DATASET_NAME
with the name of the Axiom dataset where you send your data.Send Prometheus metrics to Axiom
To send Prometheus scrape metrics using the Axiom sink, you need to create a configuration file, for example,vector.toml
, with the following code:
Replace
API_TOKEN
with the Axiom API token you have generated. For added security, store the API token in an environment variable.Replace DATASET_NAME
with the name of the Axiom dataset where you send your data.Timestamp in legacy Vector versions
If you use Vector version v0.41.1 (released on September 11, 2024) or earlier, use the@timestamp
field instead of _time
to specify the timestamp in the event data you send to Axiom. For example: {"@timestamp":"2022-04-14T21:30:30.658Z..."}
. For more information, see Requirements of the timestamp field. In the case of Vector version v0.41.1 or earlier, the requirements explained on the page apply to the @timestamp
field, not to _time
.
If you use Vector version v0.42.0 (released on October 21, 2024) or newer, use the _time
field as usual for other collectors.
Upgrade from legacy Vector version
If you upgrade from Vector version v0.41.1 or earlier to a newer version, change all references from thetimestamp
field to the _time
field and remap the logic.
Example vrl
file:
example.vrl
Set compression algorithm
Upgrading to Vector version v0.42.0 or newer automatically enables thezstd
compression algorithm by default.
To set another compression algorithm, use the example below: