perspectives / ecosystem

Is Splunk a data lake?

The index is Splunk's operational store. The Machine Data Lake is a separate raw retention product.

Axiom · · 8 min read

Splunk is not one storage model. The core product is an index-centric search platform. In 2026 Splunk added a Machine Data Lake product that sits beside that index as a raw retention tier. An accurate answer to "is Splunk a data lake" depends on which of those you mean.

If "Splunk is a data lake" becomes shorthand for the whole platform, you start expecting lake economics (cheap, raw, schema-on-read) everywhere, including inside the index. The sections below define a data lake as a storage pattern, explain what Splunk's index does, what the Machine Data Lake product adds, how SmartStore differs from both, and how Axiom's event path keeps retained data in one query path.

What a data lake is

A "data lake" is a storage pattern. It is a repository that holds raw data in its native format, usually in object storage, with schema applied at read time rather than at write time. It is optimized for holding everything cheaply, and on its own it gives you no query engine built for operational speed.

That is a useful tradeoff for data whose shape is still unknown. It is a poor fit for the dashboard someone refreshes every five minutes, the alert that has to fire within seconds of a threshold breach, or the investigation where you are filtering, grouping, and re-filtering against billions of events in real time. Those workloads need an index, a cache, or a purpose-built query engine sitting in front of or on top of the raw data.

Splunk's index is an operational store

Splunk's core product has never been a lake by that definition. It is a search and analytics platform, and its operational store is an index: Splunk parses and indexes events at ingest, including core metadata (host, source, sourcetype, timestamp). Most richer fields are extracted at search time. That index is what makes Search Processing Language (SPL) fast against recent data, and it is what powers Splunk's dashboards, scheduled searches, alerts, and correlation rules.

The index is also where most of Splunk's value lives for existing customers. SPL fluency, years of saved searches, familiar dashboards, and trained judgment about what a spike in a particular field means sit between an event and a decision. That operating layer is a large part of existing-customer value and switching cost. Splunk pioneered machine data in 2003 and built two decades of that workflow on top of an indexed architecture, even as the company has invested heavily in security.

Because ingest builds a paid searchable index, you cannot treat the index as a lake. Raw files in object storage, with nothing built to query them at operational speed, will not give you the same real-time alerting or repeated high-performance search.

What the Machine Data Lake product adds

Splunk's Machine Data Lake is a newer, separate product: a raw retention tier that lands data in managed object storage rather than the index. It lands supported logs and events, not traces. A Catalog helps you discover what has landed there. Raw search lets you look at that raw layer on source, sourcetype, host, and time without first indexing it. A dual-routing path can keep matching events flowing into an existing Splunk index while they also land in the lake.

To get the parsed fields, repeated search, dashboards, alerts, scheduled searches, or correlation Splunk customers already rely on, you promote data out of the lake. Promotion writes a new index or analytics table, but you cannot promote into an existing index, so dashboards and saved searches built on current indexes have to be repointed before they see promoted data. Static promotion writes a defined historical range to a Splunk index or an analytics table. Streaming promotion writes matching new events to a Splunk index only; it does not backfill history and cannot target an analytics table. The raw copy can remain in the lake after a promoted copy exists elsewhere, each with its own representation and retention policy. Where Splunk enables Open Sharing, selected lake data can be read by an authorized external tool without promotion.

That gives Splunk customers a cheaper place to keep data they are not ready to commit to the index. It also means an ongoing decision: which data becomes active, when it becomes active, and for which future question. You budget and operate the lake as a retention tier and the index as the search, dashboard, and alerting layer. How Axiom and Splunk have diverged starts from that split.

Where the confusion comes from

Part of the mix-up is an older Splunk mechanism: SmartStore. SmartStore moves warm indexed buckets to remote object storage to reduce local disk footprint, but a bucket that has not been cached locally still has to download before it can be searched. SmartStore is still about the index, with cheaper storage underneath it. The Machine Data Lake is a separate raw tier that sits outside the index until something promotes data into it.

Treat SmartStore, the index, and the Machine Data Lake as three separate layers. SmartStore is cheaper storage for data that is already indexed. The lake is a raw copy that is not in the index unless you dual-routed it at landing or later promoted it. Which layer a workload sits in determines whether you can search it at operational speed without a promotion job.

How Axiom's event path differs

Axiom does not require you to pick a subset of fields or sources for a later promotion job. Recent events are queryable through a live columnar path as soon as they land, and that same data then settles into highly compressed, field-aware blocks on object storage that remain in the ordinary query path. There is no separate raw lake tier and no second promotion step to schedule, monitor, or budget for. A retained event stays in one dataset, under one retention policy, and in one query path, whether it arrived five minutes ago or five years ago. Axiom keeps data in object storage the way a lake does, and queries it directly the way an index does, so there is no promotion step between the two.

You can land beside Splunk on sources that would otherwise sit un-promoted, then move more sources as the comparison proves out, sending data to Axiom over a Splunk HEC-compatible ingest endpoint without changing existing forwarders or collectors. From there you choose the Axiom for Splunk app and the Axiom Portal for Splunk depending on how much of the existing SPL workflow needs to stay untouched. The Portal pushes filters, projections, and aggregations down into Axiom, so a stats count over a billion events runs where the data lives and returns a small result set to the Splunk search head, computed exactly rather than sampled.

Axiom handles logs, traces, metrics, and events on one platform, with EventDB for logs, traces, and events, and MetricsDB for metrics. How long you keep a source is also how long you can query it at full fidelity, because retained events stay in the query path. That is the same reason organizations decide they should be able to see and query 100 percent of their event data rather than sampling it down to fit a budget.

What to check on a given workload

For any source, ask which layer it sits in today: the index, SmartStore (still the index, with remote buckets), the Machine Data Lake, or both index and lake if dual-routing is on. If it is in the lake only, dashboards, alerts, and correlation wait on a promotion job into a Splunk index. Open Sharing gives authorized external tools read access for BI, notebooks, or ML. It is not a Splunk dashboard or alert path. Axiom keeps raw and operational data on one path, so a retained event is queryable without that extra step. Machine Data Lake pricing is covered in a separate article.

FAQs

Related Reading

Try coexistence on a live source

Start free and send a live source over a Splunk HEC-compatible endpoint or standard OTLP. You’ll see ingest, query, and retention on one path, with no promotion step. To estimate cost at your own volume, use the calculator on our pricing page. If you’d rather walk through it with our team, see Axiom on your data.