Chapter 2: Enabling and Operationalization

We have just learned the basics of what Elastic ML is doing to accomplish both unsupervised automated anomaly detection and supervised data frame analysis. Now it is time to get detailed about how Elastic ML works inside the Elastic Stack (Elasticsearch and Kibana).

This chapter will focus on both the installation (really, the enablement) of Elastic ML features and a detailed discussion of the logistics of the operation, especially with respect to anomaly detection. Specifically, we will cover the following topics:

• Enabling Elastic ML features
• Understanding operationalization

Technical requirements

The information in this chapter will use the Elastic Stack as it exists in v7.10 and the workflow of the Elasticsearch Service of Elastic Cloud as of November 2020.

Enabling Elastic ML features

The process for enabling Elastic ML features inside the Elastic Stack is slightly different if you are doing so within a self-managed cluster versus using the Elasticsearch Service (ESS) of Elastic Cloud. In short, on a self-managed cluster, the features of ML are enabled via a license key (either a commercial key or a trial key). In ESS, a dedicated ML node needs to be provisioned within the cluster in order to utilize Elastic ML. In the following sections, we will explain the details of how this is accomplished in both scenarios.

Enabling ML on a self-managed cluster

If you have a self-managed cluster that was created from the downloading of Elastic's default distributions of Elasticsearch and Kibana (available at elastic.co/downloads/), enabling Elastic ML features via a license key is very simple. Be sure to not use the Apache 2.0 licensed open source distributions that do not contain the X-Pack code base.

Elastic ML, unlike the bulk...

Understanding operationalization

At some point on your journey with using Elastic ML, it will be helpful to understand a number of key concepts regarding how Elastic ML is operationalized within the Elastic Stack. This includes information about how the analytics run on the cluster nodes and how data that is to be analyzed by ML is retrieved and processed.

Note

Some concepts in this section may not be intuitive until you actually start using Elastic ML on some real examples. Don't worry if you feel like you prefer to skim (or even skip) this section now and return to it later following some genuine experience of using Elastic ML.

ML nodes

First and foremost, since Elasticsearch is, by nature, a distributed multi-node solution, it is only natural that the ML feature of the Elastic Stack works as a native plugin that obeys many of the same operational concepts. As described in the documentation (elastic.co/guide/en/elasticsearch/reference/current/ml-settings.html),...

Summary

To summarize, in this chapter, we covered the procedures around the enabling of Elastic ML's features in both a self-managed on-premises Elastic Stack and within the Elasticsearch Service of Elastic Cloud. Additionally, we looked under the hood to see the deep integration points with the rest of the Elastic Stack and how Elastic ML works from an operational perspective.

As we look ahead to future chapters, the focus will now shift away from the conceptual and background information into the realm of practical usage. Starting with the next chapter, we will jump right into the comprehensive capabilities of Elastic ML's anomaly detection and we will learn how to configure jobs to solve some practical use cases in log analytics, metric analysis, and user behavior analytics.