Using alerting to monitor data events

Alerting is one of the most used X-Pack components because it allows us to fire alert events on data that is processed in the cluster.

The main concepts behind Elasticsearch alerting are as follows:

• Conditions: These define what needs to be detected.
• Schedule: These define the frequency of how the checks run.
• Actions: These define how to respond to an alert.

Elasticsearch is able to cover the following:

• Infrastructural alerting such as issues about load on the server, disk space, and node being down
• ETL flow alerting such as the reduction of ingested records in some indices
• Business alerting with rules defined by a business user on data quality or features on their data
• Predictive alerting using the Machine Learning (ML) X-Pack component, which is able to detect an anomaly in ingested data

Getting ready

Alerting only works on a full setup environment with security enabled; we will use the...