Monitoring Elasticsearch
Monitoring distributed systems is difficult because as the number of nodes, the number of users, and the amount of data increase, problems will begin to crop up.
Furthermore, it may not be immediately obvious if there is an error. Often, the cluster will keep running and try to recover from the error automatically. As shown in Figures 1.2, 1.3, and 1.4 earlier, a node failed, but Elasticsearch brought itself back to a green state without any action on our part. Unless monitored, failures such as these can go unnoticed. This can have a detrimental impact on system performance and reliability. Fewer nodes means less processing power to respond to queries, and, as in the previous example, if another node fails, our cluster won't be able to return to a green state.
The aspects of an Elasticsearch cluster that we'll want to keep track of include the following:
Cluster health and data availability
Node failures
Elasticsearch JVM memory usage
Elasticsearch cache size
System utilization (CPU, Memory, and Disk)
Query response times
Query rate
Data index times
Data index rate
Number of indices and shards
Index and shard size
System configuration
In this book, we'll go over how to understand each of these variables in context and how understanding them can help diagnose, recover from, and prevent problems in our cluster. It's certainly not possible to preemptively stop all Elasticsearch errors. However, by proactively monitoring our cluster, we'll have a good idea of when things are awry and will be better positioned to take corrective action.
In the following chapters, we'll go over everything from web-based cluster monitoring tools to Unix command line tools and log file monitoring. Some of the specific tools this book covers are as follows:
Elasticsearch-head
Bigdesk
Marvel
Kopf
Kibana
Nagios
Unix command-line tools
These tools will give us the information we need to effectively diagnose, solve, and prevent problems with Elasticsearch.
