User Interfaces

In an Elasticsearch ecosystem, it can be immensely useful to monitor nodes and clusters in order to manage and improve their performance and state. There are several issues that can arise at the cluster level, such as the following:

• There can be node overheads; for instance, where some nodes can have too many shards allocated and can become a bottleneck for the entire cluster
• Node shutdown can occur due to many reasons, such as, full disks, hardware failures, and power problems
• Shard relocation problems or corruptions, in which some shards are unable to be initialized and go online due to some issues.
• Having very large shards can also be an issue; index performance can decrease due to large Lucene segments merging
• Empty indices and shards waste memory and resources; however, because every shard has a lot of active threads, if there is a huge number of unused indices...

Installing and using Cerebro

Cerebro is the evolution of the previous Elasticsearch plugin, Elasticsearch Kopf (https://github.com/lmenezes/elasticsearch-kopf) – this doesn't work in Elasticsearch version 5.x or later versions due to the removal of site plugins.

Cerebro is one of the most useful interfaces for looking at shard allocations and executing common index operations through a graphic interface. It's completely open source, and it allows you to add a user, password, or LDAP authentication for accessing the web interface.

Cerebro is a partial rewrite of the previous plugin, and it is available as a self-working application server, based on Scala's Play Framework.

Getting ready

You will need an up-and-running...

Installing and using Elasticsearch HQ

ElasticSearch HQ (http://www.elastichq.org) is a monitoring and management application, which manages both instances and clusters. It's an open source solution, and it is free for both private and commercial use. 

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started. 

Python version 3.4, or a later version, must be installed in order to run ElasticSearch HQ.

If you are installing using Docker Compose, which is available in the ch01 directory, then you don't need to manually install it. 

...

Installing Kibana

The most famous Elasticsearch interface is Kibana; from Elasticsearch version 7.x, they share the same version number. Kibana is an open source pluggable interface and is free to use with Elasticsearch. It provides data visualization and it can be extended with a commercial product called X-Pack that provides security, graph capabilities, and cluster monitoring.

In this chapter, we will mainly cover the Kibana open source components. Kibana with X-Pack offers a lot of functionalities and as these are beyond the scope of this book, I suggest that you look for books related to Kibana for a full description of all Kibana's capabilities.

Getting ready

You will need an up-and-running Elasticsearch installation –...

Managing Kibana discovery

One of the most popular aspects of Kibana is the discovery dashboard. This is because it allows you to dynamically navigate your data. With the evolution of Kibana, a lot of new features have been added to the Discovery dashboard in order to allow you to easily filter and analyze your data.

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started. Additionally, a working Kibana instance is required, as described in the Installing Kibana recipe of this chapter.

If you have used Docker Compose, which is available in the ch01 directory...

Visualizing data with Kibana

Kibana allows you to create reusable data representations called visualizations. These are representations of aggregations and can be used to power up the dashboard using custom graphs. In general, you can consider visualization as a building block for your dashboard.

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started. Additionally, a working Kibana instance is required, as described in the Installing Kibana recipe of this chapter.

If you have used Docker Compose, which is available in the ch01 directory, then everything...

Using Kibana Dev tools

Kibana provides a very handy section for developers: Dev Tools. In the open source version, this section is composed of three tools:

• Dev-Console: The place where the developer tests and execute commands
• Search Profiler: A tool that is used to profile queries
• Grok Debugger: This is useful for debugging Grok regular expressions

Getting ready

You will need an up-and-running Elasticsearch installation—similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started. Additionally, a working Kibana instance is required, as described in the Installing Kibana recipe of this chapter.

If you have used Docker Compose, which is available...