Chapter 12. User Interfaces

In this chapter we will cover the following recipes:

• Installing and using Cerebro

• Installing Kibana and X-Pack

• Managing Kibana dashboards

• Monitoring with Kibana

• Using Kibana console

• Visualizing data with Kibana

• Installing Kibana plugins

• Generating Graph with Kibana

Introduction

In Elasticsearch ecosystem, it can be immensely useful to monitor nodes and cluster to manage and improve their performance and state. There are several issues that can arise at cluster level, such as:

• Node overheads, where some nodes can have too many shards allocated and can become a bottleneck for the entire cluster

• Node shutdown can happen due to many reasons, for example, full disks, hardware failures, and power problems

• Shard relocation problems or corruptions, in which some shards are unable to get an online status

• Too large shards happens when a shard is too big; the index performance decreases due to Lucene massive segments merging

• Empty indices and shards waste memory and resources, but because every shard has a lot of active threads if there is a huge number of unused indices and shards, the general cluster performance is degraded

• Node problems such as high CPU usage or disk full

Detecting malfunction or bad performances can be done via API or via some frontends that are...

Installing and using Cerebro

Cerebro is the evolution of the previous Elasticsearch plugin Elasticsearch kopf (https://github.com/lmenezes/elasticsearch-kopf) that doesn't work in Elasticsearch 5.x or above due to removing of site plugins.

Cerebro is a partial rewrite of the previous plugin available as a self-working application server.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

Java JVM version 8.x or above must be installed to run Cerebro.

How to do it...

For installing Cerebro, you need to download it and manually install its plugin. We will now perform the following steps:

1. You can download a binary distro of Cerebro at https://github.com/lmenezes/cerebro/releases. For Linux/MacOSX, we can use the following command:

           wget 
           https://github.com/lmenezes/cerebro/releases/download
           /v0.4.2/cerebro-0.4.2.tgz
   

2. Now you can extract it...

Installing Kibana and X-Pack

The most famous Elasticsearch interface is Kibana, which from Elasticsearch 5.x version has the same version of Elasticsearch. Kibana is an opensource pluggable interface, free to change to be used for Elasticsearch. It provides data visualization and data discovery and with commercial products such as X-Pack, and also supports security, graph, and cluster monitoring.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

How to do it...

For installing Kibana, we will perform the following steps:

1. Download a binary version for Elasticsearch website and unpack it. For Linux, the commands are as follows:

           wget https://artifacts.elastic.co/downloads/kibana/kibana-
           5.1.1-linux-x86_64.tar.gz
           tar -xzf kibana-5.1.1-linux-x86_64.tar.gz
   

2. On MacOsX, you can install Kibana via the following command:

           brew install kibana...

Managing Kibana dashboards

The core of Kibana are the dashboards--an aggregation of widgets that are results of queries and aggregations.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

You also need a functional Kibana installation as described in the Installing Kibana and X-Pack recipe.

How to do it...

For managing Kibana dashboards, we will perform the following steps:

1. We access the Discovery section of Kibana as shown in the following screenshot:

   

2. After a few seconds, the default search frontend should appear. The default query is *, which is executed against the _all field.

How it works...

The Kibana interface is divided into sections: Discovery, Visualize, Dashboards, Dev Tools, and Management are available in the opensource version of Kibana. The X-Pack adds Graph and Monitoring ones.

The dashboard top menu allows us to do as follows:

• Create a new dashboard starting...

Monitoring with Kibana

X-Pack provides cluster functionalities that allows to control and monitor your nodes and cluster. This is a very useful component of X-Pack as it is the lifesaver on large installations.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

You also need a functional Kibana installation as described in the Installing Kibana and X-Pack recipe.

Note

Monitor plugin (X-Pack) must be installed in every Elasticsearch node of the cluster, to be able to correctly collect the data metrics.

How to do it...

To use X-Pack Elasticsearch monitor, we will perform the following steps:

1. We access the Monitoring section of Kibana as shown in the following screenshot:

   

2. In the main page, you can access the different sections on monitoring applications as shown in the following screenshot:

   

3. If you click on the Overview cluster, you are able to see global cluster status and statistics...

Using Kibana dev-console

X-Pack provides cluster functionalities that allow you to control and monitor your nodes and cluster. This is a very useful component of X-Pack as it is the lifesaver on large installations.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

You also need a functional Kibana installation as described in the Installing Kibana and X-Pack recipe. The X-Pack is not needed.

How to do it...

To use X-Pack Elasticsearch monitor, we will perform the following steps:

1. We access the Dev Tools section of Kibana as shown in the following screenshot:

   

2. Now we can use the dev console to create/execute/test queries and other Elasticsearch HTTP API via it as shown in the following screenshot:

   

How it works...

The Kibana console is very similar to the Cerebro interface that we have previously seen.

It allows us to execute every kind of REST API call via the http interface...

Visualizing data with Kibana

Kibana allows you to create reusable data representations called Visualizations. They are representations of aggregations and can be used to power up the dashboard with custom graphs.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

You also need a functional Kibana installation as described in Installing Kibana and X-Pack recipe. The X-Pack is not needed.

How to do it...

To use Kibana to create custom widgets, we will perform the following steps:

1. We access the Visualize section of Kibana as shown in the following screenshot:

   

2. Now we can choose the visualization that we want to create as shown in the following screenshot:

   

3. If we want to create a Tag cloud visualization, we select it and populate the required fields as shown in the following screenshot:

   

How it works...

Aggregations and searches can be grouped in the visualization widgets that can...

Installing Kibana plugins

As Elasticsearch, Kibana allows to be extended with plugins that can be installed to increment its functionalities.

In this recipe, we will install 3D pie chart, bars chart, and bubbles chart visualizations available at https://github.com/aparo/3D_kibana_charts_vis.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

You also need a functional Kibana installation as described in the Installing Kibana and X-Pack recipe. The X-Pack is not needed.

As the plugins that we are installing is very new, they are not packaged as binary zip, we need to install via source code using git and npm of Node.js.

How to do it...

To install a source Kibana plugin, we will perform the following steps:

1. We need to be in a shell in the kibana/plugins directory.

2. We fetch the source from github via following command:

           git clone https://github.com/aparo/3D_kibana_charts_vis...

Generating graph with Kibana

X-Pack for Elasticsearch and Kibana 5.x or above provides a graph API to discover relations in your data. The Elasticsearch graph is built in real-time, extracting relations from the indexed data.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

You also need a functional Kibana installation as described in the Installing Kibana and X-Pack recipe.

How to do it...

To create a custom graph view, we will perform the following steps:

1. We access the Graph section of Kibana as shown in the following screenshot:

   

2. A graph works on an index pattern, so we need to select the one that is available to start creating our graph as shown in the following screenshot:

   

3. In an Elasticsearch graph, values in a field are our vertices: we need to select the field that contains our vertices. The field list is extracted from the index pattern as shown in the following...