Discovering the Data through Discover

So, you’ve got your data in. Now it’s time to answer questions. What countries in the world have the maximum number of people working in IT? What events have the runtime error 404? Which customers are making the highest contribution to the profit?

All of these questions (and more!) can be discovered right on this interface. It is finally time to start exploring our dataset through the very interesting interface in Kibana, called Discover. Right when you start collecting and ingesting your data from your data sources, Discover in Kibana will be one place you would like to go and kickstart your exploration of the dataset. From which fields are coming in to how your data is structured with respect to the datatypes of the fields, everything can be visible in Discover. It also has an amazing query feature, which serves as your own custom search functionality placed over whatever dataset you have in the Elasticsearch cluster.

In...

Exploring your data

Let us now start looking into how the data gets stored in the Elasticsearch cluster, which takes us to the concept of a document. Anything that we ingest in the cluster gets stored in the cluster as a document.

Elasticsearch – a document store

Before starting to understand how exploration of data can be done, Elasticsearch is called a distributed document store as it stores the data in the form of serialized JSON documents:

Figure 3.1 – An index with a collection of documents can be stored in Elasticsearch

These JSON documents are distributed across all the nodes of the cluster. If we go into where this document is stored in an index, it would be a logical namespace called an Index. It can be thought of as a collection of JSON documents that has data stored in the form of key-value pairs that contain the data. See the following example:

Figure 3.2 – A sample of a record of data

...

Working with queries

Imagine a world where a famous e-commerce website that you wish to shop from doesn’t have a search bar! It is next to inevitable to have a feature to search, which in turn allows a user to type and do a quick look-up on the data. The constant need to search is increasing now, more than ever. In fact, every website, page, or frontend requires a search bar that facilitates the way a user can search the data:

Figure 3.6 – The Search bar

Let’s start understanding the query language that is provided in the search bar of Discover.

Structuring KQL query (example)

Kibana Query Language, abbreviated as KQL, is a text-based query language that is used to filter data in Kibana. It works on a simple syntax of just searching for the terms we’re looking for in the search bar, or being more specific on what we’re searching for by providing a field against the word, shown as follows:

"field": ...

Creating a saved search

A saved search is a search created by saving the search results of the query done by KQL or a filter on Discover. It can be further used to add in as a saved search on a dashboard to display a specific set of documents representing a specific use case/query.

A saved search can also be a great way to reuse the search made on Discover. Furthermore, the results in the form of documents can be added to the dashboard along with the other visualizations. Let us see how we can create a saved search, step by step.

Steps to create a saved search

Creating a saved search is a fairly simple process where you have a specific use case in mind and need to host your search in that exact direction. Let’s see how to begin with creating a saved search:

1. Go to the main menu of Kibana and open Discover:

Figure 3.14 – The navigation of the main menu and Discover

2. Select the data view according to the dataset you...

Summary

In this chapter, we studied how to explore different search features to create a query language syntax that can filter our data and show us the results in Discover. We can play around with this UI to leverage the features as needed.

Next up, we shall explore how we can draw distinct types of visualizations in Kibana based on the data in an Elasticsearch cluster.