Exploring Elastic Stack (Elastic Cloud, Security, Graph, and Alerting)

Elastic, the company behind Elasticsearch, also offers managed cloud hosting and X-Pack which is available as a part of their premium subscriptions. In the previous chapters, we used Kibana to create visualizations of your data. X-Pack adds more functionality, such as graph, monitoring to Kibana. The graph will let you discover relations in your data. We will discuss how to use graph to make recommendations based on the previous purchases in an e-commerce store. Along with graph and monitoring, X-Pack also provides security, extremely flexible alerts, and a notification system. We will briefly talk about the various products in Elastic Stack and Elastic Cloud in this chapter. By the end of this chapter, we will discuss the following:

• Elasticsearch Cloud (Managed Elasticsearch)
• Security
• Graph
• Alerting

...

Elastic Cloud

Elastic Cloud is the scalable cloud offering by the company that built Elasticsearch. Elasticsearch and Kibana are offered as a service. Your cluster is hosted on Amazon AWS and is completely managed by Elastic Cloud. You can create a new cluster across multiple data centers with the click of a button. Once you create a cluster, depending on the application needs, you can scale up or down very easily. The monitoring of clusters is available via Kibana monitoring. X-Pack is automatically included for every cluster running on Elastic Cloud.

Elastic offers a 14-day trial to new customers to try the service. You can sign up for a trial by going to https://cloud.elastic.co

The advantages of Elastic Cloud are as follows:

1. Monitored and managed by Elastic - If a node or the cluster goes down, its taken care by the Elastic team. There is no need of a dedicated operations...

Security

Security is also part of X-Pack gold and premium subscriptions. Security for Elasticsearch is two-fold:

• User authentication to access the cluster using username and password
• Securing the communication between the nodes and the client using SSL

Authentication and roles

X-Pack provides role-based authentication. When you install X-Pack, authentication is enabled automatically. To get started, two default users are created:

Username	Password	Role
elastic	changeme	This is a superuser
kibana	changeme	To access Kibana, the user has the required permissions to talk to Elasticsearch

You can add more users or roles in the Kibana Management Console by going to the Users page. Along with creating new users, you can...

Graph

In this section, we will discuss Graph, which is part of the X-Pack Gold and Platinum subscription. Graph lets you discover and analyze relationships in your data. It works on your existing indexes and doesn't require any special configuration. The Graph has two components:

• The functionality required for Elasticsearch to compute the Graph.
• The UI in Kibana to visualize the graphical representation.

To better explain the functionality of Graph, let's build a recommendation system for an online store. We want to know the relations between items frequently bought together and use that information to make suggestions to the users. This information can be very valuable. For example, in a physical store, items frequently bought together can be placed adjacent to each other. We can also use this information to give the user a coupon or e-mail the user about various...

Alerting

Just like Graph, alerting is a part of X-Pack Gold and Platinum subscription. Alerting was formerly known as . You can define watchers on cluster events and existing indexes. Watcher can be configured to be executed on schedule. Elasticsearch alerting is very flexible, and the input can be the response to an HTTP request or a query on existing Elasticsearch index. A watch is triggered if the input matches a predefined condition. For example, the input can be the cluster status and condition is "if the cluster status is red". When a condition is triggered, you can define what action has to be taken. An action can be sending an e-mail, indexing a document, sending a slack message, and so on.

For an e-commerce store, let's define a watcher to alert if less than five orders have been placed within the last 30 minutes. The action we will take when the condition...

Summary

In this chapter, we discussed Elastic Cloud, which is the scalable cloud offering for Elasticsearch and Kibana. We also discussed various features of X-Pack. With monitoring and alerting, you can make sure your cluster is up and running and that your data is always available.

Although Elasticsearch started as a search engine, it is evolving as an analytics engine. In this chapter, we discussed Graph, which can be used to discover relations in your data. We used the example of an e-commerce store to make recommendations, but this functionality has great potential. For example, for fraud detection or to find out the similarity between customers who like or don't like your product have in common. The graph provides actionable insights into your data.

The alert and notification system that we discussed in this chapter is very flexible. We used alerting to get notified...