Home Data Monitoring Elasticsearch

Monitoring Elasticsearch

By Dan Noble , Pulkit Agrawal , Mahmoud Lababidi
books-svg-icon Book
eBook $29.99 $20.98
Print $38.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $29.99 $20.98
Print $38.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
About this book
ElasticSearch is a distributed search server similar to Apache Solr with a focus on large datasets, a schema-less setup, and high availability. This schema-free architecture allows ElasticSearch to index and search unstructured content, making it perfectly suited for both small projects and large big data warehouses with petabytes of unstructured data. This book is your toolkit to teach you how to keep your cluster in good health, and show you how to diagnose and treat unexpected issues along the way. You will start by getting introduced to ElasticSearch, and look at some common performance issues that pop up when using the system. You will then see how to install and configure ElasticSearch and the ElasticSearch monitoring plugins. Then, you will proceed to install and use the Marvel dashboard to monitor ElasticSearch. You will find out how to troubleshoot some of the common performance and reliability issues that come up when using ElasticSearch. Finally, you will analyze your cluster’s historical performance, and get to know how to get to the bottom of and recover from system failures. This book will guide you through several monitoring tools, and utilizes real-world cases and dilemmas faced when using ElasticSearch, showing you how to solve them simply, quickly, and cleanly.
Publication date:
July 2016
Publisher
Packt
Pages
180
ISBN
9781784397807

 

Chapter 1. Introduction to Monitoring Elasticsearch

Elasticsearch is a distributed and horizontally scalable full-text search engine with built-in data redundancy. It is a powerful and incredibly useful tool. However, as with any distributed system, problems may arise as it scales with more nodes and more data.

The information provided by Elasticsearch monitoring tools can drastically improve your ability to solve cluster issues and greatly increase cluster reliability and performance as a result. This chapter gives an overview of Elasticsearch and talks about why and how to monitor a cluster.

Specifically, this chapter covers the following topics:

  • An overview of Elasticsearch

  • Monitoring Elasticsearch

  • Resourcefulness and problem solving

 

An overview of Elasticsearch


This section gives a high-level overview of Elasticsearch and discusses some related full-text search products.

Learning more about Elasticsearch

Elasticsearch is a free and open source full-text search engine that is built on top of Apache Lucene. Out of the box, Elasticsearch supports horizontal scaling and data redundancy. Released in 2010, Elasticsearch quickly gained recognition in the full-text search space. Its scalability features helped the tool gain market share against similar technologies such as Apache Solr.

Elasticsearch is a persistent document store and retrieval system, and it is similar to a database. However, it is different from relational databases such as MySQL, PostgreSQL, and Oracle in many ways:

  • Distributed: Elasticsearch stores data and executes queries across multiple data nodes. This improves scalability, reliability, and performance.

  • Fault tolerant: Data is replicated across multiple nodes in an Elasticsearch cluster, so if one node goes down, data is still available.

  • Full-text search: Elasticsearch is built on top of Lucene, a full-text search technology, allowing it to understand and search natural language text.

  • JSON document store: Elasticsearch stores documents as JSON instead of as rows in a table.

  • NoSQL: Elasticsearch uses a JSON-based query language as opposed to a sequel query language (SQL).

  • Non-relational: Unlike relational databases, Elasticsearch doesn't support JOINS across tables.

  • Analytics: Elasticsearch has built-in analytical capabilities, such as word aggregations, geospatial queries, and scripting language support.

  • Dynamic Mappings: A mapping in Elasticsearch is analogous to a schema in the relational database world. If the data type for a document field isn't explicitly defined, Elasticsearch will dynamically assign a type to it.

Data distribution, redundancy, and fault tolerance

Figures 1.1 through 1.4 explain how Elasticsearch distributes data across multiple nodes and how it automatically recovers from node failures:

Figure 1.1: Elasticsearch Data Distribution

In this figure, we have an Elasticsearch cluster made up of three nodes: elasticsearch-node-01, elasticsearch-node-02, and elasticsearch-node-03. Our data index, is broken into three pieces, called shards. These shards are labeled 0, 1, and 2. Each shard is replicated once; this means that there is a redundant copy of all shards. The cluster is colored green because the cluster is in good health; all data shards and replicas are available.

Let's say that the elasticsearch-node-03 host experiences a hardware failure and shuts down. The following figures show what happens to the cluster in this scenario:

Figure 1.2: Node failure

Figure 1.2 shows elasticsearch-node-03 experiencing a failure, and the cluster entering a yellow state. This state means that there is at least one copy of each shard active in the cluster, but not all shard replicas are active. In our case, a copy of the 1 and 2 shards were on the node that failed, elasticsearch-node-03. A yellow state also warns us that if there's another hardware failure, it's possible that not all data shards will be available.

When elasticsearch-node-03 goes down, Elasticsearch will automatically start rebuilding redundant copies of the 1 and 2 shards on the remaining nodes; in our case, this is elasticsearch-node-01 and elasticsearch-node-02. This is shown in the following figure:

Figure 1.3: Cluster recovering

Once Elasticsearch finishes rebuilding the data replicas, the cluster enters a green state once again. Now, all data and shards are available to query.

Figure 1.4: Cluster recovered

The cluster recovery process demonstrated in Figures 1.3 and 1.4 happens automatically in Elasticsearch. No extra configuration or user action is required.

Full-text search

Full-text search refers to running keyword queries against natural-language text documents. A document can be something, such as a newspaper article, a blog post, a forum post, or a tweet. In fact, many popular newspapers, forums, and social media websites, such as The New York Times, Stack Overflow, and Foursquare, use Elasticsearch.

Assume that we were to store the following text string in Elasticsearch:

We demand rigidly defined areas of doubt and uncertainty!

A user can find this document by searching Elasticsearch using keywords, such as demand or doubt. Elasticsearch also supports word stemming. This means that if we searched for the word define, Elasticsearch would still find this document because the root word of defined is define.

This piece of text, along with some additional metadata, may be stored as follows in Elasticsearch in the JSON format:

{
    "text" : "We demand rigidly defined areas of doubt and uncertainty!",
    "author" : "Douglas Adams",
    "published" : "1979-10-12",
    "likes" : 583,
    "source" : "The Hitchhiker's Guide to the Galaxy",
    "tags" : ["science fiction", "satire"]
}

If we let Elasticsearch dynamically assign a mapping (think schema) to this document, it would look like this:

{
    "quote" : {
        "properties" : {
            "author" : {
                "type" : "string"
            },
            "likes" : {
                "type" : "long"
            },
            "published" : {
                "type" : "date",
                "format" : "strict_date_optional_time||epoch_millis"
            },
            "source" : {
                "type" : "string"
            },
            "tags" : {
                "type" : "string"
            },
            "text" : {
                "type" : "string"
            }
        }
    }
}

Note that Elasticsearch was able to pick up that the published field looked like a date.

An Elasticsearch query that searches for this document looks like this:

{
    "query" : {
        "query_string" : {
            "query" : "demand rigidly"
        }
    },
    "size" : 10
}

Specifics about Elasticsearch mappings and the Search API are beyond the scope of this book, but you can learn more about them through the official Elasticsearch documentation at the following links:

Note

Elasticsearch should not be your primary data store. It does not provide guarantees, such as the Atomicity, Consistency, Isolation, and Durability (ACID) of a traditional SQL data store, nor the reliability guarantees of other NoSQL databases such as HBase or Cassandra. Even though Elasticsearch has built-in data redundancy and fault tolerance, it's best practice to archive your data in a separate data store in order to re-index data into Elasticsearch if needed.

Similar technologies

This section explains a few of the many open source full-text search engines available, and discusses how they match up to Elasticsearch.

Apache Lucene

Apache Lucene (https://lucene.apache.org/core/) is an open source full-text search Java library. As mentioned earlier, Lucene is Elasticsearch's underlying search technology. Lucene also provides Elasticsearch's analytics features such as text aggregations and geospatial search. Using Apache Lucene directly is a good choice if you perform full-text search in Java on a small scale, or are building your own full-text search engine.

The benefits of using Elasticsearch over Lucene are as follows:

  • REST API instead of a Java API

  • JSON document store

  • Horizontal scalability, reliability, and fault tolerance

On the other hand, Lucene is much more lightweight and flexible to build custom applications that require full-text search integrated from the ground up.

Note

Lucene.NET is a popular .NET port of the library written in C#

Solr

Solr is another full-text search engine built on top of Apache Lucene. It has similar search, analytic, and scaling capabilities to Elasticsearch. For most applications that need a full-text search engine, choosing between Solr and Elasticsearch comes down to personal preference.

Ferret

Ferret is a full-text search engine for Ruby. It's similar to Lucene, but it is not as feature-rich. It's generally better used for Ruby applications that don't require the power (or complexity) of a search engine, such as Elasticsearch or Solr.

 

Monitoring Elasticsearch


Monitoring distributed systems is difficult because as the number of nodes, the number of users, and the amount of data increase, problems will begin to crop up.

Furthermore, it may not be immediately obvious if there is an error. Often, the cluster will keep running and try to recover from the error automatically. As shown in Figures 1.2, 1.3, and 1.4 earlier, a node failed, but Elasticsearch brought itself back to a green state without any action on our part. Unless monitored, failures such as these can go unnoticed. This can have a detrimental impact on system performance and reliability. Fewer nodes means less processing power to respond to queries, and, as in the previous example, if another node fails, our cluster won't be able to return to a green state.

The aspects of an Elasticsearch cluster that we'll want to keep track of include the following:

  • Cluster health and data availability

  • Node failures

  • Elasticsearch JVM memory usage

  • Elasticsearch cache size

  • System utilization (CPU, Memory, and Disk)

  • Query response times

  • Query rate

  • Data index times

  • Data index rate

  • Number of indices and shards

  • Index and shard size

  • System configuration

In this book, we'll go over how to understand each of these variables in context and how understanding them can help diagnose, recover from, and prevent problems in our cluster. It's certainly not possible to preemptively stop all Elasticsearch errors. However, by proactively monitoring our cluster, we'll have a good idea of when things are awry and will be better positioned to take corrective action.

In the following chapters, we'll go over everything from web-based cluster monitoring tools to Unix command line tools and log file monitoring. Some of the specific tools this book covers are as follows:

  • Elasticsearch-head

  • Bigdesk

  • Marvel

  • Kopf

  • Kibana

  • Nagios

  • Unix command-line tools

These tools will give us the information we need to effectively diagnose, solve, and prevent problems with Elasticsearch.

 

Resourcefulness and problem solving


Monitoring tools do a great job of telling you what is going on in your cluster, and they can often point out if there is a problem. However, these tools won't give you a recipe for how to actually fix a problem. Resolving issues takes critical thinking, attention to detail, and persistence. Some of the problem-solving themes this book talks about are as follows:

  • Always try to recreate the problem

  • Be on the lookout for configuration and user errors

  • Only make one configuration change at a time before testing

This book also provides some real-world case studies that help you turn the information provided by monitoring tools into insights to resolve Elasticsearch issues.

 

Summary


This chapter gave you an overview of Elasticsearch and why it's important to proactively monitor a cluster. To summarize the points from the chapter:

  • Elasticsearch is an open source scalable, fast, and fault-tolerant search engine

  • Elasticsearch is built on top of Apache Lucene, the same library that powers Apache Solr

  • Monitoring tools will help us get a better understanding of our cluster and will let us know when problems arise

  • As helpful as monitoring tools are, it's up to us to actually diagnose and fix cluster issues

In the next chapter, we'll cover how to get a simple Elasticsearch cluster running and loaded with data, and how to install several monitoring tools.

About the Authors
  • Dan Noble

    About the Author Dan is a software engineer with a passion for writing secure, clean, and articulate code. He enjoys working with a variety of programming languages and software frameworks, particularly Python, Elasticsearch, and frontend technologies. Dan currently works on geospatial web applications and data processing systems. Dan has been a user and advocate of Elasticsearch since 2011. He has given talks about Elasticsearch at various meetup groups, and is the author of the Python Elasticsearch client “rawes.” Dan was also a technical editor for the Elasticsearch Cookbook, Second Edition, by Alberto Paro (ISBN: 1783554835). Acknowledgements I would like to thank my beautiful wife, Julie, for putting up with me while I wrote this book. Thanks for supporting me every step of the way. I would also like to thank my friends and colleagues James Cubeta, Joe McMahon, and Mahmoud Lababidi, who shared their insight, time, and support. I would like to give a special thanks to Abe Usher – you have been an incredible mentor over the years. Finally, thanks to everyone at Packt Publishing for helping to make this book happen. A special thanks to Merint Mathew, Sonali Vernekar, Husain Kanchwala, and Amey Varangaonkar for your valuable and careful feedback.

    Browse publications by this author
  • Pulkit Agrawal
  • Mahmoud Lababidi
Latest Reviews (2 reviews total)
Easy and Safety. I like to pay by paypal.
Monitoring Elasticsearch
Unlock this book and the full library FREE for 7 days
Start now