Search icon Close icon
Search icon
Subscription
0
Cart icon
Close icon
You have no products in your basket yet
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Monitoring Elasticsearch

You're reading from  Monitoring Elasticsearch

Product type Book
Published in Jul 2016
Publisher Packt
ISBN-13 9781784397807
Pages 180 pages
Edition 1st Edition
Languages
Concepts
Authors (3):
Dan Noble Dan Noble
Profile icon Dan Noble
LinkedIn
About the Author Dan is a software engineer with a passion for writing secure, clean, and articulate code. He enjoys working with a variety of programming languages and software frameworks, particularly Python, Elasticsearch, and frontend technologies. Dan currently works on geospatial web applications and data processing systems. Dan has been a user and advocate of Elasticsearch since 2011. He has given talks about Elasticsearch at various meetup groups, and is the author of the Python Elasticsearch client “rawes.” Dan was also a technical editor for the Elasticsearch Cookbook, Second Edition, by Alberto Paro (ISBN: 1783554835). Acknowledgements I would like to thank my beautiful wife, Julie, for putting up with me while I wrote this book. Thanks for supporting me every step of the way. I would also like to thank my friends and colleagues James Cubeta, Joe McMahon, and Mahmoud Lababidi, who shared their insight, time, and support. I would like to give a special thanks to Abe Usher – you have been an incredible mentor over the years. Finally, thanks to everyone at Packt Publishing for helping to make this book happen. A special thanks to Merint Mathew, Sonali Vernekar, Husain Kanchwala, and Amey Varangaonkar for your valuable and careful feedback.
See other products by Dan Noble
Pulkit Agrawal Pulkit Agrawal
Profile icon Pulkit Agrawal
LinkedIn
See other products by Pulkit Agrawal
Mahmoud Lababidi Mahmoud Lababidi
Profile icon Mahmoud Lababidi
LinkedIn
See other products by Mahmoud Lababidi
View More author details

Table of Contents (15) Chapters

Monitoring Elasticsearch
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
1. Introduction to Monitoring Elasticsearch 2. Installation and the Requirements for Elasticsearch 3. Elasticsearch-head and Bigdesk 4. Marvel Dashboard 5. System Monitoring 6. Troubleshooting Performance and Reliability Issues 7. Node Failure and Post-Mortem Analysis 8. Looking Forward Index

Index

A

  • Apache Lucene
    • reference link / Apache Lucene
  • Atomicity, Consistency, Isolation, and Durability (ACID) / Full-text search

B

  • Bigdesk
    • installing / Installing Bigdesk
    • about / Bigdesk
    • reference / Bigdesk
  • bulk indexing operations
    • about / Bulk indexing operations

C

  • case studies, node failures
    • ES process, terminating / The ES process quits unexpectedly
    • slow queries requests / Query requests slow and timing out
    • queries, timing out / Query requests slow and timing out
  • cluster
    • about / Understanding your cluster
    • Elasticsearch-head, installing / Installing Elasticsearch-head
    • Bigdesk, installing / Installing Bigdesk
    • Marvel / Marvel
    • Marvel Dashboard, installing / Marvel
  • Cluster API
    • reference / Background
  • cluster management tools, more dropdown
    • Create Index / The more dropdown
    • Cluster Settings / The more dropdown
    • Aliases / The more dropdown
    • Analysis / The more dropdown
    • Percolator / The more dropdown
    • Warmers / The more dropdown
    • Snapshot / The more dropdown
    • Index Templates / The more dropdown
    • Cat APIs / The more dropdown
    • Hot Threads / The more dropdown
  • cold indices
    • about / Cold indices
  • command line tools, for system and process management
    • about / Command line tools for system and process management
    • top command / top
    • tail command / tail
    • grep command / grep
    • ps command / ps
    • kill command / kill
    • free command / free
    • du command / du and df
    • df command / du and df

D

  • data ingestion
    • optimizing / Optimizing data ingestion
    • bulk indexing operations / Bulk indexing operations
    • drive configuration / Drive configuration
  • disk size
    • reducing / Reducing disk size
    • compression level, increasing / Compression
    • _source field, storing / Storing the _source and analyzed fields
    • analyzed field, storing / Storing the _source and analyzed fields
  • disk space
    • about / Disk space
    • issue, resolving / Resolving the issue
  • drive configuration
    • about / Drive configuration

E

  • eager field data
    • URL / Cold indices
  • Elasticsearch
    • overview / An overview of Elasticsearch
    • about / Learning more about Elasticsearch
    • data distribution / Data distribution, redundancy, and fault tolerance
    • redundancy / Data distribution, redundancy, and fault tolerance
    • fault tolerance / Data distribution, redundancy, and fault tolerance
    • full-text search / Full-text search
    • open source full-text search engines / Similar technologies
    • monitoring / Monitoring Elasticsearch
    • installing / Installing Elasticsearch
    • reference link / Installing Elasticsearch
    • version, reference link / Installing Elasticsearch
    • process crashes / Elasticsearch process crashes
  • Elasticsearch, Logstash, and Kibana (ELK)
    • about / ELK
    • uses / ELK
  • Elasticsearch-head
    • installing / Installing Elasticsearch-head
    • about / Elasticsearch-head
    • Overview tab / The Overview tab
    • cluster states / Cluster states
    • node and index actions / Node and index actions
    • Indices tab / The Indices tab
    • Browser tab / The Browser tab
    • Structured Query tab / The Structured Query tab
    • Any Request tab / The Any Request tab
    • reference, for official website / The official website
  • elasticsearch.yml configuration options
    • marvel.enabled / Marvel agent configuration settings
    • marvel.agent.interval / Marvel agent configuration settings
    • marvel.agent.indices / Marvel agent configuration settings
    • marvel.agent.cluster.state.timeout / Marvel agent configuration settings
    • marvel.agent.cluster.stats.timeout / Marvel agent configuration settings
    • marvel.history.duration / Marvel agent configuration settings
  • Elasticsearch 5
    • overview / Elasticsearch 5 overview
    • performance and reliability improvements / Performance and reliability
    • data loss / Data loss
    • upgrading / Upgrading to Elasticsearch 5.0
    • upgrading, situation / When to upgrade
    • monitoring / Monitoring Elasticsearch 5
  • Elasticsearch cat API
    • about / The Elasticsearch cat API
    • reference / The Elasticsearch cat API
    • background / Background
    • document count / Count
    • health color code / Health
    • indices / Indices
    • shards / Shards
  • Elasticsearch cluster
    • configuring / Configuring an Elasticsearch cluster
  • Elasticsearch cluster configuration
    • about / Configuring an Elasticsearch cluster
    • cluster name / Cluster name
    • memory configuration / Memory configuration
    • open files limit / Open file limit
    • maximum files limit / The maximum file limit
    • max file descriptors, updating on Ubuntu Linux / Updating max file descriptors on Ubuntu Linux
    • pluggable authentication modules, enabling / Enabling pluggable authentication modules
    • open files limit, verifying / Verifying the open file limit
    • swapping, disabling / Disabling swapping
  • Elasticsearch installation
    • reference link / DEB/RPM installation
    • DEB/RPM installation / DEB/RPM installation
    • yum and apt-get repositories, using / The yum and apt-get repositories
    • Ubuntu/Debian, using / Ubuntu/Debian and apt-get
    • yum / CentOS/RHEL and yum
    • CentOS/RHEL / CentOS/RHEL and yum
    • verification / Verification
    • configuration files / Configuration files
  • Elasticsearch Mappings
    • reference link / Full-text search
  • Elasticsearch Migration Helper / Upgrading to Elasticsearch 5.0
  • Elasticsearch Search API
    • reference link / Full-text search
  • Elastic Stack / Elasticsearch 5 overview, Monitoring Elasticsearch 5

F

  • features, Elasticsearch
    • distributed / Learning more about Elasticsearch
    • fault tolerant / Learning more about Elasticsearch
    • full-text search / Learning more about Elasticsearch
    • JSON document store / Learning more about Elasticsearch
    • NoSQL / Learning more about Elasticsearch
    • non-relational / Learning more about Elasticsearch
    • analytics / Learning more about Elasticsearch
    • Dynamic Mappings / Learning more about Elasticsearch
  • fielddata cache
    • about / The fielddata cache
    • size, limiting / The fielddata cache
    • verifying / The fielddata cache

H

  • Hot-Warm architecture
    • about / Hot-Warm architecture
    • master nodes / Master nodes
    • hot nodes / Hot nodes
    • warm nodes / Warm nodes
    • URL / Warm nodes
  • hot nodes
    • about / Hot nodes
  • human readable (-h flag) / du and df

I

  • Indices API
    • reference / Background
  • installing
    • Kopf / Installing Kopf
    • about / Installation
    • Logstash / Installing Logstash
    • Kibana / Installing Kibana

K

  • Kibana
    • working with / Working with Logstash and Kibana
    • installing / Installing Kibana
    • download link / Installing Kibana
  • Kopf
    • working with / Working with Kopf
    • installing / Installing Kopf
  • Kopf installation
    • cluster page / The cluster page
    • nodes page / The nodes page
    • REST page / The rest page
    • more dropdown / The more dropdown

L

  • Logstash
    • working with / Working with Logstash and Kibana
    • installing / Installing Logstash
    • configuration file, reference link / Loading NGINX logs

M

  • Marvel
    • about / Marvel
    • components / Marvel
    • setting up / Setting up Marvel
    • upgrading / Upgrading Marvel
    • configuring / Configuring Marvel
    • agent configuration settings / Marvel agent configuration settings
    • index configuration / Marvel index configuration
    • dashboard / Understanding the Marvel dashboard
  • Marvel dashboard
    • about / Understanding the Marvel dashboard
    • Overview dashboard / Overview dashboard
    • Indices dashboard / Indices dashboard
    • Nodes dashboard / Nodes dashboard
  • master nodes
    • about / Master nodes
  • Monitoring / Elasticsearch 5 overview

N

  • Nagios
    • working with / Working with Nagios
    • installing / Installing Nagios
    • check script for Elasticsearch, reference link / Installing Nagios
  • Nagios Remote Plugin Executor (NRPE) / Installing Nagios
  • NGINX logs
    • loading / Loading NGINX logs
  • node configuration
    • case study / Node configuration
  • node failures
    • monitoring / Monitoring node failures
    • problems, diagnosing / Diagnosing problems
    • OutOfMemoryError exceptions / OutOfMemoryError exceptions
    • Elasticsearch process crashes / Elasticsearch process crashes
    • disk space / Disk space
    • case studies / Reviewing some case studies

O

  • open source full-text search engines
    • Apache Lucene / Apache Lucene
    • Solr / Solr
    • Ferret / Ferret
  • OutOfMemoryError exceptions
    • about / OutOfMemoryError exceptions
    • shard failures / Shard failures
    • slow queries / Slow queries
    • resolving / Resolving OutOfMemoryError exceptions

P

  • problem solving
    • themes / Resourcefulness and problem solving

Q

  • queries
    • analyzing / Analyzing queries
    • slow log / Slow log
  • query optimization
    • case study / Query optimization
  • query performance
    • improving / Improving query performance
    • high-cardinality fields / High-cardinality fields
    • smaller indices, querying / Querying smaller indices
    • cold indices / Cold indices
    • shard query cache / The shard query cache
    • script queries / Script queries
    • testing, meticulously / Testing meticulously

R

  • RAID 0
    • about / Drive configuration

S

  • script queries
    • about / Script queries
  • Search API query / The rest page
  • shard failures
    • about / Shard failures
    • example / Shard failures
  • shard query cache
    • about / The shard query cache
    • URL / The shard query cache
  • shards / Data distribution, redundancy, and fault tolerance
  • slow queries
    • about / Slow queries
    • example / Slow queries
  • states, Elasticsearch-head
    • green / Cluster states
    • yellow / Cluster states
    • red / Cluster states
  • stream2es utility
    • reference / Sample data
  • system configuration
    • about / System configuration
  • system performance
    • improving / System and data architecting

T

  • three-node Elasticsearch cluster
    • setting up / Cluster setup
    • configuring / Cluster configuration
    • sample data / Sample data

U

  • Ubuntu Linux
    • max file descriptors, updating / Updating max file descriptors on Ubuntu Linux

W

  • warmers
    • URL / Cold indices
  • warm nodes
    • about / Warm nodes
  • web application optimization
    • case study / Web application performance

X

  • X-Pack, tools
    • monitoring / Monitoring Elasticsearch 5
    • shield / Monitoring Elasticsearch 5
    • watcher / Monitoring Elasticsearch 5
    • graph / Monitoring Elasticsearch 5
    • reporting / Monitoring Elasticsearch 5
lock icon The rest of the chapter is locked
arrow left Previous Chapter
Chapter 1 of 15
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Sign up now
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Start free trial
Renews at $15.99/month. Cancel anytime}

Authors (3)

Left arrow icon
Profile icon Dan Noble
LinkedIn
About the Author Dan is a software engineer with a passion for writing secure, clean, and articulate code. He enjoys working with a variety of programming languages and software frameworks, particularly Python, Elasticsearch, and frontend technologies. Dan currently works on geospatial web applications and data processing systems. Dan has been a user and advocate of Elasticsearch since 2011. He has given talks about Elasticsearch at various meetup groups, and is the author of the Python Elasticsearch client “rawes.” Dan was also a technical editor for the Elasticsearch Cookbook, Second Edition, by Alberto Paro (ISBN: 1783554835). Acknowledgements I would like to thank my beautiful wife, Julie, for putting up with me while I wrote this book. Thanks for supporting me every step of the way. I would also like to thank my friends and colleagues James Cubeta, Joe McMahon, and Mahmoud Lababidi, who shared their insight, time, and support. I would like to give a special thanks to Abe Usher – you have been an incredible mentor over the years. Finally, thanks to everyone at Packt Publishing for helping to make this book happen. A special thanks to Merint Mathew, Sonali Vernekar, Husain Kanchwala, and Amey Varangaonkar for your valuable and careful feedback.
Read more
See other products by Dan Noble
Profile icon Pulkit Agrawal
LinkedIn
See other products by Pulkit Agrawal
Profile icon Mahmoud Lababidi
LinkedIn
See other products by Mahmoud Lababidi
Right arrow icon

Personalised recommendations for you

Based on your interests and search pattern
Left arrow icon
Et al.
Et al.
Ever wonder why speech recognition systems don't understand the Scottish accent, or what would happen if an astronaut only ate mac 'n' cheese, or other spurious reflections you'd have at a bar? We did, then collated those deliberations into absurd research articles with fake figures and methodologies inspired by even more fictionally absurd studies.
Aug 2023 7 hours 40 minutes
Generative AI with LangChain
Generative AI with LangChain
This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.
Dec 2023 12 hours 0 minutes
Generative AI with LangChain
Generative AI with LangChain
This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.
Dec 2023 12 hours 0 minutes
Generative AI with LangChain
Generative AI with LangChain
This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.
Dec 2023 12 hours 0 minutes
Generative AI with LangChain
Generative AI with LangChain
This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.
Dec 2023 12 hours 0 minutes
Mastering Tableau 2023
Mastering Tableau 2023
This book is a comprehensive resource to mastering your Tableau skills and becoming a BI expert. As you progress, you will learn how to build advanced dashboards and improve your storytelling to derive key business insight, as well as make you well-versed with advanced functionalities of Tableau in the business intelligence domain.
Aug 2023 22 hours 48 minutes
Building AI Applications with ChatGPT APIs
Building AI Applications with ChatGPT APIs
This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.
Sep 2023 8 hours 36 minutes
Building AI Applications with ChatGPT APIs
Building AI Applications with ChatGPT APIs
This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.
Sep 2023 8 hours 36 minutes
Data Engineering with AWS
Data Engineering with AWS
Embark on a journey to master data engineering pipelines on AWS! Our book offers a hands-on experience of AWS services for ingesting, transforming, and consuming data. Whether you're an absolute beginner or someone with basic data engineering experience, this guide is an indispensable resource.
Oct 2023 21 hours 12 minutes
Modern Data Architecture on AWS
Modern Data Architecture on AWS
Every organization wants an agile, performant, and cost-effective data platform that meets all their current and future business needs. Purpose-built AWS analytics services and their features play a big part in building such a modern data platform. This book brings to you all the design and architectural patterns that’ll help you achieve this goal.
Aug 2023 14 hours 0 minutes
Practical Guide to Applied Conformal Prediction in Python
Practical Guide to Applied Conformal Prediction in Python
Discover the power of Conformal Prediction with the "Practical Guide to Applied Conformal Prediction in Python." Master the latest techniques to quantify uncertainty in machine learning and computer vision models, and seamlessly apply them to your industry applications.
Dec 2023 8 hours 0 minutes
TinyML Cookbook
TinyML Cookbook
With over 70 project-based recipes, the TinyML Cookbook is a practical guide that will help you to get the most out of your microcontrollers. It provides a comprehensive understanding of the theoretical foundations while giving you hands-on experience training ML models for deployment on Arduino Nano 33 BLE Sense, Raspberry Pi Pico, and SparkFun RedBoard Artemis Nano microcontrollers.
Nov 2023 22 hours 8 minutes
Right arrow icon