Chapter 2: Installing and Running the Elastic Stack

Now that you know about the core components of the Elastic Stack and what they do, it is time to get them installed and running. As described in the previous chapter, there are a few options available to orchestrate the stack. All core components can be run on multiple platforms (such as Linux, Windows, containers, and so on). This chapter will look at the most common platforms for installation, rather than exhaustively covering all installation options supported by the components.

While it is good to understand how the different installation options work, you can choose your preferred method to run your lab environment to follow along with the examples in future chapters of this book.

In this chapter, we will specifically look at the following:

• Manual installation of Elasticsearch, Kibana, Logstash, and Beats
• Automating the installation of components using Ansible
• Using Elastic Cloud Enterprise to orchestrate...

Technical requirements

The following technical requirements should be considered in order to follow the instructions in the following sections:

• Access to a cloud platform to provision compute instances and managed Kubernetes clusters is recommended. Most cloud providers provide free tiers for those interested.
• Ansible installed on your machine. Instructions to install Ansible can be found in the Ansible installation guide: https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html.
• The kubectl command-line tool installed for running on Kubernetes. Instructions to install kubectl can be found in the Kubernetes documentation guide: https://kubernetes.io/docs/tasks/tools/install-kubectl/.
• Access to code samples for Chapter 2 of this book's git clone can be found at https://github.com/PacktPublishing/Getting-Started-with-Elastic-Stack-8.0/tree/main/Chapter2.

Manual installation of the stack

Elastic Stack components can be run on both Linux and Windows environments easily. The components are run as standalone products and do not offer any type of orchestration or automation capabilities. We will focus on Linux environments in this book; instructions for Windows-based environments can be found on the Elasticsearch reference guide:

https://www.elastic.co/guide/en/elasticsearch/reference/8.0/zip-windows.html

In order to complete the setup of the lab Elasticsearch cluster, please complete both the installation and configuration steps for your operating system/installation method of choice.

Elastic Stack components are often installed on dedicated hosts/machines depending on architecture, resource, and performance requirements. This chapter looks at installation while Chapter 13, Architecting Workloads on the Elastic Stack, delves into the architectural considerations around each component.

Elastic Stack components support various...

Automating the installation

Configuration management tools such as Ansible and Puppet can be used to automate the installation and configuration of Elasticsearch clusters and other components. Using automation for installation comes with the following benefits:

• Quick deployment times, especially for large clusters.
• Reduces the risk of the misconfiguration of nodes.
• Automation configuration can be tracked in source control and integrated as part of your CI/CD processes.
• Automation can be run at regular intervals in an idempotent manner to revert any manual configuration changes to the environment.
• Components can be easily replicated in other environments (such as dev/staging before rolling out to production).
• Can be used as part of a disaster recovery strategy to quickly re-create the cluster and components in an alternate cloud region or data center in the event of a disaster.

You can use the tool and framework of your choice to automate the installation...

Using Elastic Cloud Enterprise (ECE) for orchestration

Managing large Elasticsearch clusters manually can often require a lot of work. Scaling clusters and managing version upgrades and so on requires administrators to plan and execute changes, often in sensitive environments. This effect can be compounded if a team is managing multiple Elasticsearch deployments within an organization. Orchestration capabilities can significantly help in such scenarios.

ECE is a subscription product offered by Elastic, giving teams the ability to provision and manage Elasticsearch deployments using an easy-to-use API or a web interface. ECE can handle rolling upgrades to deploy changes in configuration, stack version upgrades, and deployment scaling events without any manual intervention. It also supports multi-zone setups to provide high availability to Elasticsearch deployments.

ECE architecture

ECE uses Docker to run platform services as virtualized components. It relies on the following...

Running on Kubernetes

Elastic Stack components can be run on Kubernetes using Elastic Cloud on Kubernetes (ECK). Kubernetes is an open source platform for managing containerized workloads using declarative configuration. Kubernetes is a popular platform used by many organizations for existing workloads. ECK is a Kubernetes operator that supports the orchestration of Elasticsearch, Kibana, and Beats deployments. It can also handle rolling upgrades and configuration changes without downtime. The ECK operator allows teams that run existing workloads on Kubernetes to deploy and manage multiple Elasticsearch deployments on the same platform they already use with ease.

Note

ECK is a paid subscription feature offered by Elastic.

Provision a Kubernetes cluster to test the deployment of Elasticsearch and Kibana before running the following commands. You can use either a local or a cloud-based managed Kubernetes cluster, such as Google GKE or AWS EKS:

1. Install the ECK Kubernetes...

Configuration of your lab environment

For the rest of the book, the default lab environment should consist of at least the following:

• A single-node Elasticsearch cluster with at least a 500 MB heap
• A single instance of Kibana connected to the Elasticsearch cluster
• A single instance of Logstash
• The ability to install and run Beats as required

For convenience, you may choose to run all of the above on your local machine. Archives of each component can be downloaded and run whenever needed. Exercises in the book will assume you have Elasticsearch and Kibana up and running. Instructions to start Logstash/Beats instances will be included.

Some of you may also choose to use a cloud-based managed deployment of Elasticsearch and Kibana. Elastic Cloud provides a free trial for those interested in this option.

Summary

In this chapter, we explored multiple options to install and run the core components of the Elastic Stack. We looked at manual installation options on Linux hosts before automating installations using Ansible. We also looked at orchestrating Elasticsearch deployments using ECE and ECK. Finally, we looked at what a lab environment might look like for you to follow along in the chapters ahead.

Now that we can successfully install and run Elastic components, the following chapter will dive deep into Elasticsearch. We will explore Elasticsearch concepts and approaches to configuring our first Elasticsearch cluster.