Deploying Prometheus

Now that we understand a little more about observability and monitoring, as well as their distinctions and how Prometheus fits into it all, we can deploy Prometheus. In this book, we’ll be deploying Prometheus to a managed Kubernetes environment using Linode Kubernetes Engine (LKE). However, you can substitute AWS’s EKS, Azure’s AKS, or any other engine as you see fit.

You may already have experience with deploying Prometheus, but we’re going to use the Prometheus environment that we’ll deploy in this chapter as the basis for future chapters in which we’ll be deploying additional applications that extend Prometheus and make configuration changes to Prometheus. Consequently, it is recommended that you follow along with the deployment, even if you have prior experience.

In this chapter, we will cover the following topics:

• Components of a Prometheus stack
• Provisioning Kubernetes
• Deploying the Prometheus...

Technical requirements

You’ll need to install the following programs and make them available in your PATH:

• linode-cli: https://github.com/linode/linode-cli
• kubectl: https://kubernetes.io/docs/tasks/tools/#kubectl
• helm: https://helm.sh/docs/intro/install/

This chapter’s code is available at https://github.com/PacktPublishing/Mastering-Prometheus.

Components of a Prometheus stack

When discussing Prometheus, people rarely mean just the Prometheus project (https://github.com/prometheus/prometheus) itself. Instead, they refer to the following:

• Prometheus as a full solution in which you deploy Prometheus for metrics collection, storage, and alerting rule evaluation
• Prometheus’s Alertmanager (https://github.com/prometheus/alertmanager) for routing alerts coming from Prometheus
• One or more exporters, such as Node Exporter (https://github.com/prometheus/node_exporter), to expose metrics
• Grafana (https://github.com/grafana/grafana) to visualize metrics from Prometheus via dashboards

Each of these components fits together to form a complete metrics solution for observability.

Prometheus

Understandably, Prometheus is the core technology in a Prometheus stack. Prometheus is comprised of four main parts: the time series database (TSDB), the scrape manager, the rule manager, and the web UI (along with...

Provisioning Kubernetes

Let’s dive into setting Prometheus up! But first, we need to set up some infrastructure so that we can run Prometheus.

Throughout this book, I’ll be running Prometheus on Linode using LKE. However, the same principles can be transferred to other Kubernetes environments (AKS, EKS, Minikube, and so on). Additionally, since we’ll be using the Prometheus Operator – which abstracts away some pieces of Prometheus – I’ll mention how to do things when running the Prometheus stack directly on Linux too.

To begin, you can register for a Linode account and get $100/60-day credit by going to https://www.linode.com/mastering-prometheus.

Configuring the linode-cli

With your Linode account created, we’ll install the Linode CLI to handle creating our Kubernetes infrastructure via the command line.

Presuming you have Python 3 installed, the Linode CLI can be installed via pip:

$ pip install linode-cli

At this...

Deploying Prometheus

Our Prometheus deployment will be done using a Helm chart that installs kube-prometheus, which installs and builds upon the Prometheus Operator by providing a convenient set of defaults. We’ll begin by briefly going over what you’re going to be deploying so that you understand it instead of blindly applying it because someone in a book told you to.

Prometheus Operator overview

The Prometheus Operator is a project that was originally started by Prometheus maintainer Frederic Branczyk to make it simpler to deploy and manage multiple Prometheus instances in a Kubernetes cluster. Consequently, it abstracts away some pieces of Prometheus behind various Kubernetes Custom Resource Definitions (CRDs).

The main CRDs to be aware of with the Prometheus Operator are Prometheus, ServiceMonitor, PrometheusRule, and Alertmanager.

Prometheus and Alertmanager are straightforward, but PrometheusRule and ServiceMonitor are key abstractions. PrometheusRule...

Summary

In this chapter, we went over the key components of a basic Prometheus stack: Prometheus, Alertmanager, Node Exporter, and Grafana. We looked at Prometheus’s internal components to understand what it does and what it delegates to other services, such as Alertmanager. Finally, we went hands-on and built a Kubernetes cluster and Prometheus stack from the ground up in record time.

Going forward, we’ll build upon this architecture to see how Prometheus scales and how to extend it. But first, we’re going to dive into some of the nitty-gritty of Prometheus by focusing on its data model and query language so that we can get the most out of the data we’re collecting from Prometheus.

Further reading

To learn more about the topics that were covered in this chapter, take a look at the following resource:

• The prometheus-operator project: https://github.com/prometheus-operator/prometheus-operator