Prometheus Query Language - PromQL

Prometheus offers a powerful and flexible query language in order to leverage its multi-dimensional data model that allows ad hoc aggregation and a combination of time series data. In this chapter, we'll introduce PromQL, its syntax, and semantics. Armed with the knowledge and features of this language, we'll be able to unlock the true potential of Prometheus.

In brief, the following topics will be covered in this chapter:

• The test environment for this chapter
• Getting to know the basics of PromQL
• Common patterns and pitfalls
• Moving on to more complex queries

The test environment for this chapter

In this chapter, we will be using a Kubernetes-based environment to generate all the metrics we need to test the PromQL examples that are covered in this chapter. Using the Prometheus Operator, the setup of this environment is quite simple; go through the following steps to get yourself up and running:

1. To start the Kubernetes test environment, we first must ensure there's no instance of minikube running:

minikube status
minikube delete

2. Start a new minikube instance with the following specifications:

minikube start \
  --cpus=2 \
  --memory=3072 \
  --kubernetes-version="v1.14.0" \
  --vm-driver=virtualbox

When the previous command finishes, a new Kubernetes environment should be ready to be used.

For our Kubernetes test environment, we'll be building upon the lessons we learned about in Chapter 5, Running a Prometheus Server...

Getting to know the basics of PromQL

Understanding the Prometheus Query Language is essential to be able to perform insightful dashboarding, capacity planning, and alerting. But for that, we need to begin by learning the basics. The following topics will cover the components that available to construct queries and look into how they behave together.

Selectors

Prometheus is designed to handle hundreds of thousands of time series. Each metric name can have several different time series, depending on the combination of labels; querying the right data can look difficult, or even downright perplexing, when similarly-named metrics from different jobs are mixed together. In Prometheus, a selector refers to a set of label matchers...

Common patterns and pitfalls

With such a powerful language at your disposal, it's easy to become overwhelmed with so many options. In the following sections, we'll provide some common patterns and pitfalls to ensure the intended use of PromQL for each situation described, and in this way, further enforcing the knowledge that we have provided you with so far.

Patterns

While the power and flexibility of PromQL allows for a world of possibilities in terms of information extraction, there are a few query patterns that make a set of common problems much easier to understand and help increase the level of insight into the monitored services. In the following topics, we'll be covering a few of our favorites, in the...

Moving on to more complex queries

With the information that's provided so far, we can move on to more complex queries, understand how they are built, and what to expect from them. In the following sections, we'll go over some intricate scenarios that demand the use of PromQL to explore and solidify the concepts we've covered so far.

In which node is Node Exporter running?

This scenario is designed to assist your understanding of concepts such as info metrics and the group_left modifier.

Scenario rationale

When running on Kubernetes, you might need to troubleshoot...

Summary

In this chapter, we learned about the basics of PromQL, from selectors to functions, covering concepts such as binary operators, vector matching, and aggregations. Going through the common patterns and pitfalls, we were introduced to how this language allows much more than simple querying and how it has become an essential infrastructure tool, helping with the design and management of SLIs and SLOs. We also demonstrated several scenarios where PromQL shines, and how seemingly complex queries are not that complex after all.

In the next chapter, Chapter 8, Troubleshooting and Validation, we'll delve into how to validate a healthy Prometheus setup and learn how to troubleshoot issues quickly, ensuring the stability of the monitoring stack.

Questions

1. What are the six available comparison operators in PromQL?
2. When should a group_right modifier be used instead of a group_left one?
3. Why shouldn't you use the sort() function when applying the topk aggregation operator?
4. What is the major difference between rate() and irate()?
5. Which type of metric has an _info suffix and what is its purpose?
6. Should you sum and then rate or rate and then sum?
7. How can you get the average CPU usage for the last five minutes in a percentage?

Further reading

• Querying Prometheus: https://prometheus.io/docs/prometheus/latest/querying/basics/
• SRE Book - Service Level Objectives: https://landing.google.com/sre/sre-book/chapters/service-level-objectives/