Extracting and Visualizing Data with InfluxDB and Grafana

In the previous chapter, we concentrated our efforts on understanding how a data source is primary to the Grafana visualization workflow. We launched a Prometheus Docker container along with a Grafana server, scraped data from both applications, and then configured a Grafana data source to connect to the Prometheus server. Finally, we used the Explore module to get a feel for how to make various queries to the data source and get immediate feedback in the graph display.

While Explore is a powerful mechanism for browsing a data source, it is somewhat limited in functionality compared to the time series visualization This is not surprising as it’s mostly intended to support ad hoc, transient queries with more permanent graphs living on a dashboard. Those graphs have the advantage of providing several significant features that benefit presentation and alerting.

With that in mind, we’re going to take what we...

Technical requirements

In this chapter, we’ll be setting up InfluxDB and Grafana containers, so make sure you have installed Docker and Docker Compose.

Information

The tutorial code, dashboards, and other helpful files for this chapter can be found in this book’s GitHub repository at https://github.com/PacktPublishing/Learn-Grafana-10/tree/main/Chapter05.

Making advanced queries

Before we can start playing with our pretty data, we’ll need to put together a simple data pipeline. Patience! This is likely to be one of the rare opportunities where you will have significant control of the data that goes into your data source. Even if you don’t ever plan to involve yourself in data acquisition, it helps to know some of the techniques and issues surrounding it, if only to appreciate the work that often goes into tailoring and cleaning data so that it can be analyzed or visualized.

Our plan of attack for this part of our tutorial is quite straightforward:

1. Spin up both an InfluxDB and a Grafana server.
2. Code-review a simple extract, transform, load (ETL) script to gather weather data.
3. Execute the script to populate an InfluxDB database.
4. Configure an InfluxDB data source.

Let’s get started!

Launching server Docker containers

The first step is to run a docker-compose script that will download...

Understanding the time series data visualization

In this section, we are going to cover some important concepts surrounding time aggregation. For us to do that, we’re going to craft a time series visualization panel that illustrates those concepts. Along the way, we’ll be covering some of the more advanced drawing features of the time series visualization.

The concepts are a bit technical, but understanding them is essential to mastering the depiction of time-based data in Grafana and other time series visualization tools:

1. Start by creating a new dashboard and then click + Add visualization.
2. Select the default InfluxDB data source.
3. Set the time range to Last 24 hours.
4. In the Query tab, click on the copy (two pages) icon to make a copy of the current query.
5. Click the visibility (eye) icon for the B query to disable it. We’ll set it in a moment.

We are going to modify the A query so that it concentrates on a single data series &...

Setting the axis

Now that we’ve broken down how data points are graphed horizontally in time, let’s look at how they are graphed vertically on the y axis. I’m sure whole books have been devoted to documenting how the y axis has been used and abused, from using a logarithmic scale instead of linear or vice versa to improper scale to truncation, but space doesn’t permit going into all these issues.

Like any tool, we can abuse the flexibility of Grafana’s y axis display. In this section, we’re going to point out the opportunities for leveraging the y axis display to hopefully clarify or illuminate our data. We’ll be creating a series of panels depicting various weather observations and then concentrating on different ways to adjust the y axis, including scaling, units, and even the use of multiple y axes on a single graph.

Setting axis units

Let’s start by creating a new dashboard panel by clicking the Add | Visualization...

Working with legends

In the previous sections, we spent some time learning how to manage the horizontal and vertical display of our graph data. Now, we’ll look at a key piece of graph display that is often overlooked: the legend. On many graphs, the legend seems like an afterthought, often floating in some non-specific whitespace where there’s a convenient lack of data.

Grafana is somewhat more restrained about the legend. It can live below the graph (or to its right) and can take on a list or a table format; that’s it. However, as we’ve seen, the label content of the graph can be set by the Alias field, and that field can be matched in field overrides. It’s that functionality that we can leverage when interacting with the legend interface.

Setting legend contents

Let’s start with another graph, again for temperature. Use the following query settings:

• FROM: temperature
• SELECT: field(value) | mean()
• GROUP BY: time($_interval...

Summary

We’ve covered a lot of ground in this chapter. We wrote a simple Python ETL script to scrape data from a web-based API and import that dataset into InfluxDB. We also learned about key concepts behind time and field value aggregations. Then, we tried out different drawing styles and learned how to instruct Grafana on how to connect the dots when there is missing data.

We also set axis units, converted our data from one unit of measure into another, and displayed multiple series with different units on the same graph. Finally, we worked with the legend display to make it more space-efficient and aesthetically pleasing.

In the next chapter, we’ll be diversifying our display panels so that they include panels that are more specialized in functionality. While these panels are somewhat more limited, they still complement the graph panel by characterizing data in truly unique ways.