Chapter 4: Data Sources for Grafana

In this chapter, you will learn about the various data sources that you can use in Grafana. You will also learn about data concepts that are used in these data sources.

Grafana accepts different types of data sources, but we will focus mainly on NoSQL databases and, specifically, on time series databases (TSDBs). Also, you will learn how to feed Grafana with Excel, CSV files, and live data.

The main topics of this chapter are as follows:

• What is a data source?
• SQL versus NoSQL databases.
• What are TSDBs?
• Google Sheets and CSV files.
• How to use live data. 

Data is the cornerstone of any Internet of Things (IoT) platform. A data source, as its name implies, is the origin of all the data that Grafana uses. So, this is a very important subject for understanding the upcoming chapters.

Let’s see the meaning of data sources in the context of Grafana.

What is a data source?

A data source is just a resource that Grafana uses to obtain the data.

Then, Grafana uses this data to perform visualizations, transformations, and analytics. You can control every aspect of this process from the web interface of Grafana.

So, data sources are data repositories that Grafana can access through its API.

There are different types of data sources, and the list is always growing. To connect to the data sources, Grafana uses plugins (we will see how to connect to data sources in Part 3).

Let’s see some data sources available to use with Grafana:

• MySQL
• MongoDB
• Graphite
• InfluxDB
• OpenTSDB
• PostgreSQL
• Elasticsearch
• Azure Monitor
• Prometheus
• Loki
• Test data
• CSV files and Excel files

This is just a short list of data sources. You can find all the available data sources at https://grafana.com/grafana/plugins/?type=datasource.

To navigate and explore the data source plugins installed...

SQL versus NoSQL databases

In this section, we will explore these two types of databases and define their characteristics and differences. Learning about databases is crucial to understanding how to manage, query, and process data in the context of Grafana.

Let’s start with SQL databases.

SQL databases

This type of database was invented in the 1970s, and it is widely used today.

A relational database management system (RDBMS) is a system that manages data according to a relational model. This type of database represents data in tables, where each of the rows has a unique ID, called an index or ID. This field serves to relate the information among the tables.

Imagine you have a user database for accessing a web application. One table can store all the information that belongs to a specific user. Another table can have the permissions granted to each user. In this case, the user ID is the link between both tables. So, you can relate each user with the corresponding...

What are TSDBs?

Every single IoT device out there is sending data to some IoT platform. As we have discussed in the previous section, this data can come in different types and formats. However, all this data shares one thing in common: the timestamp.

The timestamp is as valuable as the data itself. Each data point is associated with timestamp information, so you can see when it was created. Knowing every data point timestamp allows you to order the information and build a historical trend. Then, you can visualize it, run analytics on it, perform transformations, and so on. You can see an example in the following figure:

Figure 4.2 – Time series data

Although the previous discussion can seem a little obvious, it reveals the critical value of the timestamp in IoT applications.

A TSDB allows you to manage time-related data by just performing simple queries. Every record in the database is associated with a timestamp. So, you can obtain only the data...

Google Sheets and CSV files

Grafana not only allows you to connect to databases, but it also admits parsing files and cloud services, such as CSV files and Google Sheets.

In simple IoT applications, you may find that data is being saved in Google Sheets or text files. Also, you could have data stored on SD cards or in old files.

So, you can have either living data ingressing in a cloud spreadsheet or historical data stored in the cloud or local files.

In any case, you may want to build a dashboard with this information to analyze it or even show it to other people.

Google Sheets data source

Although you will see how to manage and install plugins in Chapter 7, Managing Plugins, let’s see how to install the Google Sheets plugin.

You have two options: local installation and cloud installation.

Local installation

The Google Sheets plugin doesn’t come by default with Grafana, so you have to install it.

In a local instance, you will have to use the...

How to use live data

Using live data is a must in any IoT project. Fortunately, Grafana allows using live data since version 8.0. It uses WebSocket connections to exchange data in a quasi-real-time way.

This feature is enabled by default, with a maximum number of simultaneous connections of 100. WebSocket is a resource-intense protocol so it has to be used carefully. If you want to extend this feature to more clients, take into account the availability of resources (CPU and memory).

As we have seen in Chapter 3, Connecting IoT Devices, WebSocket initiates with an HTTP request, followed by an upgrade request. Then, the HTTP session remains open and data is exchanged between client and server using the WebSocket connection.

Features

Currently, Grafana supports the following features:

• It processes data as soon as it arrives. Instead of polling data from data sources at a specified polling period, it can manage asynchronous data in real-time.
• Whenever a dashboard...

Summary

In this chapter, you have met Grafana data sources. We have explored the different types of data sources that you can use in Grafana. We have addressed some database concepts, and you have learned the difference between SQL and NoSQL databases. Finally, you have learned how to use files as data sources, from the cloud and locally. In the next chapter, you will learn about TSDBs.