Chapter 6. Visualizing AWS IoT Data

Some sensor data, as a result of sensing from IoT devices, is probably stored in a database engine. Getting insightful data from the IoT sensor data is more interesting when focusing on your business continuity. In this chapter, we will learn how to work with sensor data and visualize it in order to easily read and obtain insightful data.

The following is a list of topics that we will explore in this chapter:

• Introducing IoT massive data
• Introducing brief data visualization
• Data visualization in the web
• Consuming real-time sensor data in web applications
• Visualizing data from AWS IoT
• Introducing Amazon QuickSight
• Consuming sensor data from Amazon QuickSight
• Building data visualization for your IoT projects 

Introducing IoT massive data

These days, there is a lot of data we can find from various sources. Customer profiles, e-commerce transactions, sensor data from IoT, and social media are massive data. Regarding the data from IoT,  Gartner reported that 8.4 billion connected things were used in 2017. For more information, refer to https://www.gartner.com/newsroom/id/3598917. Gartner predicts that connected things, such as IoT devices, will reach 20.4 billion by 2020. 

As we know, there are a lot of new IoT platforms to implement IoT projects. These boards can generate sensor data. Each data has its own data type that represents sensor information, such as temperature and humidity.

The following figure illustrates that data comes from any source, such as sensor, transaction, social media, news, email, and medical. This data is regarded as massive data in the internet era:

Introducing brief data visualization

Data visualization is designed to present data in a graphical style. We can obtain more insight data from the data visualization result. For instance, we have temperature data from sensor measurements, as depicted in the following table. Sensor Data-Temperature is a value from the temperature sensor device:

No	Sensor Data-Temperature
1	20
2	21
3	22
4	23
5	24
6	24
7	25
8	25
9	26
10	27
11	28
12	27
13	27
14	26
15	24
16	23
17	22
18	21
19	20
20	19

We can see the temperature value at a certain hour. From this data, we can visualize the graph, as shown in the following image:

Comparing visualizing data in the table to the graph seems to show that we can get more insight if we present our data in the form of graphs. In this chapter, we will focus on how to visualize sensor data in a graphical format.

Data visualization in the web

Most people use web applications to perform their business or to present their data to customers. There are a lot of web application frameworks that we can use to develop. Corresponding to IoT data, we can apply web applications to visualize our data.

In this section, we will review some JavaScript frameworks to visualize IoT data. I will not review all the frameworks but I will start with three JavaScript frameworks in order to inspire you to use data visualization.

Data-Driven Documents (D3)

D3.js is a JavaScript library for manipulating documents based on data. D3.js has a large number of community members. It supports HTML, SVG, and CSS. Officially, you can get more information about D3.js at https://d3js.org. You can get source code for this library at https://github.com/d3/d3.

For the demo, we will develop the web program using Dj3. library. The program will display data using the Dj3 library. You can follow these steps to perform the demo:

1. In general, you...

Consuming real-time sensor data in a web application

In this section, we will learn how to visualize sensor data in real time. The scenario is to perform sensing from an IoT device and then send sensor data to the client app. This concept is similar to the publisher/subscriber application. This scenario is depicted in the following figure:

To implement how to visualize sensor data in real time, we will develop a web application that utilizes HTML5 and JavaScript. Once the sensing process is done, the sensor data will be sent to the web client application. The data pushing engine can apply the Socket.IO library to address data transferring in real time.

Socket.IO provides a JavaScript/Node.js that implements WebSocket to push data from the backend to the frontend. This library can  serve request sensor data and push the data. You can get more information about Socket.IO at https://socket.io/.

In this section, we will develop a web application using Node.js and apply Express for the web framework...

Visualizing data from AWS IoT

In the previous section, we learned how to visualize data from the backend server that was built from Node.js. In this section, we will continue to work with data visualization. Now we will consume sensor data from AWS IoT. We will develop a simple project to consume sensor data from AWS IoT and then visualize it in the browser environment.

Our demo scenario can be described in the following figure. We develop the Node.js application that performs sensing of the physical environment, such as temperature and humidity. You can use a sensor device, such DHT11 and DHT22. For this demo, I will generate random values for the temperature sensor:

The sensor data will be sent to AWS IoT with the publisher/subscriber model approach. We can use AWS IoT SDK for JavaScript/Node.js to send data to AWS IoT on a specific topic channel.  Furthermore, we will implement a client application-based on web application. We will create a simple web application using HTML5 and JS to retrieve...

Introducing Amazon QuickSight

In the previous section, we learned how to visualize sensor data from AWS IoT. We used Flot to generate data visualization. Some configurations and steps should be executed in order to visualize the data.

In this section, we will learn about Amazon QuickSight. This is one of the AWS services to visualize data for analytic purposes. Amazon QuickSight can visualize any data from any source. You should configure your data source in order to work with Amazon QuickSight.

Note

For further information on Amazon QuickSight, visit the official website at https://quicksight.aws.

A sample of Amazon QuickSight can be seen in the following screenshot:

You can watch an overview of Amazon QuickSight on YouTube. Check it out at https://www.youtube.com/watch?v=C_eT0xRNjCs.

In general, we can upload data from CSV or Excel files to Amazon QuickSight. Then, we perform data visualization. Amazon QuickSight supports AWS data sources. The following is a list of AWS data source that works...

Consuming sensor data from Amazon QuickSight

To consume sensor data from Amazon QuickSight, we should provide supported sensor data sources. For the demo, we will consume sensor data from the Excel file. I have prepared this data.  You can download it from https://github.com/agusk/awsiot/raw/master/data/Sensor-Data.xlsx.

To implement this demo, perform the following steps:

1. The first step to implement this demo is to register with Amazon QuickSight. At the time of writing, it provides a free scheme with a limited 1 GB storage. You can register at https://quicksight.aws.
2. Next, we create sensor data. For this demo, I will prepare sensor data in Excel. You can download it at https://github.com/agusk/awsiot/raw/master/data/Sensor-Data.xlsx. You can also create your own sensor data. In general, this sensor data consists of temperature and humidity, as shown in the following screenshot. This data will be uploaded to Amazon QuickSight:

3. Open Amazon QuickSight and select Manage Data. Then, you can create...

Building data visualization for your IoT projects

We have learned how to visualize sensor data using the JavaScript library. The data can come from the IoT device directly or probably be consumed from Amazon AWS. In the context of data streaming to serve huge IoT device requests, we can use AWS IoT as the frontend from the cloud system to address huge requests. Since Amazon AWS is built from cloud technology, we are confident about its availability. 

In general, we can build an IoT project by combining various AWS services, as shown in the following figure:

You select the IoT device that should be supported by AWS IoT. If not supported, you should build your own library that enables accessing AWS IoT. Within AWS IoT, you can save in the local AWS using various AWS services, such as Amazon RDS, Amazon EC2 with the installed SQL Server, MySQL, or PostgreSQL.

We have learned to visualize sensor data directly from AWS IoT with the publisher/subscriber. You can implement data visualization with...

Summary

We have now learned how to work with data visualization. We have implemented data visualization with various JavaScript engines. We have also performed data visualization from AWS IoT. Finally, we explored Amazon QuickSight and performed data visualization.

In the next chapter, we will learn how to build predictive analytics with AWS IoT and AWS Machine Learning.