Analytics in the IoT Context

In any non-trivial IoT use case, a huge volume of data is generated at a high speed. This high-volume data needs to be analyzed at similar speeds so that meaningful insights can be deduced, and the required actions can be triggered quickly. Most of the advancements in (generic) analytics can be applied directly to IoT use cases, but two key characteristics of data ingestion (that is, high volume and high frequency) necessitate that some special considerations are taken while reusing generic learnings/algorithms in the context of IoT. For example, IoT visualizations (dashboards) need to be displayed at reasonable granularity while not missing out on crucial/anomalous data points.

In addition to data volume and data velocity, IoT data is different as it can be a combination of structured (sensed values in time series format, such as temperature values captured at intervals of 1 second, and inventory data), semi-structured (operator comments), and unstructured...

Key terms/definitions

In this section, we’ll look at some key concepts that are relevant to IoT analytics:

• AI: AI intends to replicate human intelligence by using systems that can learn from past decisions, predict future scenarios, and continuously improve decision-making capabilities. AI has special relevance in the IoT context as the data that needs to be processed is high in volume, velocity, and variety, as discussed in the An overview of IoT section in Chapter 1 (the seven Vs of IoT data). This can’t be processed by traditional computing systems that are strictly rule-based (if X happens, do Y) and can only serve a very narrow purpose. Complex decisions need to be made based on the values in the data stream that are beyond the capabilities of traditional computing/programming systems. Chapter 12, Exploring Synergies with Emerging Technologies, describes in detail how AI and IoT act as complementary technologies to solve real-world challenges such as the...

Implementing IoT analytics

Although this section specifies the characteristics/considerations of IoT analytics from a technical standpoint, it is worth noting that when implementing an IoT use case, domain know-how is equally important. This know-how varies vastly from one domain to another – for example, the mechanism for detecting anomalies in the agriculture domain would be quite different from the one used for detecting anomalies in the manufacturing domain. Some of the typical scenarios/use cases for which IoT analytics is used are shown in the following figure:

Figure 10.3 – Application scenarios for IoT analytics

IoT analytics is categorized into four different areas, depending on the insights that are generated:

• Descriptive analytics: In this type of analytics, stored historical data is analyzed to provide a view of historical performance, anomalies, and more. Even the real-time data stream can be analyzed, but the focus remains...

Understanding the importance of data quality

The different types of data quality issues that are generally found in IoT use cases can be seen in the following figure:

Figure 10.8 – Typical data quality issues

The source of these data quality issues can be traced to different layers of the IoT reference stack, as shown in the following figure:

Figure 10.9 – Data quality issues at different layers of the IoT stack

The scale of IoT deployments (for example, a large number of field devices generating humongous data) tends to amplify even minor quality issues. The tolerance for data quality issues varies across organizations and use cases, and accordingly, the rigor of data quality mechanisms will also vary.

Applicable data quality initiatives will depend on factors such as the nature of the data collected and the purpose for which the data is being collected.

Closely related to the concept of data quality is the concept...

Relevance of edge analytics

IoT devices are not permanently connected to a central server, so some amount of processing/analytics needs to be done locally so that these devices can function independently if they’re not connected to a central server. This is one scenario where edge analytics is required. Essentially, edge analytics refers to processing IoT data near the point at which it is generated. In other words, edge analytics refers to the scenario where analytics data is sent to the point of data generation rather than being sent to the point where analytics and algorithms are hosted or deployed. This definition points to the fact that edge analytics can be implemented on a variety of physical infrastructures (device gateways, on-premises servers, or data centers physically located close to field devices).

Distributing data processing workloads between edge and central server depends on use case requirements – most IoT use cases rely on a hybrid approach. Usually...

Considerations for IoT visualization

The main objective of IoT visualization is to make it easier for information consumers to understand data trends and obtain insights by highlighting patterns, relationships, trends, and outliers. Visualization should also help disseminate insights to non-technical users. The rationale regarding the decisions or actions taken by AI/ML algorithms is generally hidden from the user. Effective visualization can also help fill this gap by providing transparency into how AI/ML algorithms have arrived at a decision. Similarly, data lineage can be better understood if it is represented in the form of visualizations.

IoT data and insights are consumed on a diverse set of devices. In addition to traditional devices such as desktops and mobile devices, information is consumed on human-machine interface (HMI) devices (such as industrial control panels with buttons and indicator lights). This requires entirely different layout considerations than those that...

Summary

This chapter covered the importance of analytics in the context of IoT. This process converts raw data that’s received from diverse sources (static data sources such as enterprise systems, as well as dynamic data sources – for example, data received in real time from sensors) into meaningful insights, which is a prerequisite for effective/efficient decision making. We covered specific considerations that you need to be aware of while tailoring generic analytics for IoT applications. Then, we covered the steps that are normally followed in any analytics data pipeline. We also looked at the benefits and nuances of edge analytics, where data is processed close to the data source.

After that, we introduced edge analytics; we will cover this in more detail in Chapter 12, Exploring Synergies with Emerging Technologies. We looked at the importance of data quality and how it can be ensured at the different layers of the IoT stack, as well as the importance of IoT visualizations...