Chapter 10. Data Science for IoT Analytics

"Revenues are up 5% due to your little geospatial search trick," the VP of Connected Services says, "You know your former boss's position is still open. Maybe we should fill it from the inside..."

Your pulse quickens, you were hoping he might come to this conclusion. You deserve a promotion after what your analytics has brought to the company. You now have one person working for you focusing on geospatial analysis. You can just imagine what you could do with a whole team.

"There is something that we have been toying around with though," he continues, "With all this data we are collecting, we should be able to tap into machine learning models to predict equipment failures. Some think we should be hiring an outside consulting company to handle all of it. Sounds expensive to me. I sure wish we could coordinate this ourselves, work with data scientists of our own choosing... know anyone that might be up to it?"

He winks and walks off, hands behind his...

Machine learning (ML)

As a wise uncle of a human-arachnid hybrid once said, With great power, comes great responsibility. This is very true of ML. There are many ways to go wrong. When in the hands of a skilled practitioner, it truly is a form of art. It can be used to do some incredible things on a grand scale, but it should come with a big caution sign. Use it carefully. Be paranoid and validate, validate, validate.

Although we will be going over some core concepts and providing code that you can take and run yourself, this is a big field with lots to learn. It takes years to skillfully and competently apply it. Each section in this chapter is really a book in itself. No, many books. If you plan to use it yourself on IoT data, read, read, read, and then read some more. This chapter is meant to able to provide you with a good foundation to have meaningful conversations with data scientists on the subject.

What is machine learning?

Ask a hundred experts for the definition of ML, and you are...

Anomaly detection using R

Anomaly detection is a way to use historical data to identify unusual observations without requiring a labeled training set. Modern anomaly detection methods take into account long-term trends and cyclical variation in the data while determining which observations to flag as anomalies.

Twitter has recently released an advanced open source anomaly detection package for R called AnomalyDetection. It is geared toward detecting anomalies in single value high frequency (less than a day) time series data; however, it is possible to set an option to handle datasets longer than a month. It can also be used on a vector of non-time series data.

It is good at handling the effects of trends and seasonality - although seasonality, in this case, is at the minutes to days level not yearly. The GitHub page is located here (https://github.com/twitter/AnomalyDetection), and it can be installed easily as an R package using the following R code. Make sure to spell Anomaly with a capital...

Forecasting using ARIMA

Sometimes, you will have the need to forecast future values of a time series. For example, this could be a requirement to estimate the next several months of active IoT devices; or, it could be a need to project the usage hours of remote oil well pumps. One of the most popular methods to forecast time series is AutoRegressive Integrated Moving Average (ARIMA).

ARIMA is not one model but a collection of related methods that attempt to describe autocorrelations in the data in order to forecast future values. ARIMA is a combination of moving average and autoregressive techniques. Autoregressive means that the forecasting of future values of a variable is based on the linear combination of the past values of variables.

ARIMA incorporates both trend and seasonality effects into future forecasts. It can model both seasonal and nonseasonal data with a range of methods.

Using R to forecast time series IoT data

The forecast package contains ARIMA functions in R. You can install...

Deep learning

Deep learning is an area of data science that is experiencing rapid advancement and generating a lot of excitement. Some deep learning models are better at certain types of image recognition than humans. When stories in the media mention artificial intelligence, they are usually referring to deep learning models.

Deep learning models are very complex although several of the concepts are similar to the ML concepts we have discussed so far in this chapter (such as the bias-variance tradeoff). Deep learning models can have millions of features and can take days or weeks to train.

Use cases for deep learning with IoT data

Deep learning can do wonders for complex data, with thousands to millions of features and a large history of labeled examples to use as training sets. The rapid advancements in image recognition has as much to do with the vast trove of identified images that Google and others have collected over the years, as to the advances in the deep learning algorithms used.

For...

Summary

This chapter was a whirlwind tour of various areas in data science. It was covered with an eye on how to use each for the IoT analytics. Believe it or not, we covered only a small amount of topics in a very big space. Hope your head does not hurt too much.