Anomaly Detection and Recommendation Systems

This chapter will look at auto-encoder models and recommendation systems. Although these two use cases may seem very different, they both rely on finding different representations of data. These representations are similar to the embeddings we saw in Chapter 7, Natural Language Processing Using Deep Learning. The first part of this chapter introduces unsupervised learning where there is no specific outcome to be predicted. The next section provides a conceptual overview of auto-encoder models in a machine learning and deep neural network context in particular. We will show you how to build and apply an auto-encoder model to identify anomalous data. Such atypical data may be bad data or outliers, but could also be instances that require further investigation, for example, fraud detection. An example of applying anomaly detection is detecting...

What is unsupervised learning?

So far, we have focused on models and techniques that broadly fall under the category of supervised learning. Supervised learning is supervised because the task is for the machine to learn the relationship between a set of variables or features and one or more outcomes. For example, in Chapter 4, Training Deep Prediction Models, we wanted to predict whether someone would visit a store in the next 14 days. In this chapter, we will delve into methods of unsupervised learning. In contrast with supervised learning, where there is an outcome variable(s) or labeled data is being used, unsupervised learning does not use any outcomes or labeled data. Unsupervised learning uses only input features for learning. A common example of unsupervised learning is cluster analysis, such as k-means clustering, where the machine learns hidden or latent clusters in the...

How do auto-encoders work?

Auto-encoders are a form of dimensionality reduction technique. When they are used in this manner, they mathematically and conceptually have similarities to other dimensionality reduction techniques such as PCA. Auto-encoders consist of two parts: an encoder which creates a representation of the data, and a decoder which tries to reproduce or predict the inputs. Thus, the hidden layers and neurons are not maps between an input and some other outcome, but are self (auto)-encoding. Given sufficient complexity, auto-encoders can simply learn the identity function, and the hidden neurons will exactly mirror the raw data, resulting in no meaningful benefit. Similarly, in PCA, using all the principal components also provides no benefit. Therefore, the best auto-encoder is not necessarily the most accurate one, but one that reveals some meaningful structure...

Training an auto-encoder in R

In this section, we are going to train an auto-encoder in R and show you that it can be used as a dimensionality reduction technique. We will compare it with the approach we took in Chapter 2, Training a Prediction Model, where we used PCA to find the principal components in the image data. In that example, we used PCA and found that 23 factors was sufficient to explain 50% of the variance in the data. We built a neural network model using just these 23 factors to classify a dataset with either 5 or 6. We got 97.86% accuracy in that example.

We are going to follow a similar process in this example, and we will use the MINST dataset again. The following code from Chapter8/encoder.R loads the data. We will use half the data for training an auto-encoder and the other half will be used to build a classification model to evaluate how good the auto-encoder...

Using auto-encoders for anomaly detection

Now that we have built an auto-encoder and accessed the features of the inner layers, we will move on to an example of how auto-encoders can be used for anomaly detection. The premise here is quite simple: we take the reconstructed outputs from the decoder and see which instances have the most error, that is, which instances are the most difficult for the decoder to reconstruct. The code that is used here is in Chapter9/anomaly.R, and we will be using the UCI HAR dataset that we have already been introduced to in Chapter 2, Training a Prediction Model. If you have not already downloaded the data, go back to that chapter for instructions on how to do so.. The first part of the code loads the data, and we subset the features to only use the ones with mean, sd, and skewness in the feature names:

library(keras)
library(ggplot2)
train.x <...

Use case – collaborative filtering

This use-case is about collaborative filtering. We are going to build a recommendation system based on embeddings created from a deep learning model. To do this, we are going to use the same dataset we used in Chapter 4, Training Deep Prediction Models, which is the retail transactional database. If you have not already downloaded the database, then go to the following link, https://www.dunnhumby.com/sourcefiles, and select Let’s Get Sort-of-Real. Select the option for the smallest dataset, titled All transactions for a randomly selected sample of 5,000 customers. Once you have read the terms and conditions and downloaded the dataset to your computer, unzip it into a directory called dunnhumby/in under the code folder. Ensure that the files are unzipped directly under this folder, and not a subdirectory, as you may have to copy...

Summary

I hope that this chapter has shown you that deep learning is not just about computer vision and NLP problems! In this chapter, we covered using Keras to build auto-encoders and recommendation systems. We saw that auto-encoders can be used as a form of dimensionality reduction and, in their simplest forms with only one layer, they are similar to PCA. We used an auto-encoder model to create an anomaly detection system. If the reconstruction error in the auto-encoder model was over a threshold, then we marked that instance as a potential anomaly. Our second major example in this chapter built a recommendation system using Keras. We constructed a dataset of implicit ratings from transactional data and built a recommendation system. We demonstrated the practical application of this model by showing you how it could be used for cross-sell purposes.

In the next chapter, we will...