Performing Anomaly Detection on Unsupervised Data

In this chapter, we will perform anomaly detection with the Modified National Institute of Standards and Technology (MNIST) dataset using a simple autoencoder without any pretraining. We will identify the outliers in the given MNIST data. Outlier digits can be considered as most untypical or not normal digits. We will encode the MNIST data and then decode it back in the output layer. Then, we will calculate the reconstruction error for the MNIST data.

The MNIST sample that closely resembles a digit value will have low reconstruction error. We will then sort them based on the reconstruction errors and then display the best samples and the worst samples (outliers) using the JFrame window. The autoencoder is constructed using a feed-forward network. Note that we are not performing any pretraining. We can process feature inputs in...

Technical requirements

The code for this chapter can be found here: https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/08_Performing_Anomaly_detection_on_unsupervised%20data/sourceCode/cookbook-app/src/main/java/MnistAnomalyDetectionExample.java.

The JFrame-specific implementation can be found here:
https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/08_Performing_Anomaly_detection_on_unsupervised%20data/sourceCode/cookbook-app/src/main/java/MnistAnomalyDetectionExample.java#L134.

After cloning our GitHub repository, navigate to the Java-Deep-Learning-Cookbook/08_Performing_Anomaly_detection_on_unsupervised data/sourceCode directory. Then, import the cookbook-app project as a Maven project by importing pom.xml.

Note that we use the MNIST dataset from here: http://yann.lecun.com/exdb/mnist/.

However, we don't have to download...

Extracting and preparing MNIST data

Unlike supervised image classification use cases, we will perform an anomaly detection task on the MNIST dataset. On top of that, we are using an unsupervised model, which means that we will not be using any type of label to perform the training process. To start the ETL process, we will extract this unsupervised MNIST data and prepare it so that it is usable for neural network training.

How to do it...

1. Create iterators for the MNIST data using MnistDataSetIterator:

DataSetIterator iter = new MnistDataSetIterator(miniBatchSize,numOfExamples,binarize);

2. Use SplitTestAndTrain to split the base iterator into train/test iterators:

DataSet ds = iter.next();
 SplitTestAndTrain split = ds...

Constructing dense layers for input

The core of the neural network design is the layer architecture. For autoencoders, we need to design dense layers that do encoding at the front and decoding at the other end. Basically, we are reconstructing the inputs in this way. Accordingly, we need to make our layer design.

Let's start configuring our autoencoder using the default settings and then proceed further by defining the necessary input layers for our autoencoder. Remember that the number of incoming connections to the neural network will be equal to the number of outgoing connections from the neural network.

How to do it...

1. Use MultiLayerConfiguration to construct the autoencoder network:

NeuralNetConfiguration.Builder...

Constructing output layers

As a final step, we need to decode the data back from the encoded state. Are we able to reconstruct the input just the way it is? If yes, then it's all good. Otherwise, we need to calculate an associated reconstruction error. Remember that the incoming connections to the output layer should be the same as the outgoing connections from the preceding layer.

How to do it...

1. Create an output layer using OutputLayer:

OutputLayer outputLayer = new OutputLayer.Builder().nIn(250).nOut(784)
 .lossFunction(LossFunctions.LossFunction.MSE)
 .build();

2. Add OutputLayer to the layer definitions:

builder.layer(new OutputLayer.Builder().nIn(250).nOut(784)
 .lossFunction(LossFunctions.LossFunction.MSE)
 .build...

Training with MNIST images

Once the layers are constructed and the neural network is formed, we can initiate the training session. During the training session, we reconstruct the input multiple times and evaluate the reconstruction error. In previous recipes, we completed the autoencoder network configuration by defining the input and output layers as required. Note that we are going to train the network with its own input features, not the labels. Since we use an autoencoder for anomaly detection, we encode the data and then decode it back to measure the reconstruction error. Based on that, we list the most probable anomalies in MNIST data.

How to do it...

1. Choose the correct training approach. Here is what is expected to...

Evaluating and sorting the results based on the anomaly score

We need to calculate the reconstruction error for all the feature sets. Based on that, we will find the outlier data for all the MNIST digits (0 to 9). Finally, we will display the outlier data in the JFrame window. We also need feature values from a test set for the evaluation. We also need label values from the test set, not for evaluation, but for mapping anomalies with labels. Then, we can plot outlier data against each label. The labels are only used for plotting outlier data in JFrame against respective labels. In this recipe, we evaluate the trained autoencoder model for MNIST anomaly detection, and then sort the results and display them.

How to do it...

...

Saving the resultant model

Model persistence is very important as it enables the reuse of neural network models without having to train more than once. Once the autoencoder is trained to perform outlier detection, we can save the model to the disk for later use. We explained the ModelSerializer class in a previous chapter. We use this to save the autoencoder model.

How to do it...

1. Use ModelSerializer to persist the model:

File modelFile = new File("model.zip");
 ModelSerializer.writeModel(multiLayerNetwork,file, saveUpdater);

2. Add a normalizer to the persisted model:

ModelSerializer.addNormalizerToModel(modelFile,dataNormalization);

...