Constructing an LSTM Network for Time Series

In this chapter, we will discuss how to construct a long short-term memory (LSTM) neural network to solve a medical time series problem. We will be using data from 4,000 intensive care unit (ICU) patients. Our goal is to predict the mortality of patients using a given set of generic and sequential features. We have six generic features, such as age, gender, and weight. Also, we have 37 sequential features, such as cholesterol level, temperature, pH, and glucose level. Each patient has multiple measurements recorded against these sequential features. The number of measurements taken from each patient differs. Furthermore, the time between measurements also differs among patients.

LSTM is well-suited to this type of problem due to the sequential nature of the data. We could also solve it using a regular recurrent neural network (RNN)...

Technical requirements

A concrete implementation of the use case discussed in this chapter can be found here: https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/06_Constructing_LSTM_Network_for_time_series/sourceCode/cookbookapp-lstm-time-series/src/main/java/LstmTimeSeriesExample.java.

After cloning the GitHub repository, navigate to the Java-Deep-Learning-Cookbook/06_Constructing_LSTM_Network_for_time_series/sourceCode directory. Then, import the cookbookapp-lstm-time-series project as a Maven project by importing pom.xml.

Download the clinical time series data from here: https://skymindacademy.blob.core.windows.net/physionet2012/physionet2012.tar.gz. The dataset is from the PhysioNet Cardiology Challenge 2012.

Unzip the package after the download. You should see the following directory structure:

The features are contained in a directory called sequence...

Extracting and reading clinical data

ETL (short for Extract, Transform, and Load) is the most important step in any deep learning problem. We're focusing on data extraction in this recipe, where we will discuss how to extract and process clinical time series data. We have learned about regular data types, such as normal CSV/text data and images, in previous chapters. Now, let's discuss how to deal with time series data. We will use clinical time series data to predict the mortality of patients.

How to do it...

1. Create an instance of NumberedFileInputSplit to club all feature files together:

new NumberedFileInputSplit(FEATURE_DIR+"/%d.csv",0,3199);

2. Create an instance of NumberedFileInputSplit to club...

Loading and transforming data

After the data extraction phase, we need to transform the data before loading it into a neural network. During data transformation, it is very important to ensure that any non-numeric fields in the dataset are transformed into numeric fields. The role of data transformation doesn't end there. We can also remove any noise in the data and adjust the values. In this recipe, we load the data into a dataset iterator and transform the data as required.

We extracted the time series data into record reader instances in the previous recipe. Now, let's create train/test iterators from them. We will also analyze the data and transform it if needed.

Getting ready

Before we proceed, refer to the...

Constructing input layers for the network

LSTM layers will have gated cells that are capable of capturing long-term dependencies, unlike regular RNN. Let's discuss how we can add a special LSTM layer in our network configuration. We can use a multilayer network or computation graph to create the model.

In this recipe, we will discuss how to create input layers for our LSTM neural network. In the following example, we will construct a computation graph and add custom layers to it.

How to do it...

1. Configure the neural network using ComputationGraph, as shown here:

ComputationGraphConfiguration.GraphBuilder builder = new NeuralNetConfiguration.Builder()
 .seed(RANDOM_SEED)
 .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT...

Constructing output layers for the network

The output layer design is the last step in configuring the neural network layer. Our aim is to implement a time series prediction model. We need to develop a time series classifier to predict patient mortality. The output layer design should reflect this purpose. In this recipe, we will discuss how to construct the output layer for our use case.

How to do it...

1. Design the output layer using RnnOutputLayer:

new RnnOutputLayer.Builder(LossFunctions.LossFunction.MCXENT)
 .activation(Activation.SOFTMAX)
 .nIn(LSTM_LAYER_SIZE).nOut(labelCount).build()

2. Use the addLayer() method to add an output layer to the network configuration:

builder.addLayer("predictMortality", new RnnOutputLayer...

Training time series data

So far, we have constructed network layers and parameters to define the model configuration. Now it's time to train the model and see the results. We can then check whether any of the previously-defined model configuration can be altered to obtain optimal results. Be sure to run the training instance multiple times before making any conclusions from the very first training session. We need to observe a consistent output to ensure stable performance.

In this recipe, we train our LSTM neural network against the loaded time series data.

How to do it...

1. Create the ComputationGraph model from the previously-created model configuration:

ComputationGraphConfiguration configuration = builder.build...

Evaluating the LSTM network's efficiency

After each training iteration, the network's efficiency is measured by evaluating the model against a set of evaluation metrics. We optimize the model further on upcoming training iterations based on the evaluation metrics. We use the test dataset for evaluation. Note that we are performing binary classification for the given use case. We predict the chances of that patient surviving. For classification problems, we can plot a Receiver Operating Characteristics (ROC) curve and calculate the Area Under The Curve (AUC) score to evaluate the model's performance. The AUC score ranges from 0 to 1. An AUC score of 0 represents 100% failed predictions and 1 represents 100% successful predictions.

How to do it...

...