Forecasting with PyTorch Lightning

In this chapter, we’ll build forecasting models using PyTorch Lightning. We’ll touch on several aspects of this framework, such as creating a data module to handle data preprocessing or creating a LightningModel structure that encapsulates the training process of neural networks. We’ll also explore TensorBoard to monitor the training process of neural networks. Then, we’ll describe a few metrics for evaluating deep neural networks for forecasting, such as Mean Absolute Scaled Error (MASE) and Symmetric Mean Absolute Percentage Error (SMAPE). In this chapter, we’ll focus on multivariate time series, which contain more than one variable.

This chapter will guide you through the following recipes:

• Preparing a multivariate time series for supervised learning
• Training a linear regression model for forecasting with a multivariate time series
• Feedforward neural networks for multivariate time series forecasting...

Technical requirements

In this chapter, we’ll leverage the following Python libraries, all of which you can install using pip:

• PyTorch Lightning (2.1.4)
• PyTorch Forecasting (1.0.0)
• torch (2.2.0)
• ray (2.9.2)
• numpy (1.26.3)
• pandas (2.1.4)
• scikit-learn (1.4.0)
• sktime (0.26.0)

The code for this chapter can be found in this book’s GitHub repository: https://github.com/PacktPublishing/Deep-Learning-for-Time-Series-Data-Cookbook.

Preparing a multivariate time series for supervised learning

The first recipe of this chapter addresses the problem of preparing a multivariate time series for supervised learning. We’ll show how the sliding window method we used in the previous chapter can be extended to solve this task. Then, we’ll demonstrate how to prepare a time series using TimeSeriesDataSet, a PyTorch Forecasting class that handles the preprocessing steps of time series.

Getting ready

We’ll use the same time series we analyzed in Chapter 1. We’ll need to load the dataset with pandas using the following code:

import pandas as pd
data = pd.read_csv('assets/daily_multivariate_timeseries.csv',
                   parse_dates=['Datetime'],
                   index_col=&apos...

Training a linear regression model for forecasting with a multivariate time series

In this recipe, we’ll use PyTorch to train a linear regression model as our first forecasting model fit on a multivariate time series. We’ll show you how to use TimeSeriesDataSet to handle the preprocessing steps for training the model and passing data to it.

Getting ready

We’ll start this recipe with the mvtseries dataset that we used in the previous recipe:

import pandas as pd
mvtseries = pd.read_csv('assets/daily_multivariate_timeseries.csv',
            parse_dates=['datetime'],
            index_col='datetime')

Now, let’s see how we can use this dataset to train a PyTorch model.

How to do it…

In the following code, we’ll describe the necessary steps to prepare the time series and build a linear...

Feedforward neural networks for multivariate time series forecasting

In this recipe, we’ll return our attention to deep neural networks. We’ll show you how to build a forecasting model for multivariate time series using a deep feedforward neural network. We’ll describe how to couple the DataModule class with TimeSeriesDataSet to encapsulate the data preprocessing steps. We’ll also place the PyTorch models within a LightningModule structure, which standardizes the training process of neural networks.

Getting ready

We’ll continue to use the multivariate time series related to solar radiation forecasting:

import pandas as pd
mvtseries = pd.read_csv('assets/daily_multivariate_timeseries.csv',
                        parse_dates=['datetime'],
           ...

LSTM neural networks for multivariate time series forecasting

In this recipe, we’ll continue the process of building a model to predict the next value of solar radiation using multivariate time series. This time, we’ll train an LSTM recurrent neural network to solve this task.

Getting ready

The data setup is similar to what we did in the previous recipe. So, we’ll use the same data module we defined there. Now, let’s learn how to build an LSTM neural network with a LightningModule class.

How to do it…

The workflow for training an LSTM neural network with PyTorch Lightning is similar, with one small but important detail. For LSTM models, we keep the input data in a three-dimensional structure with a shape of (number of samples, number of lags, number of features). Here’s what the module looks like, starting with the constructor and the forward() method:

class MultivariateLSTM(pl.LightningModule):
    def __init__...

Monitoring the training process using Tensorboard

Training deep learning models often involves tuning numerous hyperparameters, assessing different architectures, and more. To facilitate these tasks, visualization and monitoring tools are essential. tensorboard is a powerful tool for tracking and visualizing various metrics during the training process. In this section, we will guide you through integrating tensorboard with PyTorch Lightning for monitoring the training process.

Getting ready

Before using tensorboard with PyTorch Lightning, you’ll need to have tensorboard installed. You can install it using the following command:

pip install -U tensorboard

Once installed, make sure that you are utilizing PyTorch Lightning’s built-in tensorboard logging capabilities.

How to do it…

Here’s how to use tensorboard to monitor the training process:

1. First, ensure that tensorboard is imported into your script.
2. Next, you’ll need to...

Evaluating deep neural networks for forecasting

Evaluating the performance of forecasting models is essential to understand how well they generalize to unseen data. Popular metrics include the Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), Mean Absolute Scaled Error (MASE), and Symmetric Mean Absolute Percentage Error (SMAPE), among others. We will implement these metrics in Python and show you how they can be applied to evaluate our model’s performance.

Getting ready

We need predictions from our trained model and the corresponding ground truth values to calculate these metrics. Therefore, we must run our model on the test set first to obtain the predictions.

To simplify the implementation, we will use the scikit-learn and sktime libraries since they have useful classes and methods to help us with this task. Since we have not installed sktime yet, let’s run the following command:

pip install sktime

Now, it is time to import the classes...

Using callbacks – EarlyStopping

Callbacks in PyTorch Lightning are reusable components that allow you to inject custom behavior into various stages of the training, validation, and testing loops. They offer a way to encapsulate functionalities separate from the main training logic, providing a modular and extensible approach to manage auxiliary tasks such as logging metrics, saving checkpoints, early stopping, and more.

By defining a custom class that inherits from PyTorch Lightning’s base Callback class, you can override specific methods corresponding to different points in the training process, such as on_epoch_start or on_batch_end. When a trainer is initialized with one or more of these callback objects, the defined behavior is automatically executed at the corresponding stage of the training process. This makes callbacks powerful tools for organizing the training pipeline, adding flexibility without cluttering the main training code.

Getting ready

After...