ARIMA Models

In this chapter, we will discuss univariate time series models. These are models that only consider a single variable and create forecasts based only on the previous samples in the time series. We will start by looking at models for stationary time series data and then progress to models for non-stationary time series data. We will also discuss how to identify appropriate models based on the characteristics of time series. This will provide a powerful set of models for forecasting time series.

In this chapter, we’re going to cover the following main topics:

• Models for stationary time series
• Models for non-stationary time series
• More on model evaluation

Technical requirements

In this chapter, we use two additional Python libraries for time series analysis: sktime and pmdarima. Please install the following versions of these libraries to run the provided code. Instructions for installing libraries can be found in Chapter 1, Sampling and Generalization.

• sktime==0.15.0
• pmdarima==2.02

More information about sktime can be found at this link: https://www.sktime.org/en/stable/get_started.html

More information about pmdarima can be found at this link: http://alkaline-ml.com/pmdarima/

Models for stationary time series

In this section, we will discuss Autoregressive (AR), Moving Average (MA), and Autoregressive Moving Average (ARMA) models that are useful for stationary data. These models are useful when modeling patterns and variance around process means that output over time. When we have data that does not exhibit autocorrelation, we can use statistical and machine learning models that do not make assumptions about time, such as Logistic Regression or Naïve Bayes, so long as the data supports such use cases.

Autoregressive (AR) models

The AR(p) model

In Chapter 10, Introduction to Time Series we considered how the Partial Auto-Correlation Function (PACF) correlates one data point to another lag, controlling for those lags between. We also discussed how inspection of the PACF plot is a frequently used method for assessing the ordering of an autoregressive model. Thereto, the autoregressive model is one that considers specific points in the past to...

Models for non-stationary time series

In the previous section, we discussed ARMA models for stationary time series data. In this section, we will look at non-stationary time series data and extend our model to work with non-stationary data. Let us start by taking a look at some sample data (shown in Figure 11.17). There are two series: US GDP (left) and airline passenger volume (right).

Figure 11.17 – US GDP (left) and airline passenger (right) time series

The US GDP series appears to exhibit an upward trend with some variations in the series. The airline passenger volume series also exhibits an upward trend, but there also appears to be a repeated pattern in the series. The repeated pattern in the airline series is called seasonality. Both series are non-stationary because of the apparent trend. Additionally, the airline passenger volume series appears to exhibit non-constant variance. We will model the GDP series with ARIMA, and we will model...

Seasonal ARIMA models

Let’s look at another characteristic of time series called seasonality. Seasonality is the presence of a pattern in a time series that repeats at regular intervals. Seasonal time series are common in nature. For example, yearly weather patterns and daily sunshine patterns are seasonal patterns. Back at the start of the non-stationary section, we showed an example of a non-stationary time series with seasonality. This time series is shown again in Figure 11.22 along with its ACF plot.

Figure 11.22 – Airline volume data and ACF plot

The time series shown in Figure 11.22 is the monthly total of international airline passengers from 1949 to 1960 [3]. There is a definite repeated pattern in this time series. To model this type of data, we will need to an additional term to the ARIMA model to account for seasonality.

Seasonal differencing

We will use a similar approach for modeling this type of time series as we did...

More on model evaluation

In the previous sections, we discussed other methods to prepare data, test and validate models. In this section, we will discuss how to validate time series models and introduce several methods for validating time series models. We will cover the following methods for model evaluation: resampling, shifting, optimized persistence forecasting, and rolling window forecasting.

The real-world dataset considered in this section is Coca Cola stock data collected from Yahoo Finance databases from 01/19/1962 to 12/19/2021 for stock price prediction. This is a time series analysis to forecast the future stock value of a given stock. The reader can download the dataset from the Kaggle platform for this analysis. To motivate the study, we first go to explore the Coco Cola stock dataset:

data = pd.read_csv("COCO COLA.csv", parse_dates=["Date"], index_col="Date")

Figure 11.26 – Coco Cola dataset

The...

Summary

In this chapter, we discussed various methods for modeling univariate time series data from stationary time series models such as ARMA to non-stationary models such as ARIMA. We started with stationary models and discussed how to identify modeling approaches based on the characteristics of time series. Then we built on the stationary models by adding a term in the model to stationarize time series. Finally, we talked about seasonality and how to account for seasonality in an ARIMA model. While these methods are powerful for forecasting, they do not incorporate potential information from other external variables. As in the previous chapter, we will see that external variables can help improve forecasts. In the next chapter, we will look at multivariate methods for time series data to take advantage of other explanatory variables.

References

Please refer to the final word file for how the references should look.

1. APPLIED TIME SERIES ANALYSIS WITH R, W. Woodward, H. Gray, A. Elliott. Taylor & Francis Group, LLC. 2017.
2. Box, G. E. P., Jenkins, G. M. and Reinsel, G. C. (1976) Time Series Analysis, Forecasting and Control. Third Edition. Holden-Day. Series G.
3. Brownlee, J, (2017) Simple Time Series Forecasting Models to Test So That You Don’t Fool Yourself (https://machinelearningmastery.com/simple-time-series-forecasting-models/).