Probabilistic Models for Time-Series

Probability is a measure of how likely something is to occur. In sales forecasting, an estimate of uncertainty is crucial because these forecasts, providing insights into cash flow, margin, and revenue, drive business decisions on which depend the financial stability and the livelihoods of employees. This is where probabilistic models for time-series come in. They help us make decisions when an estimate of certainty is important.

In this chapter, I'll introduce Prophet, Markov models, and Fuzzy time-series models. At the end, we'll go through an applied exercise with these methods.

Another application for probabilistic modeling is estimating counterfactuals, where we can estimate treatment effects in experiments. We'll discuss the concept of Bayesian Structural Time-Series Models, and we'll run through a practical example with a time-series in the practice section.

We're going to cover the following topics...

Probabilistic Models for Time-Series

As mentioned in the introduction, probabilistic models can help us make decisions under uncertainty, and in situations where estimates have to come with quantified confidence, such as in financial forecasting, this can be crucial. For predictions of sales or cash flow, attaching probabilities to model predictions can make it easier for financial controllers and managers to act on the new information.

Some well-known algorithms include Prophet, explicitly designed for monitoring operational metrics and key performance indicators (KPIs), and Markov models. Others are stochastic deep learning models such as DeepAR and DeepState. Since we are dealing with deep learning models in Chapter 10, Deep Learning Models, we'll not deal with them in detail in this chapter.

The Prophet model comes from Facebook (Taylor and Letham, 2017) and is based on a decomposable model with interpretable parameters. A guiding design principle was that...

Prophet

Facebook's Prophet is both a Python/R library and the algorithm that comes with it. The algorithm was published in 2017 ("Forecasting at Scale" by Sean Taylor and Benjamin Letham). The authors write that the problems of forecasting and anomaly detection in practice involve the complexity of handling a variety of idiosyncratic forecasting problems at Facebook with piecewise trends, multiple seasonalities, and floating holidays, and building trust across the organization in these forecasts.

With these goals in mind, Prophet was designed to be scalable to many time-series, flexible enough for a wide range of business-relevant, possibly idiosyncratic time-series, and at the same time intuitive enough to be configurable by domain experts who might have little knowledge of time-series methods.

The Prophet algorithm is similar to the Generalized Additive Model (GAM) and formalizes the relationship between the forecast for the three model components, trend...

Markov Models

A Markov chain is a probabilistic model describing a sequence of possible events that satisfies the Markov property.

Markov property: In a sequence or stochastic process that possesses the Markov property, the probability of each event depends only on the immediately preceding state (rather than earlier states). These sequences or processes can also be called Markovian, or a Markov Process.

Named after Russian mathematician Andrey Markov, the Markov property is very desirable since it significantly reduces the complexity of a problem. In forecasting, instead of taking into account all previous states, t-1, t-2, …, 0, only t-1 is considered.

Similarly, the Markov assumption, for a mathematical or machine learning model is that the sequence satisfies the Markov property. In models such as the Markov chain and Hidden Markov model, the process or sequence is assumed to be a Markov process.

In a discrete-time Markov chain (DTMC), the...

Fuzzy Modeling

Fuzzy logic and fuzzy set theory were developed by Lotfi Zadeh in the 1960s and 70s while a professor at the University of California, Berkeley. Born to Persian and Jewish Russian parents in Baku, Azerbaijan, he completed his schooling in Tehran, Iran, and later moved to the USA, where he studied at MIT and Columbia. As a result, he was familiar with how concepts are understood in different cultures and expressed in different languages. This inspired his research approach to approximate reasoning and linguistic variables that he formalized as fuzzy theory.

Fuzzy set theory is an approach that can deal with problems relating to ambiguous, subjective, and imprecise judgments. Vagueness is inherent in everyday language, and fuzziness was invented to express this and work with it in an intuitive manner. Fuzzy logic expresses subjective belief and vagueness. It can and has been claimed that probability theory is a subset of fuzzy logic.

Fuzzy sets are sets whose...

Bayesian Structural Time-Series Models

In causal inference, we want to analyze the effect of a treatment. The treatment can be any action that interacts with the system or environment that we care about, from changing the colors of a button on a website to the release of a product. We have the choice of taking the action (for example, releasing the product), thereby observing the outcome under treatment, or not taking the action, where we observe the outcome under no treatment. This is illustrated in the diagram here:

Figure 9.3: Causal effect of a treatment

In the diagram, an action is taken or not (medicine is administered to a patient), and depending on whether the action is taken we see the patient recovering (cycling) or going into intensive care.

A causal effect is the difference between what happens under treatment and what happens under no treatment. The problem with this is that we can't observe both potential outcomes at the same time.

However,...

Python Exercise

Let's put into practice what we've learned in this chapter so far. We'll be doing a model in Prophet, a Markov Switching model, a Fuzzy time-series model, and a BSTS model.

Let's get started with Prophet!

Prophet

First, let's make sure we have everything installed that we need. Let's quickly install the required libraries. We can do this from the terminal (or similarly from the Anaconda navigator):

pip install -U pandas-datareader plotly

You'll need a recent version of pandas-datareader, otherwise you might get a RemoteDataError.

We'll use the Prophet model through Facebook's Prophet library. Let's install it:

pip install prophet

Once this is done, we are set to go.

In this example, we'll use the daily Yahoo closing stock values in this chapter that we used in several examples in Chapter 7, Machine Learning Models for Time-Series.

To recap, we can download the daily Yahoo...

Summary

In this chapter, we've discussed how probabilistic models for time-series can help us make decisions with an estimate of uncertainty in the context of financial forecasting. These forecasts drive business decisions for financial planning.

I've introduced Prophet, Markov models, and Fuzzy time-series models. We've discussed the components of Facebook's Prophet model. For Markov models, we've discussed the main ideas, such as the Markov property, and we've discussed more details regarding Switching Models. Then I've explained some basics of fuzzy set theory and how this is applied to time-series.

Finally, we've delved into the intuition and some of the theory of BSTS models in the context of estimating treatment effects in experiments.

Finally, we went through an applied exercise with each method. In the BSTS practice, we've looked at the effect of the Volkswagen emissions scandal.