What is a time series?

A time series is a sequence of data points, typically measured at successive times, spaced at uniform time intervals. Time Series mostly come as discrete-time, where the time difference between each point is the same.Many disciplines, such as finance, public administration, energy, retail, and healthcare, are dominated by time series data. Large areas of micro- and macroeconomics rely on applied statistics with an emphasis on time series analyses and modeling.The following are examples of time series data:

• Daily closing values of a stock index
• Number of weekly infections of a disease
• Weekly series of train accidents
• Rainfall per day
• Sensor data such as temperature measurements per hour
• Population growth per year
• Quarterly earnings of a company over a number of years
• The number of views or visitors to a website over time

This is only to name but a few. Loosely speaking, any data that deals with changes over time is a time series. Since this is a book about time series data, it’s worth defining more formally what is considered a time series.Definition: Time Series are datasets where observations are arranged in chronological order.This is a very broad definition. Alternatively, we could have said that a time series is a sequence of data points taken sequentially over time, or that a time series is the result of a stochastic process.Formally, we can define a time series in two ways. The first one is as a mapping from the time domain to the domain of real numbers:

Where and .Another way to define a time series is as a stochastic process:

Here, X(t) or X_t denotes the value of the random variable X at time point t.If T is a set of real numbers, it's a continuous-time stochastic process. If T is a set of integers, we call it a stochastic process in discrete time. The convention in the latter case is to write {X_n}.Since time is the primary index of the dataset, by implication, time series datasets describe how the world changes over time. They often deal with the question of how the past influences the presence or future.The increase of monitoring and data collection brings with it the need for both statistical and machine learning techniques applied to time series to predict and characterize the behavior of complex systems or components within a system. An important part of working with time series is the question of how the future can be predicted based on the past. This is called forecasting.Some methods allow adding business cycles or other descriptors as additional features. These additional features are called exogenous features - they are time-dependent, explanatory variables. We'll go through examples of feature generation in Chapter 4, Machine Learning for Time Series.

Characteristics of time series

There are many different types of time series data, such as financial data, economic data, weather data, and social media data. Each type of data has its own challenges and requires its own methods for time series analysis and forecasting.Financial data is often non-stationary, meaning that the statistical properties of the data change over time. This makes it difficult to build models that can accurately predict future values. Economic data is often seasonal, meaning that there is a regular pattern of ups and downs over the course of a year. This can be difficult to model, as the seasonality may be caused by factors that are not easily observed or measured. Weather data is often chaotic, meaning that it is very difficult to predict even short-term future values. Social media data is often volatile, meaning that it can change rapidly and unexpectedly. This makes it difficult to build models that can accurately predict future values.One of the most common examples of time series data is stock prices. Stock prices are constantly changing, and they can be used to describe how a company is doing over time. Another example of time series data is weather data. Weather data can be used to describe changes in the weather over time, and it can be used to predict future weather patterns. A time series can be univariate, where there's one value against time, for example temperatures over time in London, United Kingdom. Such a time series can be represented using a line chart, where the x-axis is the time and the y-axis is the value of the time series. Time series can be multivariate as well. In this case, there are more than one variables that vary with time.Here's an extract of a time series dataset as an example, exported from Google Trends, on searches for Python, R, and Julia:

This is a multivariate time series, with columns for Python, R, and Julia. The first column is the index, a date column, and its period is the month. In cases, where we have only a single variable, we speak of a univariate series. This dataset would be univariate if we had only one programming language instead of three.For many applications, it is useful to split a time series into its component parts, such as trend, seasonality, and noise.A trend is the general direction in which something is developing or changing, such as a long-term increase or decrease in a sequence. An example of where a trend can be observed would be global warming, the process by which the temperatures on our planet have been rising over the last half-century.Here's a plot of global surface temperature changes over the last 100 years from the GISS Surface Temperature Analysis dataset released by NASA:

As you can see in Figure 1.2, temperature changes have been varying around 0 until the mid-20^th century; however, since then, there's been a clearly visible trend of an overall rise in the yearly temperature.Seasonality is a variation that occurs at specific regular intervals of less than a year. Seasonality can occur on different time spans, such as daily, weekly, monthly, or yearly. An example of weekly seasonality would be sales of ice cream picking up each weekend. Also, depending on where you live, ice cream might only be sold in spring and summer. This is a yearly variation.Other than seasonal changes and trends, there is variability that's not of a fixed frequency or that rises and falls in a way that's not based on seasonal frequency. Some of these we might be able to explain based on the knowledge we have.As an example of cyclic variability that's irregular, bank holidays can fall on different calendar days each year, and promotional campaigns could depend on business decisions, such as the introduction of a new product.In electroencephalography (EEG), the electrical activity of the brain is recorded through electrodes placed on the scalp. Its signal typically shows strong oscillations (also referred to as brain waves) at a variety of frequency ranges. Here's a graph of an EEG signal (from the EEG Eye State dataset uploaded by Oliver Roesler from DHBW, Germany):

Such changes at the scale of seconds or milliseconds would not be called seasonal effects. Other changes that take place over time periods longer than a year would not be called seasonal either.The task of identifying, quantifying, and decomposing these and other characteristics is called time series analysis. Exploratory time series analysis is often the first step before any feature transformation and machine learning.

Which sets of techniques are there for time series?

There are many different techniques that can be used for each of these problem scenarios. In this book, we'll discuss the most popular methods for analyzing time series data, including traditional statistical methods, machine learning methods, and deep learning methods.Problems such as clustering, and classification/regression are generic to machine learning, signal processing, or statistical methods. Often, being aware of this can help establish a context for problem solving and allows one to draw from the broader set of methods within these areas.How is machine learning for time series similar or different from other ML disciplines? There are quite a few challenges specific to machine-learning with time series:

• The main difference to - more generally - machine learning methods on tabular data is that the data is that the data are indexed by time. The temporal ordering of the data points is often important for the task at hand. This means that the data must be processed in a specific order and cannot simply be shuffled as is often done in machine learning on tabular data.
• Time series data is often non-stationary, meaning that the statistical properties of the data change over time. This makes it difficult to build models that can generalize from the training data to the test data.

Machine learning has evolved as a powerful method for understanding hidden complexity in time series data. Recently, probabilistic models for time series such as Facebook Prophet, Markov models, or even fuzzy models, and counter-factual causal models such as Bayesian structural time series models as proposed by Google have gained in popularity. At the same time, multivariate forecasting has found practical use in multivariate multistep forecasts of energy demand with deep learning models. Finally, time series techniques such as bandit algorithms and Deep Q-Learning have found their applications in recommender systems and trading algorithms.Some models are adaptive or robust to certain kinds of drift. For example, ARIMA models are generally robust to small amounts of additive and multiplicative drift. The Random Forest algorithm, on the other hand, can be more robust under these conditions and does not require a fundamental update of the algorithm for each instance. Adaptive algorithms include effective resampling methods and adaptive operators that can cope with different types of concept drifts without complex optimizations for different data sets. Unfortunately, there are no hard and fast rules and you should experiment with both types of methods to see which works best for your data.The most prominent time series data technique is forecasting - the task to predict future values of the series. This can be done using methods such as ARIMA or many others, each coming with their own set of assumptions, drawbacks, and advantages.

What are the technical problems associated with time series?

The three main machine learning problems with time series are forecasting, classification, regression, and anomaly detection. In this section, we'll provide a broad overview of applications of time series and a history of machine learning and analysis techniques applied to time series.Common problem scenarios when dealing with time series data include:

• Forecasting: The use of a model to predict future values based on previously observed values. This technique is used to predict future values of the time series based on past values. This can be done using methods such as ARIMA or exponential smoothing.
• Decomposition: This technique is used to separate the data into its constituent parts such as trend, seasonality, and residuals. This can be done using methods such as additive or multiplicative decomposition. This involves trend analysis, where we identify long-term trends in a time series, and seasonality analysis, which is about identifying repeating patterns in a time series.
• Classification/Regression: This technique is used to predict a target variable based on the time series data. This can be done using methods such as support vector machines or logistic regression.
• Anomaly detection: Identifying unusual patterns (outliers) in a time series. Outliers are unusual observations that fall outside of the typical pattern.
• Clustering: This technique is used to group together similar time series. This can be done using methods such as k-means clustering.
• Drift detection: This technique is used to identify systematic changes in the time series data. This can be done using methods such as the Mann-Kendall test.
• Smoothing: This technique is used to remove the noise from the data. This can be done using a simple moving average or a more sophisticated technique such as exponential smoothing.

An example for forecasting is ARIMA modeling, where we are using autoregressive and moving average models to forecast future values in a time series. Which technique you use will depend on the nature of your data and the problem you are trying to solve.Multivariate forecasting is a natural extension of the univariate case, where the goal is to predict multiple variables simultaneously. For example, in a company, one might want to forecast sales, price, inventory, and production simultaneously. This can be done with a vector autoregression model, where each variable is predicted as a function of its own past values and the past values of the other variables. Multistep forecasting is a generalization of the univariate case, where the goal is to predict multiple steps into the future. For example, in a company, one might want to forecast sales for the next 6 months. This can be done with a vector autoregression model, where each variable is predicted as a function of its own past values and the past values of the other variables.

How can we approach time series problems?

Given a time series problem, there is no one-size-fits-all approach - as it depends on the specific problem that you are trying to solve. In fact, there are many different approaches and ways to solve problems. The choice of method will depend on the type of data, the nature of the problem, and the desired outcome. However, there is a set of problem solving techniques that can help guide you approach any time series problem.Time series analysis (TSA) comprises methods for analyzing time series data in order to extract meaningful statistics and other characteristics of the data. Importing the data can be considered prior to time series analysis, and data cleaning, feature engineering, and training a machine learning model are not strictly part of time series analysis. We'll discuss TSA in chapter 2, Time-Series Analysis.The steps belonging to TSA and leading to preprocessing (feature engineering) and machine learning are highly iterative, and can be visually appreciated in the following time series machine learning flywheel:

This flywheel emphasizes the iterative nature of the work. For example, data cleaning comes often after loading the data, but will come up again after we've made another discovery about our variables. I've highlighted TSA in dark, while steps that are not strictly part of TSA are grayed out.In summary, we have these steps:

1. Define the problem: Is it a classification or regression problem? What are the inputs and outputs? What are the steps involved in solving the problem?
2. Data Exploration: This step involves visualizing the data to understand the trends and patterns. This helps in developing the intuition for building the machine learning model.
3. Preprocess the data: This step involves cleaning the data, dealing with missing values, and transforming the data so that it can be used by the machine learning algorithm. This step is crucial and includes feature engineering, scaling, normalization.
4. Decompose the time series: This step is useful in understanding the trend, seasonality, and noise in the data. This step is necessary for certain kinds of methods, but not with others.
5. Build models: Once the data is preprocessed, you can build models using traditional machine learning algorithms or time series specific algorithms.
6. Evaluate the models: This step includes comparing the performance of different models and choosing the best one. It involves assessing the performance of the machine learning model on unseen data. This helps in fine-tuning the model and making it ready for deployment.
7. Make predictions: This step involves using the final model to make predictions on new data.

There are many ways to deal with time series. Let's briefly go through the main categories of approaches, traditional time series methods, machine learning, and deep learning. All of them can be used for problems such as forecasting, classification and regression.Traditional time series methods are the most common approach to dealing with time series data. They involve using statistical methods to model the data and make predictions. They are simple and easy to interpret. A downside is that many of them assume that the data is stationary.Machine Learning methods can be applied to data that is not stationary. The downside is that applications can be more complex and results more difficult to understand.Deep learning is a branch of machine learning that uses neural networks to learn from data. These neural networks often have many parameters giving the model the potential to capture a wide range of phenomena, but require a lot of data to tune, and often predictions can be highly variable.To summarize: the best approach depends on the data, the problem, and the resources available.

Historical Development of Time Series Applications

An early work in time series analysis was The Art of Statistics, published in 1833 by Adolphe Quetelet. The first general work on time series analysis is generally credited to Francis Galton and his work Natural Inheritance, published in 1889. A significant contribution to the development of time series analysis was made by the statistician and biologist Ronald Fisher in his work Statistical Methods for Research Workers, published in 1925.Classical time series modeling approaches were introduced by George Box and Gwilym Jenkins in 1970 in their book "Time-Series Analysis Forecasting and Control." Most importantly, they formalized the ARIMA and ARMAX models and described how to apply them to time series forecasting. We'll talk about these types of models in Chapter 5, Forecasting with Moving Averages and Autoregressive Models.The study of time series in different scientific disciplines was foundational to fields such as demography, astronomy, medicine, and economics. We won't cover all these fields, but just give a few highlights, especially from astronomy and meteorology, which are particularly interesting cases for their development.

Meteorology

The modern science of meteorology has its roots in ancient times as well. The Babylonians are thought to have had some rudimentary knowledge about atmospheric processes, and they are known to have observed and recorded the regular ebb and flow of the tides.The Greek philosopher Aristotle was one of the first to write about weather and its measurement; however, it took much longer for the first weather predictions to be made. Vice Admiral Robert FitzRoy founded the United Kingdom's national weather service, the Meteorological Office, in 1854. FitzRoy had already reserved his place in history as the captain of the HMS Beagle, the ship that carried a recently graduated naturalist by the name of Charles Darwin around the world. It was FitzRoy, in fact, who coined the word forecast, although at the time, many contemporaries referred to them as "quack weather prognostications".The first computerized weather models were programmed on the Electronic Numerical Integrator and Computer (ENIAC). The ENIAC, designed by John Mauchly and J. Presper Eckert, could run arbitrary sequences of operations; however, it didn't read the programs from tapes but from plugboard switches. The giant 15x9-meter machine is exhibited today at the Smithsonian Institute in Washington, D.C. Consisting of 17,500 vacuum tubes, it first produced calculations for the construction of a hydrogen bomb and was then exploited to extend forecasting past one or two days using new methods of numerical weather prediction. The computer was programmed by Klara von Neumann.Here's a photo of the ENIAC (source: Wikimedia Commons):

You can see Betty Snyder, one of the earliest programmers of the ENIAC, standing in front of the ENIAC.Later, Joseph Smagorinsky and Douglas Lilly developed a mathematical model for turbulence used in computational fluid dynamics. This model, the Smagorinsky-Lilly model, which is still in use today, used data about the wind, cloud cover, precipitation, atmospheric pressure, and radiation emanating from the earth and sun as input. Smagorinsky continued to lead research on global warming, investigating the climate's sensitivity to increasing carbon dioxide levels.The introduction of mobile sensor arrays and computerized models has greatly increased the accuracy of weather predictions. Valuable temperature and wind data is collected by sensors deployed by meteorology offices or other sources, most importantly by commercial aircraft as they fly. Today, within a seven-day window, a forecast is accurate about 80% of the time. The grounding of commercial flights during the COVID pandemic, where there were about 75% fewer flights for some periods, has led to less accurate forecasts recently.DeepMind, a Google subsidiary, is working on artificial intelligence (AI) models to predict the weather. The technology could improve the accuracy of predictions by up to 50%.DeepMind’s AI model uses a neural network architecture called a recurrent neural network (RNN). It takes advantage of the fact that the weather depends on past weather patterns. A deep learning algorithm is trained on historical data to predict the weather. DeepMind’s AI model is able to predict weather patterns over a five-day period. The neural network is trained on 30 years of weather data from the United States’ Global Forecast System (GFS) weather model.

Summary

In this chapter, we’ve started with a definition of time series and its main properties. We gave examples for time-series and their basic characteristics. Further, we went through different kinds of problems such as forecasting, regression, classification, anomaly detection, and other types of analyses relevant to machine learning and statistics. Finally, we’ve tried to give a broad overview of applications of time series and a history of time series applications in astronomy and meteorology.In the next chapter, we'll look at time series analysis with Python.