Training an RNN from scratch

Recurrent neural networks (RNNs) are a group of neural networks that are built to model sequential data. In the last few chapters, we looked at using convolutional layers to learn features from images. Recurrent layers are equally as useful when we want to learn features from a sequence of values that are all related: x_t, x_t-1, x_t-2, x_t-3.

In this chapter, we will talk about how to use RNNs for time series problems, which are unsurprisingly problems involving a sequence of data points placed in temporal or chronological order.

We will cover the following topics in this chapter:

• Introducing recurrent neural networks
• Time series problems
• Using an LSTM for time series prediction

Introducing recurrent neural networks

In case the definition is unclear, let's look at an example: a stock market ticker where we might observe the price of a stock changing over time, such as Alphabet Inc. in the following screenshot, which is an example of time series:

In the next chapter, we will talk about using recurrent neural networks to model language, which is another type of sequence, a sequence of words. Since you're reading this book, you undoubtedly have some intuition on language sequences already.

If you're new to time series, you might be wondering if it would be possible to use a normal multilayer perceptron to solve a time series problem. You most certainly could do that; however, practically, you almost always get better results using recurrent networks. That said, recurrent neural networks have two other advantages for modeling sequences:

• They...

A refresher on time series problems

Time series problems are problems involving a sequence of data points placed in temporal order. We most often represent those data points as a set:

Usually our goal in time series analysis is forecasting; however, there are certainly many other interesting things you can do with a time series that are outside the scope of this book. Forecasting is really just a specialized form of regression, where our goal is to predict some point x_t or points , given some set of previous points . We can do this when the time series is auto correlated, which means the data points are correlated with themselves one or more points back in time (which are called lags). The stronger the auto correlation, the easier it is to forecast.

In many books, time series problems are denoted with y, rather than x, as a hint towards the idea that we typically care to predict...

Using an LSTM for time series prediction

In this chapter, we're going to predict the minute-to-minute value of bitcoin in US dollars during the month of June 2017 by using the minute-to-minute price of bitcoin from January to May of 2017. I know this sounds really lucrative but before you buy that boat, I recommend reading through to the end of the chapter; this is something easier said and even easier modeled, than done.

Even if we were able to create the potential for an arbitrage (a difference in price between two markets due to an inefficiency) between USD and bitcoins using some model like this one, developing a trading strategy around bitcoin can be extremely complex because of the delay in finalizing bitcoin transactions. At the time of this writing, the average transaction time for a bitcoin transaction is over an hour! This "illiquidity" should be a consideration...

Summary

In this chapter, we talked about using recurrent neural networks to predict the next element in a sequence. We covered both RNNs in general and LSTMs specifically, and we focused on using LSTMs to predict a time series. In order to make sure we understood the benefits and challenges of using LSTMs for time series, we briefly reviewed some basics of time series analysis. We spent a few minutes talking about traditional time series models as well, including ARIMA and ARIMAX.

Lastly, we walked through a challenging use case where we used an LSTM to predict the price of a bitcoin.

In the next chapter, we will continue to use RNNs, now focusing on natural language processing tasks and introducing the concept of embedding layers.