3. Real-World Deep Learning: Evaluating the Bitcoin Model

Overview

This chapter focuses on how to evaluate a neural network model. We'll modify the network's hyperparameters to improve its performance. However, before altering any parameters, we need to measure how the model performs. By the end of this chapter, you will be able to evaluate a model using different functions and techniques. You will also learn about hypermeter optimization by implementing functions and regularization strategies.

Introduction

In the previous chapter, you trained your model. But how will you check its performance and whether it is performing well or not? Let's find out by evaluating a model. In machine learning, it is common to define two distinct terms: parameter and hyperparameter. Parameters are properties that affect how a model makes predictions from data, say from a particular dataset. Hyperparameters refer to how a model learns from data. Parameters can be learned from the data and modified dynamically. Hyperparameters, on the other hand, are higher-level properties defined before the training begins and are not typically learned from data. In this chapter, you will learn about these factors in detail and understand how to use them with different evaluation techniques to improve the performance of a model.

Note

For a more detailed overview of machine learning, refer to Python Machine Learning, Sebastian Raschka and Vahid Mirjalili, Packt Publishing, 2017).

Problem Categories...

Hyperparameter Optimization

So far, we have trained a neural network to predict the next 7 days of Bitcoin prices using the preceding 76 weeks of prices. On average, this model issues predictions that are about 8.4 percent distant from real Bitcoin prices.

This section describes common strategies for improving the performance of neural network models:

• Adding or removing layers and changing the number of nodes
• Increasing or decreasing the number of training epochs
• Experimenting with different activation functions
• Using different regularization strategies

We will evaluate each modification using the same active learning environment we developed by the end of the Model Evaluation section, measuring how each one of these strategies may help us develop a more precise model.

Layers and Nodes – Adding More Layers

Neural networks with single hidden layers can perform fairly well on many problems. Our first Bitcoin model (bitcoin_lstm_v0) is a good...

Summary

In this chapter, we learned how to evaluate our model using the MSE, RMSE, and MAPE metrics. We computed the latter two metrics in a series of 19-week predictions made by our first neural network model. By doing this, we learned that it was performing well.

We also learned how to optimize a model. We looked at optimization techniques, which are typically used to increase the performance of neural networks. Also, we implemented a number of these techniques and created a few more models to predict Bitcoin prices with different error rates.

In the next chapter, we will be turning our model into a web application that does two things: retrains our model periodically with new data and is able to make predictions using an HTTP API interface.