Overview

In this chapter, we will begin creating Artificial Neural Networks (ANNs) using the Keras library. Before utilizing the library for modeling, we will get an introduction to the mathematics that comprise ANNs—understanding linear transformations and how they can be applied in Python. You'll build a firm grasp of the mathematics that make up ANNs. By the end of this chapter, we will have applied that knowledge by building a logistic regression model with Keras.

In the previous chapter, we discussed some applications of machine learning and even built models with the scikit-learn Python package. The previous chapter covered how to preprocess real-world datasets so that they can be used for modeling. To do this, we converted all the variables into numerical data types and converted categorical variables into dummy variables. We used the logistic regression algorithm to classify users of a website by their purchase intention from the online shoppers purchasing intention dataset. We advanced our model-building skills by adding regularization to the dataset to improve the performance of our models.

In this chapter, we will continue learning how to build machine learning models and extend our knowledge so that we can build an Artificial Neural Network (ANN) with the Keras package. (Remember that ANNs represent a large class of machine learning algorithms that are so-called because their architecture resembles the neurons in the...

In this section, we will introduce linear transformations. Linear transformations are the backbone of modeling with ANNs. In fact, all the processes of ANN modeling can be thought of as a series of linear transformations. The working components of linear transformations are scalars, vectors, matrices, and tensors. Operations such as addition, transposition, and multiplication are performed on these components.

Scalars, Vectors, Matrices, and Tensors

Scalars, vectors, matrices, and tensors are the actual components of any deep learning model. Having a fundamental understanding of how to utilize these components, as well as the operations that can be performed on them, is key to understanding how ANNs operate. Scalars, vectors, and matrices are examples of the general entity known as a tensor, so the term tensors may be used throughout this chapter but may refer to any component. Scalars, vectors, and matrices refer to tensors with a specific number...

Building ANNs involves creating layers of nodes. Each node can be thought of as a tensor of weights that are learned in the training process. Once the ANN has been fitted to the data, a prediction is made by multiplying the input data by the weight matrices layer by layer, applying any other linear transformation when needed, such as activation functions, until the final output layer is reached. The size of each weight tensor is determined by the size of the shape of the input nodes and the shape of the output nodes. For example, in a single-layer ANN, the size of our single hidden layer can be thought of as follows:

Figure 2.16: Solving the dimensions of the hidden layer of a single-layer ANN

If the input matrix of features has n rows, or observations, and m columns, or features, and we want our predicted target to have n rows (one for each observation) and one column (the predicted value), we can determine the size of our hidden layer...

In this chapter, we covered the various types of linear algebra components and operations that pertain to machine learning. These components include scalars, vectors, matrices, and tensors. The operations that were applied to these tensors included addition, transposition, and multiplication—all of which are fundamental for understanding the underlying mathematics of ANNs.

We also learned some of the basics of the Keras package, including the mathematics that occurs at each node. We replicated the model from the previous chapter, in which we built a logistic regression model to predict the same target from the online shopping purchasing intention dataset. However, in this chapter, we used the Keras library to create the model using an ANN instead of the scikit-learn logistic regression model. We achieved a similar level of accuracy using ANNs.

The upcoming chapters of this book will use the same concepts we learned about in this chapter; however, we will continue...