Artificial Neural Networks

Artificial neural networks try to mimic the way the human brain works. They are used to solve a number of difficult problems, such as understanding written or spoken language, identifying objects in an image, or driving a car.

You will learn the basics of how an artificial neural network works, look at the steps and mathematical calculations needed to train it, and have a general view of complex neural networks.

The following topics will be covered in this chapter:

• Introducing the perceptron – the simplest type of neural network
• Building a deep network
• Understanding the backpropagation algorithm

Technical requirements

To complete this chapter, you will need to download the transfusion.xlsx file from the GitHub repository at https://github.com/PacktPublishing/Hands-On-Machine-Learning-with-Microsoft-Excel-2019/tree/master/Chapter09.

Introducing the perceptron – the simplest type of neural network

Neural networks are inspired by the human brain' more specifically, by the neuron cells that compose it. Actually, since there have been major advances in neuroscience since the first artificial neuron was designed, it would be better to say that they are inspired by what was known about the brain some years ago.

The perceptron was the first attempt to build an artificial neural network (Frank Rosenblatt, 1959). It was actually a model of a single neuron, with multiple inputs and one output. The value at the output is calculated as the weighted sum of the inputs and these weights are adjusted iteratively. This simple implementation has many disadvantages and limitations, so it was later replaced by the multilayer perceptron. The most basic model of this artificial neural network has the structure shown...

Building a deep network

Our example of artificial neural network is very simple and only contains one hidden layer. Can we add more layers? Of course we can! The next step in complexity could be something similar to the following diagram:

We added a new hidden layer with two neurons, but we could add more layers and more neurons per layer. The architecture of a network depends on the specific use we give it. Multilayer artificial neural networks are often known as deep neural networks.

The output of a deep network is calculated in analogy with the single layer one, considering all inputs to each neuron, the activation function, and the addition of all the inputs to the output neuron. Looking at the preceding diagram, it is clear that each layer in the network is affected by the previous one. It is usually the case that, in order to solve complex problems, each layer learns a...

Understanding the backpropagation algorithm

There are two phases in the training process of a deep neural network: forward and back propagation. We have seen the forward phase in detail:

1. Calculate the weighted sum of the inputs:

2. Apply the activation function to the result:

Find different activation functions in the suggested reading at the end of the chapter. The sigmoid function is the most common and is easier to use, but not the only one.

3. Calculate the output by adding all the results from the last layer (N neurons):

After the forward phase, we calculate the error as the difference between the output and the known target value: Error = (Output-y)².

All weights are assigned random values at the beginning of the forward phase.

The output, and therefore the error, are functions of the weights w_i and θ_i. This means that we could go backward from the error and see...

Summary

We have studied the basic principles of how artificial neural networks are built and how they learn from the input data. Even if the actual method, in practice, for using neural networks is different than what we have done in our example, our approach is useful in order to understand the details and to go beyond the idea that neural networks are mysterious black boxes that magically solve problems.

In the next chapter, we will see how we can use pre-built machine learning models available in Azure, connecting them to Excel to solve the problems we have presented up to now.

Questions

1. Using the results of the perceptron test, build the confusion matrix and evaluate the quality of the prediction.
2. There is one important step that is missing in the binary classification problem that we solved with our artificial neural network, which might improve the result if we implement it. What did we miss? Hint: build an histogram of the binary variable that indicates whether there was a blood donation in March 2007.

Further reading

• Brief Introduction to Neural Networks by David Kriesel, available online at http://www.dkriesel.com/_media/science/neuronalenetze-en-zeta2-1col-dkrieselcom.pdf
• Neural Networks and Deep Learning by Michael A. Nielsen, available online at http://neuralnetworksanddeeplearning.com/
• Deep Learning: Using Algorithms to Make Machines Think, https://opensourceforu.com/2017/12/deep-learning-using-algorithms-to-make-machines-think/