3.5 Summary

In this chapter, we have seen different types of common neural networks. First, we discussed the key building blocks of neural networks with a special focus on the multi-layer perceptron. Then we reviewed common neural network architectures: convolutional neural networks, recurrent neural networks, and the attention mechanism. All these components allow us to build very powerful deep learning models that can sometimes achieve super-human performance. However, in the second part of the chapter, we reviewed a few problems of neural networks. We discussed how they can be overconfident, and do not handle out-of-distribution data very well. We also saw how small, imperceptible changes to a neural network’s input can cause the model to make an incorrect prediction.

In the next chapter, we will combine the concepts learned in this chapter and in Chapter 3, Fundamentals of Deep Learning, and discuss Bayesian deep learning, which has the potential to overcome some of the...