Introduction to DL
In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton published a milestone paper titled ImageNet Classification with Deep Convolutional Neural Networks (https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf). The paper describes their use of NNs to win the ImageNet competition of the same year, which we mentioned in Chapter 2. At the end of their paper, they noted that the network’s performance degrades even if a single layer is removed. Their experiments demonstrated that removing any of the middle layers resulted in an about 2% top-1 accuracy loss of the model. They concluded that network depth is important for the performance of the network. The basic question is: what makes the network’s depth so important?
A typical English saying is a picture is worth a thousand words. Let’s use this approach to understand what DL is. We’ll use images from the highly cited paper Convolutional...
