3.3 Reviewing neural network architectures
In the previous section, we saw how to implement a fully-connected network in the form of an MLP. While such networks were very popular in the early days of deep learning, over the years, machine learning researchers have developed more sophisticated architectures that work more successfully by including domain-specific knowledge (such as computer vision or Natural Language Processing (NLP)). In this section, we will review some of the most common of these neural network architectures, including Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), as well as attention mechanisms and transformers.
3.3.1 Exploring CNNs
When looking back at the example of trying to predict London housing prices with an MLP model, the input features we used (distance to the city centre, floor area, and construction year of the house) were still ”hand-engineered,” meaning that a human looked at the problem and decided which...
