Autoencoders for nonlinear feature extraction
In Chapter 17, Deep Learning for Trading, we saw how neural networks succeed at supervised learning by extracting a hierarchical feature representation useful for the given task. Convolutional neural networks (CNNs), for example, learn and synthesize increasingly complex patterns from grid-like data, for example, to identify or detect objects in an image or to classify time series.
An autoencoder, in contrast, is a neural network designed exclusively to learn a new representation that encodes the input in a way that helps solve another task. To this end, the training forces the network to reproduce the input. Since autoencoders typically use the same data as input and output, they are also considered an instance of self-supervised learning. In the process, the parameters of a hidden layer h become the code that represents the input, similar to the word2vec model covered in Chapter 16, Word Embeddings for Earnings Calls and SEC Filings...
