Local and distributed representations
In this section, we’ll dive into local representations and distributed representations. We will go through the characteristics of two different representations and list the most widely used local and global representations to encode different modalities of data.
Local vector representation
As a classic method of text representation, local representation only makes use of the disjointed dimensions in the vector for a certain word when it is represented as a vector. Disjointed dimension means that each dimensionality of the vector represents a single token.
When only one dimension is used, it is called one-hot representation. One-hot means that the word is represented as a long vector, and the dimension of the vector is the total number of words to be represented. Most dimensions are 0, while only one dimension has a value of 1. Different words with a dimension of 1 are not used. If this method of representation is stored sparsely...