As pointed out before, if we have the problem of having more dimensions (or variables) than samples in our data, we can either augment the data or reduce the dimensionality of the data. Now, we will address the basics of the latter.
We will look into reducing dimensions both in supervised and unsupervised ways with both small and large datasets.
Supervised algorithms
Supervised algorithms for dimensionality reduction are so called because they take the labels of the data into account to find better representations. Such methods often yield good results. Perhaps the most popular kind is called linear discriminant analysis (LDA), which we'll discuss next.
Linear discriminant analysis
Scikit learn has a LinearDiscriminantAnalysis class that can easily perform dimensionality reduction on a desired number of components.
By number of components, the number of dimensions desired is understood. The name comes from principal component analysis (PCA), which is...
