Variable reduction/variable importance
Variable reduction techniques allow you to reduce the number of variables that you need to specify to a model. We will discuss three different methods to accomplish this.
- Principal Components Analysis (PCA).
- All subsets Regression.
- Variable Importance.
Principal Components Analysis (PCA)
Principle Components Analysis (PCA) is a variable reduction technique, and can also be used to identify variable importance. An interesting benefit of PCA is that all of the resulting new component variables will all be uncorrelated with each other. Uncorrelated variables are desirable in a predictive model since too many correlated variables confound predictions and make it difficult to tell which of the independent variables have the most influence. So, if you first perform an exploratory analysis of your data and you find that a high number of correlations exist, this would be a good opportunity to apply PCA.
Note
Models can tolerate some degree of correlated variables...
