Feature engineering
When trying to improve performance, we may also consider ways to provide the best features (model inputs) to our model through the process of feature engineering. The Preprocessing data section in Chapter 9, Getting Started with Machine Learning in Python, introduced us to feature transformation when we scaled, encoded, and imputed our data. Unfortunately, feature transformation may mute some elements of our data that we want to use in our model, such as the unscaled value of the mean of a specific feature. For this situation, we can create a new feature with this value; this and other new features are added during feature construction (sometimes called feature creation).
Feature selection is the process of determining which features to train the model on. This can be done manually or through another process, such as machine learning. When looking to choose features for our model, we want features that have an impact on our dependent variable without unnecessarily...
