Tuning models for interpretability
Traditionally, regularization was only achieved by imposing penalty terms such as L1, L2, or elastic net on the coefficients or weights, which shrink the impact of the least relevant features. As seen in the Embedded methods section of Chapter 10, Feature Selection and Engineering for Interpretability, this form of regularization results in feature selection while also reducing overfitting. And this brings us to another broader definition of regularization, which does not require a penalty term. Often, this comes as imposing a limitation, or a stopping criterion that forces the model to curb its complexity.
In addition to regularization, both in its narrow (penalty-based) and broad sense (overfitting methods), there are other methods that tune a model for interpretability—that is, improve the fairness, accountability, and transparency of a model through adjustments to the training process. For instance, the class imbalance hyperparameters...
