Regularization optimized by grid-search

Regularization is another way to modify the role of variables in a regression model to prevent overfitting and to achieve simpler functional forms. The interesting aspect of this alternative approach is that it actually doesn't require manipulating your original dataset, making it suitable for systems that learn and predict online from large amounts of features and observations, without human intervention. Regularization works by enriching the learning process using a penalization for too complex models to shrink (or reduce to zero) coefficients relative to variables that are irrelevant for your prediction term or are redundant, as they are highly correlated with others present in the model (the collinearity problem seen before).

Ridge (L2 regularization)

The idea behind ridge regression is simple and straightforward: if the problem is the presence of many variables, which affect the regression model because of their coefficient, all we have to do is...