Summary

Logistic regression is the prototypical method for solving classification problems, just as linear regression was the prototypical example of a model to solve regression problems. In this chapter, we demonstrated why logistic regression offers a better way of approaching classification problems compared to linear regression with a threshold, by showing that the least squares criterion is not the most appropriate criterion to use when trying to separate two classes. We presented the notion of likelihood and its maximization as the basis for training a model. This is a very important concept that features time and again in various machine learning contexts. Logistic regression is an example of a generalized linear model. This is a model that relates the output variable to a linear combination of input features via a link function, which we saw was the logit function in this case. For the binary classification problem, we used R's glm() function to perform logistic regression on a real...