Logistic regression

We will start our exploration of classifier algorithms with one of the most commonly used classification models: logistic regression. Logistic regression is similar to the linear regression method discussed in Chapter 4, Connecting the Dots with Models – Regression Methods, with the major difference being that instead of directly computing a linear combination of the inputs, it compresses the output of a linear model through a function that constrains outputs to be in the range [0,1]. As we will see, this is in fact a kind of "generalized linear model that we discussed in the last Chapter 4, Connecting the Dots with Models – Regression Methods, recall that in linear regression, the predicted output is given by:

where Y is the response variable for all n members of a dataset, X is an n by m matrix of m features for each of the n rows of data, and βT is a column vector of m coefficients (Recall that the T operator represents the transpose of a vector or matrix. Here we transpose...