Linear Discriminant Analysis

In the previous chapter, we discussed logistic regression as a classification model leveraging linear regression to model directly the probability of a target distribution given an input distribution. One alternative to this approach is LDA. LDA models the probability of target distribution class memberships given input variable distributions corresponding to each class using decision boundaries constructed using Bayes’ Theorem, which we discussed previously. Where we have k classes, using Bayes’ Theorem, we have the probability density function for LDA class membership simply as P(Y = k|X = x) for any discrete random variable, X. This relies on the posterior probability that an observation x in variable X belongs to the kth class.

Before proceeding, we must first make note that LDA makes three pertinent assumptions:

• Each input variable is normally distributed.
• Across all target classes, there is equal covariance among the predictors...