Quadratic Discriminant Analysis

In the last section, we discussed LDA. The data within each class needs to be drawn from a multivariate Gaussian distribution, and the covariance matrix is the same across different classes. In this section, we consider another type of discriminant analysis called QDA but the assumptions for QDA can be relaxed on the covariance matrix assumption. Here, we do not need the covariance matrix to be identical across different classes but only for each class to have its own covariance matrix. The multivariate Gaussian distribution with a class-specific mean vector within each class for observations is still required to conduct QDA. We assume that an observation from a k th class satisfies the following formula:

X~N(μ k, Σ k)

We’ll thus consider a generative classifier, as follows:

p(X | y = k, θ) = N(X | μ k, Σ k)

And then, its corresponding class posterior is this:

p(y = k | X, ...