1
$\begingroup$

I have a question regarding Linear Discriminant Analysis (LDA). I know that LDA chooses the coefficients of a linear model, which maximize the separability of classes - that is the ratio of "between-class scatter" and "within-class scatter". Finding the coefficients can be done by solving a generalized eigenvalue problem.

Assume now a LDA classifier which is already trained and should be used to predict new observations.

From my tests (experimenting with different data sets) it looks like a new observation is always put into the class which has the closest mean value to that new point. That would mean that LDA just predicts the class based on the class which has the smallest Mahalanobis distance to the new point.

Question: Is that true? If not: can someone show a small counterexample (I wasn't able to construct one)? If yes: Why should we then solve a generalized eigenvalue problem if instead we could just compute all distances and take the smallest one?

$\endgroup$
1
$\begingroup$

As you probably already know LDA assumes equal covariance matrices for both classes. So the first counter example might be a scenario where the two classes have different covariance matrices. LDA would assume them to be equal and compute the wrong distances.

But assuming the variance-covariance matrices are equal in both classes:

Why should we then solve a generalized eigenvalue problem if instead we could just compute all distances and take the smallest one?

The solution returns a hyperplane separating the classes. A boundary. Once you know this boundary it is a lot easier to check if the observation is above it (belong to 1st class) or below it (belong to the 2nd class) compared to computing the Mahalanobis distance to the averages of both classes and choosing the smaller one.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.