I have a question regarding Linear Discriminant Analysis (LDA). I know that LDA chooses the coefficients of a linear model, which maximize the separability of classes - that is the ratio of "between-class scatter" and "within-class scatter". Finding the coefficients can be done by solving a generalized eigenvalue problem.
Assume now a LDA classifier which is already trained and should be used to predict new observations.
From my tests (experimenting with different data sets) it looks like a new observation is always put into the class which has the closest mean value to that new point. That would mean that LDA just predicts the class based on the class which has the smallest Mahalanobis distance to the new point.
Question: Is that true? If not: can someone show a small counterexample (I wasn't able to construct one)? If yes: Why should we then solve a generalized eigenvalue problem if instead we could just compute all distances and take the smallest one?