# Does Linear Discriminant Analysis / Fisher Discriminant Analysis only generate 2 outputs as a dimensional reduction method?

Is that correct that LDA/FDA can only generate 2 outputs as a dimensional reduction method?

Suppose I have 100 features, I want to reduce to 5 features. Is LDA/FDA not usable?

Applying LDA to data with $$K$$ classes allows you to project the data onto a $$K-1$$ dimensional surface in a way that separates the data by class. You cannot arbitrarily choose the number of "features" that the data gets transformed into.

• so Let say my original dataset has 2 classes, the output will be 1 dimensionality ( 2 – 1 =1 ), likewise, if my original dataset has 5 classes, the output will be 4 dimensionality. – aan May 7 at 16:11
• Yes! That's right. If you want to do dimensionality reduction, consider applying PCA to get the first 5 principal components of your data, then project the data onto the subspace spanned by those 5 principal components. – Alex Van de Kleut May 7 at 16:26
• Van de Kluet Thanks. Because also eigenvalues for dimensions > c−1 will be 0 or imaginary (where c is the classes). – aan May 7 at 16:35
• I'm not sure what you mean by that, but the eigenvalues in PCA for real-valued data are always real-valued since PCA does not use the classes of the data. Unless two explanatory variables are perfectly correlated, PCA should give you $d$ eigenvalues/eigenvectors where $d$ is the dimensionality of the data. – Alex Van de Kleut May 7 at 16:38
• sorry, I mean because in LDA/FDA, the eigenvalues for dimensions > c−1 will be 0 or imaginary (where c is the classes). That is why the output is “c-1” where “c” is the number of classes and the dimensionality of the data is n with “n>c” – aan May 7 at 16:43

Also, note that you need labels to perform LDA, which is not always available (or relevant). That's why PCA is usually preferred because it's class-agnostic.

The number of output dimensions in LDA is linked to the number of degrees of freedom in the dataset, which is linked to the number of classes $$c$$: eigenvalues above $$c$$ will be zero and bear no information (same as with PCA and explained variance).

• thanks. I actually have a dataset 2 classes/labels (c), 50 features/dimensions/attributes (d), 150 samples/instances (n). I want to compare PCA and LDA. LDA only produce 1 feature as output because c-1 = 2-1 =1. – aan May 8 at 22:30
• @aan compare PCA and LDA for what? Classification? Data distribution? Variance analysis? If possible, you should ask questions about the goal, not about the tools... – Matthieu May 9 at 7:43
• I want to compare the output of both methods. Just to make sure the output of LDA/FDA is 1 feature from my 2 classes of dataset. – aan May 9 at 8:37
• are you familiar with standarised a data in PCA? stats.stackexchange.com/questions/466460/… – aan May 14 at 13:49