2
$\begingroup$

Is that correct that LDA/FDA can only generate 2 outputs as a dimensional reduction method?

Suppose I have 100 features, I want to reduce to 5 features. Is LDA/FDA not usable?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

Applying LDA to data with $K$ classes allows you to project the data onto a $K-1$ dimensional surface in a way that separates the data by class. You cannot arbitrarily choose the number of "features" that the data gets transformed into.

Improve this answer
$\endgroup$
6
  • $\begingroup$ so Let say my original dataset has 2 classes, the output will be 1 dimensionality ( 2 – 1 =1 ), likewise, if my original dataset has 5 classes, the output will be 4 dimensionality. $\endgroup$
    – aan
    May 7, 2020 at 16:11
  • 1
    $\begingroup$ Yes! That's right. If you want to do dimensionality reduction, consider applying PCA to get the first 5 principal components of your data, then project the data onto the subspace spanned by those 5 principal components. $\endgroup$
    – Alex Van de Kleut
    May 7, 2020 at 16:26
  • $\begingroup$ Van de Kluet Thanks. Because also eigenvalues for dimensions > c−1 will be 0 or imaginary (where c is the classes). $\endgroup$
    – aan
    May 7, 2020 at 16:35
  • $\begingroup$ I'm not sure what you mean by that, but the eigenvalues in PCA for real-valued data are always real-valued since PCA does not use the classes of the data. Unless two explanatory variables are perfectly correlated, PCA should give you $d$ eigenvalues/eigenvectors where $d$ is the dimensionality of the data. $\endgroup$
    – Alex Van de Kleut
    May 7, 2020 at 16:38
  • $\begingroup$ sorry, I mean because in LDA/FDA, the eigenvalues for dimensions > c−1 will be 0 or imaginary (where c is the classes). That is why the output is “c-1” where “c” is the number of classes and the dimensionality of the data is n with “n>c” $\endgroup$
    – aan
    May 7, 2020 at 16:43
1
$\begingroup$

Also, note that you need labels to perform LDA, which is not always available (or relevant). That's why PCA is usually preferred because it's class-agnostic.

The number of output dimensions in LDA is linked to the number of degrees of freedom in the dataset, which is linked to the number of classes $c$: eigenvalues above $c$ will be zero and bear no information (same as with PCA and explained variance).

Improve this answer
$\endgroup$
4
  • $\begingroup$ thanks. I actually have a dataset 2 classes/labels (c), 50 features/dimensions/attributes (d), 150 samples/instances (n). I want to compare PCA and LDA. LDA only produce 1 feature as output because c-1 = 2-1 =1. $\endgroup$
    – aan
    May 8, 2020 at 22:30
  • $\begingroup$ @aan compare PCA and LDA for what? Classification? Data distribution? Variance analysis? If possible, you should ask questions about the goal, not about the tools... $\endgroup$
    – Matthieu
    May 9, 2020 at 7:43
  • $\begingroup$ I want to compare the output of both methods. Just to make sure the output of LDA/FDA is 1 feature from my 2 classes of dataset. $\endgroup$
    – aan
    May 9, 2020 at 8:37
  • $\begingroup$ are you familiar with standarised a data in PCA? stats.stackexchange.com/questions/466460/… $\endgroup$
    – aan
    May 14, 2020 at 13:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.