2
$\begingroup$

Is that correct that LDA/FDA can only generate 2 outputs as a dimensional reduction method?

Suppose I have 100 features, I want to reduce to 5 features. Is LDA/FDA not usable?

$\endgroup$
2
$\begingroup$

Applying LDA to data with $K$ classes allows you to project the data onto a $K-1$ dimensional surface in a way that separates the data by class. You cannot arbitrarily choose the number of "features" that the data gets transformed into.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ so Let say my original dataset has 2 classes, the output will be 1 dimensionality ( 2 – 1 =1 ), likewise, if my original dataset has 5 classes, the output will be 4 dimensionality. $\endgroup$ – aan May 7 at 16:11
  • 1
    $\begingroup$ Yes! That's right. If you want to do dimensionality reduction, consider applying PCA to get the first 5 principal components of your data, then project the data onto the subspace spanned by those 5 principal components. $\endgroup$ – Alex Van de Kleut May 7 at 16:26
  • $\begingroup$ Van de Kluet Thanks. Because also eigenvalues for dimensions > c−1 will be 0 or imaginary (where c is the classes). $\endgroup$ – aan May 7 at 16:35
  • $\begingroup$ I'm not sure what you mean by that, but the eigenvalues in PCA for real-valued data are always real-valued since PCA does not use the classes of the data. Unless two explanatory variables are perfectly correlated, PCA should give you $d$ eigenvalues/eigenvectors where $d$ is the dimensionality of the data. $\endgroup$ – Alex Van de Kleut May 7 at 16:38
  • $\begingroup$ sorry, I mean because in LDA/FDA, the eigenvalues for dimensions > c−1 will be 0 or imaginary (where c is the classes). That is why the output is “c-1” where “c” is the number of classes and the dimensionality of the data is n with “n>c” $\endgroup$ – aan May 7 at 16:43
1
$\begingroup$

Also, note that you need labels to perform LDA, which is not always available (or relevant). That's why PCA is usually preferred because it's class-agnostic.

The number of output dimensions in LDA is linked to the number of degrees of freedom in the dataset, which is linked to the number of classes $c$: eigenvalues above $c$ will be zero and bear no information (same as with PCA and explained variance).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ thanks. I actually have a dataset 2 classes/labels (c), 50 features/dimensions/attributes (d), 150 samples/instances (n). I want to compare PCA and LDA. LDA only produce 1 feature as output because c-1 = 2-1 =1. $\endgroup$ – aan May 8 at 22:30
  • $\begingroup$ @aan compare PCA and LDA for what? Classification? Data distribution? Variance analysis? If possible, you should ask questions about the goal, not about the tools... $\endgroup$ – Matthieu May 9 at 7:43
  • $\begingroup$ I want to compare the output of both methods. Just to make sure the output of LDA/FDA is 1 feature from my 2 classes of dataset. $\endgroup$ – aan May 9 at 8:37
  • $\begingroup$ are you familiar with standarised a data in PCA? stats.stackexchange.com/questions/466460/… $\endgroup$ – aan May 14 at 13:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.