I know that LDA (linear discriminant analysis) is a way for feature extraction, but the dimension of LDA is one less than number of classes in problem (in two classes, the dimension of the result space is 1), so the number of features by LDA is always 1 (in 2 classes case). Is there anyway to extract more than 1 feature with LDA? (When we use PCA we can select n top eigen-vectors for more than 1 feature, but I couldn't find similar way for LDA)

  • $\begingroup$ No, you cannot extract more than min(p,g-1) where p is the number of input features ang g is the number of groups. stats.stackexchange.com/q/169436/3277, stats.stackexchange.com/a/190821/3277 $\endgroup$ – ttnphns Aug 16 '17 at 12:28
  • $\begingroup$ @ttnphns Thanks, so there is no extension on LDA for this purpose? $\endgroup$ – user137927 Aug 16 '17 at 12:33
  • $\begingroup$ LDA is different from PCA substantially, LDA is "supervised", PCA is not. LDA is close akin to canonical correlations (see comparison with PCA). But what do you want specifically? Under LDA assumptions hold 1 discriminant will suffice to predict class membership (see link 2) in 2-class situation. Why do you want any more latent variables, what for? $\endgroup$ – ttnphns Aug 16 '17 at 12:47
  • $\begingroup$ @ttnphns I need more than one feature for classification and the output of LDA is not enough. I though there may be other approaches that consider labels and also extract more than one feature. $\endgroup$ – user137927 Aug 16 '17 at 13:55
  • $\begingroup$ Once again, please: read my answer in the 2nd link. Citing: "Then q=g−1=2 independent dimensions will suffice to predict the class membership as precisely as formerly". If LDA assumptions hold, all the discriminants are enough to classify optimally. $\endgroup$ – ttnphns Aug 16 '17 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.