I used PCA to reduce the features of 1020 vector size into desired length like 32, 64, 128 etc.

Now I want to make LDA dimensionality reduction to compare them.

I'm stuck now can't figure out how do I reduce into desired feature vector length the original data set using LDA. (I want it to reduce it to 32, 64 etc).

I'm using the following MATLAB code to achieve it: LDA

The problem is that I have 5 classes and that code reduces the features size into 4. I tried to see what the code does but it seems the maximum what I can get is the feature size of 5.


1 Answer 1


In LDA, unlike PCA, the maximum number of features after dimensionality reduction is $c - 1$, where $c$ is the number of classes. Since you have 5 classes, the maximum number of features is 4. You can learn the math here. On slide 23, the maximum rank of the between class scatter matrix is $S_B$ is $c-1$. So, the maximum number of eigenvalues/eigenvectors is $c-1$. This is why the maximum number of features is $c-1$.

  • $\begingroup$ Sorry, just found a more complete answer here: stats.stackexchange.com/questions/167690/… $\endgroup$
    – Starz
    Feb 4, 2017 at 16:01
  • $\begingroup$ Can I know that in the context of dimensionality reduction using LDA/FDA. LDA/FDA can start with n dimensions and end with k dimensions, where k < n. Is that correct? Or The output is c-1 where c is the number of classes and the dimensionality of the data is n with n>c. $\endgroup$
    – aan
    May 6, 2020 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.