3
$\begingroup$

I used PCA to reduce the features of 1020 vector size into desired length like 32, 64, 128 etc.

Now I want to make LDA dimensionality reduction to compare them.

I'm stuck now can't figure out how do I reduce into desired feature vector length the original data set using LDA. (I want it to reduce it to 32, 64 etc).

I'm using the following MATLAB code to achieve it: LDA

The problem is that I have 5 classes and that code reduces the features size into 4. I tried to see what the code does but it seems the maximum what I can get is the feature size of 5.

$\endgroup$
3
$\begingroup$

In LDA, unlike PCA, the maximum number of features after dimensionality reduction is $c - 1$, where $c$ is the number of classes. Since you have 5 classes, the maximum number of features is 4. You can learn the math here. On slide 23, the maximum rank of the between class scatter matrix is $S_B$ is $c-1$. So, the maximum number of eigenvalues/eigenvectors is $c-1$. This is why the maximum number of features is $c-1$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Sorry, just found a more complete answer here: stats.stackexchange.com/questions/167690/… $\endgroup$ – Lam Feb 4 '17 at 16:01
  • $\begingroup$ Can I know that in the context of dimensionality reduction using LDA/FDA. LDA/FDA can start with n dimensions and end with k dimensions, where k < n. Is that correct? Or The output is c-1 where c is the number of classes and the dimensionality of the data is n with n>c. $\endgroup$ – aan May 6 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.