5
$\begingroup$

Let's say I have a $m \times n$ matrix where $m$ is the number of points and $n$ is the number of dimensions. I would like to give a target dimension parameter which is let's say d. d can be a set of values like $\{2,4,6,\ldots,n\}$. I would then approach the problem using Fisher's linear discriminant analysis to give me output matrix in $m \times d$. Is this possible to do? I still don't understand how LDA reduces dimensions from let's say 10,000 to 1,000.

Improve this question
$\endgroup$
4
  • 7
    $\begingroup$ LDA produces min(n,c-1) discriminants (c is the number of classes). They are sorted descendingly by their discriminative contribution. You may leave d first discriminants, exactly like you leave first d components in Principal Components analysis. $\endgroup$
    – ttnphns
    Commented Nov 16, 2012 at 19:12
  • $\begingroup$ Please have a look at question stats.stackexchange.com/questions/43768/… A good resource to learn more about LDA/LSI is Chapter 18 of freely available book Introduction to Information Retrieval. $\endgroup$
    – abhinavkulkarni
    Commented Nov 16, 2012 at 19:51
  • $\begingroup$ Please search discriminant LDA dimensionality reduction this site for relevant questions/answers. $\endgroup$
    – ttnphns
    Commented Apr 6, 2016 at 7:23
  • $\begingroup$ @abhinavkulkarni You might be confusing Latent Dirichlet Allocation (a probabilistic extension of Latent Semantic Indexing), with linear discriminant analysis, since they both use the acronym LDA. This question is about linear discriminant analysis, which is a very separate algorithm from latent dirichlet allocation / search indexing / topic modeling in general. The Manning Information Retrieval book does not appear to have any sections on linear discriminant analysis. $\endgroup$
    – ely
    Commented Apr 23, 2018 at 14:28

1 Answer 1

Reset to default
4
$\begingroup$

How LDA does dimension reducion has same methodology with PCA. When you get J(W) at LDA (W is result that which minimizes within class scatter matrix but maximizes between class scatter matrix) 

W = inv(Sw)*Sb

When you get W, you can get result with:

 V is eigenvectors of W
 final_result = V'*your_data

before that the trick plays role. As like at PCA, you can get eigenvectors and eigenvalues of W and you can discard some eigenvectors that has smallest corresponding eigenvalues. Then if you multiply your data with that eliminated eigenvector matrix (V) you get a lower dimension.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.