0
$\begingroup$

I am trying to implement Linear Discriminant Analysis for face recognition. I have 3 classes and each classes have 10 image each. The dimension of matrix in class A, B and C is 10*500. So each row will represent an image.

If I find the mean matrix of each class I am getting dimension of 1*500. That is I will be adding the row and divide by 10. Global mean matrix of all classes I am getting 1*500 dimension.

Within Scatter Matrix Sw= The dimension of Matrix is 10*10 Matrix.

Between Scatter Matrix Sb= The dimension of Matrix is 1*1.

Next Step is I have to find Inverse(Sw)*Sb. But the matrix dimension is totally different. I know I am doing some mistake but I don't know where?

Could you please help me to solve this problem?

Can you please tell me how the dimension of the matrix should be?

Improve this question
$\endgroup$
4
  • $\begingroup$ Between-matrix B should be the same size as Within-matrix W. Because B=T-W where T is Total scatter matrix (matrix for the whole sample). $\endgroup$
    – ttnphns
    Apr 1, 2014 at 21:09
  • $\begingroup$ You may read the algorithm here: stats.stackexchange.com/a/48859/3277. $\endgroup$
    – ttnphns
    Apr 1, 2014 at 21:11
  • $\begingroup$ Thank you, but usually when we do according to the algorithm, within Scatter Matter will be 500*500 and between scatter matter matrix will also be 500*500. Then there won't be any problem. But the thing is when I program and try to find the within scatter matrix, it will give me java heap memory error. So I am finding like (10*500)*(500*10), it will give me 10*10 matrix. but in the between matrix I am not getting that 10*10 matrix. I am doing somewhere wrong. Can u solve this?. $\endgroup$
    – user2109988
    Apr 1, 2014 at 22:35
  • $\begingroup$ It sounds like a technical/programming trouble, not statistical. If you use any Linear/Matrix algebra package available for Java (I believe there must be such) there won't be any problem to operate with 500x500 matrices. Also, note that you have more dimensions (500) than cases (30) - you will have math problems with singularity. $\endgroup$
    – ttnphns
    Apr 1, 2014 at 22:49

1 Answer 1

Reset to default
2
$\begingroup$

Your sizes of $S_W$ and $S_B$ are wrong. They have to have the same size.

But let's understand why: both of them represent the covariance among the different variables into your data, one in a single class and the other among all classes.

Therefore, being them expressing covariance among all different variables, is clear that their size should be square and $d\times d$ where $d$ is the dimensionality of your data. In your case is 500, hence you should have two matrices of $500 \times 500$.

Here is a post where they discuss correct formulas to compute LDA, have a look and make sure you are exactly computing these matrices.

Also, when you get a $1 \times 1$ MATRIX it should ring a bell that something is wrong, because it would be otherwise called 'a scalar' and not a matrix if its size would be that. Although it is formally correct having a matrix 1 by 1 in math, when it comes to scatter matrix this makes absolutely no sense.

Improve this answer
$\endgroup$
1
  • $\begingroup$ +1. Thanks a lot for answering these old questions, it is very helpful for the site. $\endgroup$
    – amoeba
    Aug 2, 2016 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.