3
$\begingroup$

Just to give a context, I want to use PCA (Principal Component Analysis) to identify which attributes are similar to others, so I can use just one (or a subset) of them. The correlation matrix of n variables X1, ..., Xn is the n×n matrix whose i,j entry is corr(Xi, Xj).

So if I have 8 attributes, I would have a correlation matrix of size of 8x8. After running PCA in Weka for 8 attributes, the output shows me a Correlation matrix of size of 44x44. And printed 26 Selected attributes.

My question: Shouldn't my correlation matrix have the size of 8x8? How does weka build such matrix?

Thanks!!

MC

$\endgroup$
10
  • 1
    $\begingroup$ If you have 8 features then your correlation matrix should be 8x8. The question about Weka is off-topic though. $\endgroup$ – Firebug Jun 21 '16 at 18:42
  • $\begingroup$ Do you have 44 observations on your 8 variables? $\endgroup$ – gung - Reinstate Monica Jun 21 '16 at 18:45
  • $\begingroup$ @user116100, do you have 44 observations on your 8 variables? $\endgroup$ – gung - Reinstate Monica Jun 21 '16 at 18:59
  • 1
    $\begingroup$ @Firebug, I believed my question is about PCA and not Weka, but I mentioned Weka just to give more details what I am doing. :) $\endgroup$ – Bigo Jun 21 '16 at 19:01
  • $\begingroup$ @gung, I have 5725 instances. Do you what these 44 columns/rows does represent if there are not my 8 attributes? $\endgroup$ – Bigo Jun 21 '16 at 19:01
1
$\begingroup$

In Weka, if you don't set the target class, it may treat each categorical value as another potential component.

Example: Iris Data

If you take the Iris data set, by default the class is not set, and the Attribute Selection Model shows "No Class". There are 4 features (sepal_length, sepal_width, petal_length, petal_width) and three class categories (setosa, versicolor, virginica). Running PCA with the default setting returns a 7x7 correlation matrix.

Correlation matrix
  1     -0.11   0.87   0.82  -0.72   0.08   0.64 
 -0.11   1     -0.42  -0.36   0.6   -0.46  -0.13 
  0.87  -0.42   1      0.96  -0.92   0.2    0.72 
  0.82  -0.36   0.96   1     -0.89   0.12   0.77 
 -0.72   0.6   -0.92  -0.89   1     -0.5   -0.5  
  0.08  -0.46   0.2    0.12  -0.5    1     -0.5  
  0.64  -0.13   0.72   0.77  -0.5   -0.5    1    

And you can see the top eigenvector returns components including the classes, which is probably not desirable:

eigenvalue  proportion  cumulative
  4.34646     0.62092     0.62092   0.476petallength+0.465petalwidth-0.451class=Iris-setosa+0.411sepallength+0.349class=Iris-virginica...
  1.76093     0.25156     0.87248   0.7  class=Iris-versicolor-0.471class=Iris-virginica-0.451sepalwidth-0.229class=Iris-setosa-0.158sepallength...
  0.68223     0.09746     0.96995   -0.75sepalwidth-0.441sepallength-0.36class=Iris-versicolor+0.325class=Iris-virginica-0.073petallength...

Setting the Attribute Selection drop down to "(Nom) Class" removes the target categories from the Correlation Matrix and eigenvectors, and will return a 4x4 Correlation matrix.

Correlation matrix
  1     -0.11   0.87   0.82 
 -0.11   1     -0.42  -0.36 
  0.87  -0.42   1      0.96 
  0.82  -0.36   0.96   1    
eigenvalue  proportion  cumulative
  2.91082     0.7277      0.7277    -0.581petallength-0.566petalwidth-0.522sepallength+0.263sepalwidth
  0.92122     0.23031     0.95801   0.926sepalwidth+0.372sepallength+0.065petalwidth+0.021petallength

set the dropdown to (Nom) class

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.