# How the Correlation Matrix is built for PCA in Weka?

Just to give a context, I want to use PCA (Principal Component Analysis) to identify which attributes are similar to others, so I can use just one (or a subset) of them. The correlation matrix of n variables X1, ..., Xn is the n×n matrix whose i,j entry is corr(Xi, Xj).

So if I have 8 attributes, I would have a correlation matrix of size of 8x8. After running PCA in Weka for 8 attributes, the output shows me a Correlation matrix of size of 44x44. And printed 26 Selected attributes.

My question: Shouldn't my correlation matrix have the size of 8x8? How does weka build such matrix?

Thanks!!

MC

• If you have 8 features then your correlation matrix should be 8x8. The question about Weka is off-topic though. – Firebug Jun 21 '16 at 18:42
• Do you have 44 observations on your 8 variables? – gung - Reinstate Monica Jun 21 '16 at 18:45
• @user116100, do you have 44 observations on your 8 variables? – gung - Reinstate Monica Jun 21 '16 at 18:59
• @Firebug, I believed my question is about PCA and not Weka, but I mentioned Weka just to give more details what I am doing. :) – Bigo Jun 21 '16 at 19:01
• @gung, I have 5725 instances. Do you what these 44 columns/rows does represent if there are not my 8 attributes? – Bigo Jun 21 '16 at 19:01

In Weka, if you don't set the target class, it may treat each categorical value as another potential component.

Example: Iris Data

If you take the Iris data set, by default the class is not set, and the Attribute Selection Model shows "No Class". There are 4 features (sepal_length, sepal_width, petal_length, petal_width) and three class categories (setosa, versicolor, virginica). Running PCA with the default setting returns a 7x7 correlation matrix.

Correlation matrix
1     -0.11   0.87   0.82  -0.72   0.08   0.64
-0.11   1     -0.42  -0.36   0.6   -0.46  -0.13
0.87  -0.42   1      0.96  -0.92   0.2    0.72
0.82  -0.36   0.96   1     -0.89   0.12   0.77
-0.72   0.6   -0.92  -0.89   1     -0.5   -0.5
0.08  -0.46   0.2    0.12  -0.5    1     -0.5
0.64  -0.13   0.72   0.77  -0.5   -0.5    1


And you can see the top eigenvector returns components including the classes, which is probably not desirable:

eigenvalue  proportion  cumulative
4.34646     0.62092     0.62092   0.476petallength+0.465petalwidth-0.451class=Iris-setosa+0.411sepallength+0.349class=Iris-virginica...
1.76093     0.25156     0.87248   0.7  class=Iris-versicolor-0.471class=Iris-virginica-0.451sepalwidth-0.229class=Iris-setosa-0.158sepallength...
0.68223     0.09746     0.96995   -0.75sepalwidth-0.441sepallength-0.36class=Iris-versicolor+0.325class=Iris-virginica-0.073petallength...


Setting the Attribute Selection drop down to "(Nom) Class" removes the target categories from the Correlation Matrix and eigenvectors, and will return a 4x4 Correlation matrix.

Correlation matrix
1     -0.11   0.87   0.82
-0.11   1     -0.42  -0.36
0.87  -0.42   1      0.96
0.82  -0.36   0.96   1

eigenvalue  proportion  cumulative
2.91082     0.7277      0.7277    -0.581petallength-0.566petalwidth-0.522sepallength+0.263sepalwidth
0.92122     0.23031     0.95801   0.926sepalwidth+0.372sepallength+0.065petalwidth+0.021petallength