Why does my loading matrix following PCA with a varimax rotation contain only ones and zeros? [duplicate]

Question

I'm running a PCA using the R function prcomp. This is the function:

d2.pca <- prcomp(sel.d2, center=TRUE, scale.=TRUE)

So variables are scaled an centered. (This always has to be done, right?)

This is my original loadings matrix:

                    PC1    PC2    PC3    PC4
var1              0.551 -0.246  0.576 -0.551
var2             -0.545 -0.233  0.736  0.328
var3             -0.427 -0.704 -0.333 -0.460
var4             -0.467  0.625  0.126 -0.613

When I apply variamx rotation:

varimax(d2.pca$rotation)

The output is this one:

$loadings

Loadings:
                 PC1 PC2 PC3 PC4
var1              1             
var2                      1     
var3                 -1         
var4                         -1 

                PC1  PC2  PC3  PC4
SS loadings    1.00 1.00 1.00 1.00
Proportion Var 0.25 0.25 0.25 0.25
Cumulative Var 0.25 0.50 0.75 1.00

$rotmat
       [,1]  [,2]   [,3]   [,4]
[1,]  0.551 0.427 -0.545  0.466
[2,] -0.246 0.704 -0.232 -0.625
[3,]  0.576 0.333  0.736 -0.125
[4,] -0.551 0.461  0.328  0.613

This looks very strange to me, how should I interpret the loadings (1 and -1 values) matrix after varimax rotation? Any help or advise will be appreciated, I'm probably missing something...

Note: KMO was 0.6 for the correlation matrix. Just in case, here it is the correlation matrix:

         var1        var2        var3        var4
var1    1.000      -0.680      -0.491      -0.771
var2   -0.680       1.000       0.697       0.550
var3   -0.491       0.697       1.000       0.166
var4   -0.771       0.550       0.166       1.000 

Answer 1

TL;DR: First choose how many components you want to keep, and then pass only those components to varimax().

Long answer:

I did the same thing once :)

Principal component analysis will find the one component that explains most of the joint variability, then the one component that explain most of the remaining joint variability, and so on. Since the aim of PCA is variable reduction, you typically don't want to keep all the components, just the most important ones (however you may define "important").

varimax() does not enforce it, but its documentation states that the first argument must be a loading matrix with less columns (components) than rows (variables).

According to Wikipedia:

Varimax is so called because it maximizes the sum of the variances of the squared loadings. [...] Intuitively, this is achieved if, (a) any given variable has a high loading on a single factor but near-zero loadings on the remaining factors and if (b) any given factor is constituted by only a few variables with very high loadings on this factor while the remaining variables have near-zero loadings on this factor.

Thus, if you rotate all of your components with varimax, you get your old variables back.

Forget about the negative values. They are said to be platform-dependent and you can multiply all loadings in a factor by -1 if you will.

What you've got to do is to choose how many components you want to keep, and pass only those components to varimax(). In example, if you want to keep 2 components:

d2.varimax <- varimax(d2.pca$rotation[, 1:2])

Answer 2

pca         <- princomp(mydata, cor=TRUE)
pcaloadings <-loadings(pca) # pc loadings
pcarotation <- varimax(pcaloadings[, 1:2], normalize=F)

Results will depend on the number of components that you select. To run the screeplot before, will help you to choose that.