4
$\begingroup$

I'm running a PCA using the R function prcomp. This is the function:

d2.pca <- prcomp(sel.d2, center=TRUE, scale.=TRUE)

So variables are scaled an centered. (This always has to be done, right?)

This is my original loadings matrix:

                    PC1    PC2    PC3    PC4
var1              0.551 -0.246  0.576 -0.551
var2             -0.545 -0.233  0.736  0.328
var3             -0.427 -0.704 -0.333 -0.460
var4             -0.467  0.625  0.126 -0.613

When I apply variamx rotation:

varimax(d2.pca$rotation)

The output is this one:

$loadings

Loadings:
                 PC1 PC2 PC3 PC4
var1              1             
var2                      1     
var3                 -1         
var4                         -1 

                PC1  PC2  PC3  PC4
SS loadings    1.00 1.00 1.00 1.00
Proportion Var 0.25 0.25 0.25 0.25
Cumulative Var 0.25 0.50 0.75 1.00

$rotmat
       [,1]  [,2]   [,3]   [,4]
[1,]  0.551 0.427 -0.545  0.466
[2,] -0.246 0.704 -0.232 -0.625
[3,]  0.576 0.333  0.736 -0.125
[4,] -0.551 0.461  0.328  0.613

This looks very strange to me, how should I interpret the loadings (1 and -1 values) matrix after varimax rotation? Any help or advise will be appreciated, I'm probably missing something...

Note: KMO was 0.6 for the correlation matrix. Just in case, here it is the correlation matrix:

         var1        var2        var3        var4
var1    1.000      -0.680      -0.491      -0.771
var2   -0.680       1.000       0.697       0.550
var3   -0.491       0.697       1.000       0.166
var4   -0.771       0.550       0.166       1.000 
$\endgroup$

marked as duplicate by whuber Jun 3 '15 at 22:37

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    $\begingroup$ You may find these threads helpful as well: PCA on correlation or covariance?, Is psych::principle function still PCA when using rotation?, & On the use of oblique rotation after PCA. $\endgroup$ – gung Oct 29 '14 at 16:49
  • 1
    $\begingroup$ Hello @DavidVR. It seems that this question received a satisfactory answer and you found it helpful yourself. If so, consider "accepting" this answer by clicking a green tick near it on the left. Looking at your profile, I see that you have never accepted any answers, so maybe you are unfamiliar with this feature. $\endgroup$ – amoeba Dec 14 '14 at 1:20
  • $\begingroup$ Thanks @amoeba I was actually unfamiliar with the feature $\endgroup$ – Flø Dec 15 '14 at 7:36
  • $\begingroup$ @DavidVR: Are you sure you accepted the correct answer??? You accepted a wrong answer with score -1, not the helpful answer with score +4! Please check, you can always change it. $\endgroup$ – amoeba Dec 15 '14 at 8:47
  • $\begingroup$ Sorry @amoeba, I think now is correct! $\endgroup$ – Flø Dec 16 '14 at 8:58
7
$\begingroup$

TL;DR: First choose how many components you want to keep, and then pass only those components to varimax().

Long answer:

I did the same thing once :)

Principal component analysis will find the one component that explains most of the joint variability, then the one component that explain most of the remaining joint variability, and so on. Since the aim of PCA is variable reduction, you typically don't want to keep all the components, just the most important ones (however you may define "important").

varimax() does not enforce it, but its documentation states that the first argument must be a loading matrix with less columns (components) than rows (variables).

According to Wikipedia:

Varimax is so called because it maximizes the sum of the variances of the squared loadings. [...] Intuitively, this is achieved if, (a) any given variable has a high loading on a single factor but near-zero loadings on the remaining factors and if (b) any given factor is constituted by only a few variables with very high loadings on this factor while the remaining variables have near-zero loadings on this factor.

Thus, if you rotate all of your components with varimax, you get your old variables back.

Forget about the negative values. They are said to be platform-dependent and you can multiply all loadings in a factor by -1 if you will.

What you've got to do is to choose how many components you want to keep, and pass only those components to varimax(). In example, if you want to keep 2 components:

d2.varimax <- varimax(d2.pca$rotation[, 1:2])
$\endgroup$
  • 2
    $\begingroup$ Welcome to CrossValidated, @Leonardo! This looks too short for an answer and would better have been a comment. However, if you elaborate a bit further, this could become a good answer. In particular: what does it mean that the rotated matrix contains only ones and zeros? does it always happen if varimax rotation is applied to all components? if so, why? what is going to be different if only a subset of components is varimax-rotated, as you suggest? $\endgroup$ – amoeba Oct 29 '14 at 14:27
  • 1
    $\begingroup$ Thank you for expanding your answer. I think it will be more helpful for people now. Welcome to CV, I hope we will see more like this in the future. Since you are new here, you may want to take our tour, which contains information for new users. $\endgroup$ – gung Oct 29 '14 at 16:39
  • 1
    $\begingroup$ (+1) I am no expert on varimax rotation, but note that what prcomp calls "loadings" are in fact eigenvectors of the correlation matrix (aka principal axes), and so they are NOT giving correlations between variables and components, which is what is called "loadings" in factor analysis. To get loadings in the FA sense ("true loadings"), one needs to multiply PCA eigenvectors with the respective eigenvalues. As varimax is something that is usually used in the FA context, I am wondering if it makes more sense to run it on the true loadings, and not on prcomp$rotation. Or does it not matter? $\endgroup$ – amoeba Oct 29 '14 at 16:53
  • 2
    $\begingroup$ @Amoeba Same disclaimer for me, but eigenvalues are variances, so I think you mean: multiply by their square roots (SDs). $\endgroup$ – Nick Cox Oct 29 '14 at 17:13
  • 1
    $\begingroup$ Thanks for receiving me, @amoeba. I've being reading Cross Validated and occasionally other StackExchange sites for quite sometime, so it's nice to be able to help someone here. I was surprised with how easy it is for outsiders to contribute an answer, and was very impressed by the way you prompted me to improve it. $\endgroup$ – Leonardo Fontenelle Oct 30 '14 at 1:27
0
$\begingroup$
pca         <- princomp(mydata, cor=TRUE)
pcaloadings <-loadings(pca) # pc loadings
pcarotation <- varimax(pcaloadings[, 1:2], normalize=F)

Results will depend on the number of components that you select. To run the screeplot before, will help you to choose that.

$\endgroup$
  • 2
    $\begingroup$ It's not clear how this answers the question of why the results are 1s & 0s. $\endgroup$ – gung Oct 30 '14 at 17:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.