# Why does my loading matrix following PCA with a varimax rotation contain only ones and zeros? [duplicate]

I'm running a PCA using the R function prcomp. This is the function:

d2.pca <- prcomp(sel.d2, center=TRUE, scale.=TRUE)


So variables are scaled an centered. (This always has to be done, right?)

                    PC1    PC2    PC3    PC4
var1              0.551 -0.246  0.576 -0.551
var2             -0.545 -0.233  0.736  0.328
var3             -0.427 -0.704 -0.333 -0.460
var4             -0.467  0.625  0.126 -0.613


When I apply variamx rotation:

varimax(d2.pca$rotation)  The output is this one: $loadings

PC1 PC2 PC3 PC4
var1              1
var2                      1
var3                 -1
var4                         -1

PC1  PC2  PC3  PC4
Proportion Var 0.25 0.25 0.25 0.25
Cumulative Var 0.25 0.50 0.75 1.00

$rotmat [,1] [,2] [,3] [,4] [1,] 0.551 0.427 -0.545 0.466 [2,] -0.246 0.704 -0.232 -0.625 [3,] 0.576 0.333 0.736 -0.125 [4,] -0.551 0.461 0.328 0.613  This looks very strange to me, how should I interpret the loadings (1 and -1 values) matrix after varimax rotation? Any help or advise will be appreciated, I'm probably missing something... Note: KMO was 0.6 for the correlation matrix. Just in case, here it is the correlation matrix:  var1 var2 var3 var4 var1 1.000 -0.680 -0.491 -0.771 var2 -0.680 1.000 0.697 0.550 var3 -0.491 0.697 1.000 0.166 var4 -0.771 0.550 0.166 1.000  • You may find these threads helpful as well: PCA on correlation or covariance?, Is psych::principle function still PCA when using rotation?, & On the use of oblique rotation after PCA. Oct 29, 2014 at 16:49 • Hello @DavidVR. It seems that this question received a satisfactory answer and you found it helpful yourself. If so, consider "accepting" this answer by clicking a green tick near it on the left. Looking at your profile, I see that you have never accepted any answers, so maybe you are unfamiliar with this feature. Dec 14, 2014 at 1:20 • Thanks @amoeba I was actually unfamiliar with the feature Dec 15, 2014 at 7:36 • @DavidVR: Are you sure you accepted the correct answer??? You accepted a wrong answer with score -1, not the helpful answer with score +4! Please check, you can always change it. Dec 15, 2014 at 8:47 • Sorry @amoeba, I think now is correct! Dec 16, 2014 at 8:58 ## 2 Answers TL;DR: First choose how many components you want to keep, and then pass only those components to varimax(). Long answer: I did the same thing once :) Principal component analysis will find the one component that explains most of the joint variability, then the one component that explain most of the remaining joint variability, and so on. Since the aim of PCA is variable reduction, you typically don't want to keep all the components, just the most important ones (however you may define "important"). varimax() does not enforce it, but its documentation states that the first argument must be a loading matrix with less columns (components) than rows (variables). According to Wikipedia: Varimax is so called because it maximizes the sum of the variances of the squared loadings. [...] Intuitively, this is achieved if, (a) any given variable has a high loading on a single factor but near-zero loadings on the remaining factors and if (b) any given factor is constituted by only a few variables with very high loadings on this factor while the remaining variables have near-zero loadings on this factor. Thus, if you rotate all of your components with varimax, you get your old variables back. Forget about the negative values. They are said to be platform-dependent and you can multiply all loadings in a factor by -1 if you will. What you've got to do is to choose how many components you want to keep, and pass only those components to varimax(). In example, if you want to keep 2 components: d2.varimax <- varimax(d2.pca$rotation[, 1:2])

• Welcome to CrossValidated, @Leonardo! This looks too short for an answer and would better have been a comment. However, if you elaborate a bit further, this could become a good answer. In particular: what does it mean that the rotated matrix contains only ones and zeros? does it always happen if varimax rotation is applied to all components? if so, why? what is going to be different if only a subset of components is varimax-rotated, as you suggest? Oct 29, 2014 at 14:27
• Thank you for expanding your answer. I think it will be more helpful for people now. Welcome to CV, I hope we will see more like this in the future. Since you are new here, you may want to take our tour, which contains information for new users. Oct 29, 2014 at 16:39
• (+1) I am no expert on varimax rotation, but note that what prcomp calls "loadings" are in fact eigenvectors of the correlation matrix (aka principal axes), and so they are NOT giving correlations between variables and components, which is what is called "loadings" in factor analysis. To get loadings in the FA sense ("true loadings"), one needs to multiply PCA eigenvectors with the respective eigenvalues. As varimax is something that is usually used in the FA context, I am wondering if it makes more sense to run it on the true loadings, and not on prcomp\$rotation. Or does it not matter? Oct 29, 2014 at 16:53
• @Amoeba Same disclaimer for me, but eigenvalues are variances, so I think you mean: multiply by their square roots (SDs). Oct 29, 2014 at 17:13
• Thanks for receiving me, @amoeba. I've being reading Cross Validated and occasionally other StackExchange sites for quite sometime, so it's nice to be able to help someone here. I was surprised with how easy it is for outsiders to contribute an answer, and was very impressed by the way you prompted me to improve it. Oct 30, 2014 at 1:27
pca         <- princomp(mydata, cor=TRUE)