# Why does varimax applied to PCA outcome fail to do anything at all?

I am trying to perform PCA with varimax rotation on a set of data with 2 variables, in order to orthogonalise them (not for dimension reduction). I used prcomp() to perform PCA and it gave me results like this:

> print(variable.pca)
Standard deviations:
[1] 1.1104314 0.8420727

Rotation:
PC1        PC2
LogFreq    -0.6646101 -0.7471903
Simulation -0.7471903  0.6646101

> summary(variable.pca)
Importance of components:
PC1    PC2
Standard deviation     1.1104 0.8421
Proportion of Variance 0.6349 0.3651
Cumulative Proportion  0.6349 1.0000


Then I applied varimax() to the variable.pca$rotation. And here's the results: $loadings

PC1    PC2
LogFreq    -0.665 -0.747
Simulation -0.747  0.665

PC1 PC2
Proportion Var 0.5 0.5
Cumulative Var 0.5 1.0

$rotmat [,1] [,2] [1,] 1.000000e+00 -7.142042e-16 [2,] 7.142042e-16 1.000000e+00  This result is completely uninterpretable to me. I thought that, after varimax rotation, one variable should load heavily on one factor and nearly 0 on the other factor, whereas the other variable should be the reverse. Was there something I did wrong? Or should I not use prcomp() to perform orthogonalisation? I'm pretty new to using PCA for orthogonalisation. • You say it is uninterpretable but then in your next sentence you explain what happened. Can you clarify? Aug 29 '16 at 14:20 • What I meant by uninterpretable is concerning my aim in doing the PCA. As I said, my aim is for orthogonalisation, that is, to separate my two variables which originally correlated to two factors that do not correlate any more. That's why I used varimax rotation. And my hope is to have the two variables loading on the two new factors separately so that I could argue that one factor is the orthogonalised LogFreq and the other is the orthogonalised Simulation. And I could manage it with SPSS. But the results I've got from R at the moment cannot be interpreted n this way. I hope that makes sense. Aug 30 '16 at 12:07 • I don't know what's going on here. When I use my MatMate-program with that data, the varimax-procedure produces perfectly a two axes solution, where each variable loads on one axis and not on the other one. I'm sure, SPSS would do the same. Perhaps prcomp works only, if there are more than 2 factors/axes? $$\small \text{loadings = csvdatei("clip") }\\ \small \text{load1 = rot(loadings,"varimax") }\\ \small \text{ load1 = \{\{ 0.00, -1.00 \}, } \\ \small \text{ \{ -1.00, 0.00 \} \} }$$ Aug 30 '16 at 13:03 • I've done the rotation using SPSS. It actually does the rotation and gives the unit-matrix as result. Here is the list of SPSS (V21)-statements: $$\small \begin{array} {l} \text{ FACTOR } \\ & \text{ /matrix =in(fac="s:\tstfac1.sav") } \\ & \text{ /ANALYSIS v1 v2 } \\ & \text{ /PRINT EXTRACTION rotation } \\ & \text{ /CRITERIA FACTORS(2) ITERATE(25) } \\ & \text{ /ROTATION varimax } \\ & \text{ /METHOD=CORRELATION. } \\ \end{array}$$ The matrix "tstfac1.sav" contains directly the loadings given by the OP in the "factor-matrix" format. Aug 31 '16 at 21:04 • @amoeba, I took the OP's 2x2 eigenvectors (Rotation) and varimax (or quartimax) rotated it in SPSS. I suppose Gottfried was doing the same thing. SPSS rotated it to binary-valued matrix, as expected and as it was with a 5x5 eigenvector matrix in the answer stats.stackexchange.com/a/154445/3277. So, SPSS does it correct way. I can't say anything to what's up there with R. Sep 1 '16 at 14:58 ## 1 Answer General remarks: 1. One should not varimax-rotate eigenvectors of PCA but loadings of PCA (i.e. eigenvectors scaled up by respective standard deviations). 2. Also, it does not make sense to rotate all PCA loadings, one would usually choose a subset of leading PCs and rotate those. If you, nevertheless, take all eigenvectors of PCA and varimax-rotate those (as you did here), you will get the original variables back; i.e. the identity matrix (possibly permuted). See Strange results of varimax rotation of principal component analysis in Stata: rotated components are all zeros and ones. However, it seems that for some strange varimax() in R does not rotate$2\times 2\$ matrices correctly.

The non-rotation in the 'VARIMAX' - procedure seems to be a(nother) bug in the R-package.

Checking your result with my matrix-tool MatMate I get the rotation to the identity matrix by the following commands:

a) using my matrix-tool MatMate:

[17] labels = {"LogFreq","Simulation"} '   //' // the [number] is command numbering
// for reference
{ -0.7471903, 0.6646101 }}

LogFreq: -0.66461    -0.74719
Simulation: -0.74719     0.66461

[VARIMAX]      vm1         vm2
LogFreq:  0.00000    -1.00000
Simulation: -1.00000     0.00000


Now using normalization of the columns to remove artifacts from decimal truncation of the loadings-values (which might have made small deviations from being eigenvectors )

[22] evectors = normsp(loadings)  // columnwise norming to sqsum = 1

[eigvectors]         ev1         ev2
LogFreq:    -0.66461    -0.74719
Simulation:    -0.74719     0.66461

// rotate the true eigenvector-type matrix to varimax
[24] vm = rot(evectors,"varimax")

[VARIMAX]      vm1         vm2
LogFreq:  0.00000    -1.00000
Simulation:  -1.00000    0.00000


b) Because there was some concern, that SPSS as well as R might not perform the rotation correctly here also the protocol using SPSS (V21). The following way I got the correct (rotated to unit-matrix) components:

data list records=1
/1 ROWTYPE_ 1-8(A) FACTOR_ 10
LogFreq 12-21 (F,8) Simulation 23-33 (F,8) .
begin data
FACTOR   1 -0.6646101 -0.7471903
FACTOR   2 -0.7471903  0.6646101
END DATA.

FACTOR
/MATRIX   IN( FAC = * )
/PRINT    extraction rotation
/ROTATION varimax    .


Part of the result:

Rotierte Komponentenmatrix ^a
Komponente
1           2
LogFreq      ,000       1,000
Simulation  1,000        ,000

Rotationsmethode: Varimax mit Kaiser-Normalisierung.
^a Die Rotation ist in 3 Iterationen konvergiert.

• +1. I would add that all of that is only of some academic interest; in practice, one should not varimax-rotate eigenvectors of PCA but loadings of PCA (i.e. eigenvectors scaled up by respective standard deviations). Also, it usually does not make sense to rotate all PCA loadings, one would usually choose a subset of leading components and rotate those. Perhaps it would make sense to add something like that to your answer (if you agree), otherwise OP might stay confused about that. Aug 31 '16 at 23:24
• @amoeba: very nice, I agree (but I'm lazy at the moment...) Would you mind to insert some nice explanation of this into the post? Otherwise I'll see what I can do in the afternoon... Sep 1 '16 at 6:39
• Sure. I have supplied "general remarks" in the beginning, feel free to edit. Sep 1 '16 at 21:35