# Rotation in PCA and Factor Analysis [duplicate]

I want to know what elements are (varimax-)rotated when I rotate after PCA and after Factor Analysis.

Let’s assume a standardized data vector $X$ of dimension $N \times q$.

In PCA, I have the formula: $F=XL$

In Factor Analysis, I have: $X=FL^T+E$

F is of dim $N \times k$, and L is of dim $q \times k$.

I obtain the loadings for PCA by applying SVD on X and reading out one of the eigenvector matrices after having sorted them according to size of their eigenvalues. To make it clear, I decide to use the two components with the largest variance. Lets call them $C_{1}$ and $C_{2}$. So the eigenvectors from the SVD are equal to the loadings which are the new axes.

For Factor Analysis, I use for simplifying reasons „Principal Component Factors“ to extract my loadings because I do not have to estimate the specific variance - it arises as the residual variance after having calculated the factors (s. Rencher 2003 p.415-421). For PCF, I apply an SVD on $X$ and multiply the values in an eigenvector with the square root of their corresponding eigenvalues (so the main difference between the procedures at this point is only the multiplication of the eigenvectors with their eigenvalues – but still: it's Factor Analysis!). Again, I decide to retain the two factors with the largest eigenvalues. Lets call them $F_{1}$ and $F_{2}$.

I acknowledge that the factor analysis solution for the loadings is unique up to an orthogonal rotation.

Now on rotation. As far as I understand, I rotate orthogonally in PCA by multiplying the eigevectors $U_{1}$ and $U_{2}$ with an appropriate 2x2 matrix. So I rotate the axes (once more), but apparently I am still in space of the original variables, right? While my eigenvector) are still orthogonal to each other, my components (calculated as above =XL) are now correlated. I understand that this is drawback.

But what happens in factor analysis? My coordinate system is now spanned by the factors and not by the original variables, right? Somehow I lack intiution for I still haven’t estimated F.

Still, the procedure seems to be absolutely the same as before, because I rotate the loading matrix and the results seem very similar. They would be equal, I suppose, if the loadings in PCA would be multiplied by the square root of their eigenvalues, too. However, the drawback of correlated factors does not occur and I can safely rotate.

Having read @amoeba’s answer to Is PCA followed by a rotation (such as varimax) still PCA? , my question could not be answered maybe because I could not understand what the approach for FA was.

## marked as duplicate by amoeba, Community♦Mar 31 '15 at 14:17

• eigenvectors from the SVD are equal to the loadings; so the main difference between the procedures is only the multiplication of the eigenvectors with their eigenvalues. I stopped reading (excuse me please) upon encountering those lines. If, for you, that is the core distinction between PCA and FA then you must be missing a lot in your understanding. – ttnphns Mar 27 '15 at 15:47