When performing orthogonal rotations for a loading matrix in principal components analysis, the mathematical interpretation is relatively simple - rotate the two axes while keeping them perpendicular. Mathematically, this comes out to be

            | cos(phi) | -sin(phi) |
A_rot = A * |----------|-----------|
            | sin(phi) |  cos(phi) |

My question is how to you extend this to three or more factors? Graphically, the interpretation seems simple, at least for three dimensions - three axes (x,y,z) that can rotate in any direction while staying perpendicular. But what about mathematically?

One thought I had was to do two separate angles of rotation, each with its own matrix:

            | cos(phi1) | -sin(phi1) |   | cos(phi2) | -sin(phi2) |
A_rot = A * |-----------|------------| * |-----------|------------|
            | sin(phi1) |  cos(phi1) |   | sin(phi2) |  cos(phi2) |

Does that seem reasonable? Or is there a better solution, one that only includes a single rotation matrix?

  • $\begingroup$ The "rotations" in PCA include orientation-reversing maps. This group (the orthogonal group) is generated by all the reflections in planes through the origin. That gives great insight into what orthogonal transformations do geometrically. In particular, since any 2D rotation fixes an $n-2$ dimensional subspace, two of them fix at least $n-4$ dimensions, demonstrating that such rotations do not comprise the entire orthogonal group. Please tell us, then, what you consider a "rotation matrix" to be. $\endgroup$
    – whuber
    Apr 18 at 14:51
  • $\begingroup$ In 2D space (where we have d original variables and 2 factors), a rotation matrix is used to rotate the correlations in the original unrotated loading matrix A. Essentially, it projects each of the loadings (which, for two factors, can be visualized using cartesian x,y coordinates on two axes where each axis represents a factor) on to a new set of axes defined by a single rotational angle, phi. This angle can be adjusted until an "optimal" alignment of variables to axes occurs - the varimax algorithm, for example, find the angle that maximized the sum of variances across each row of A_rotated $\endgroup$ Apr 18 at 15:38
  • $\begingroup$ That works because the orthogonal group for $\mathbb R^2$ is one-dimensional (it's parameterized by an angle and the sign of the determinant). In $\mathbb R^n,$ however, the group has $n(n-1)/2$ dimensions and requires that many parameters. Note, too, that any rotation in $\mathbb R^n$ has an $n-2$ dimensional axis of fixed points: this is well worth understanding when you need to work in more than $3$ dimensions where visualization is difficult. Finally, the matrix of such a rotation is $n\times n,$ not $2\times 2$ (as incorrectly suggested in this question). $\endgroup$
    – whuber
    Apr 18 at 16:48


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.