15
$\begingroup$

As far as I understand, principal components are obtained by rotating the coordinate axes to align them with the directions of maximum variance.

Nevertheless, I keep reading about "unrotated principal components" and my statistics software (SAS) gives me varimax-rotated principal components as well as the unrotated ones. Here I am confused: when we compute principal components, the axes are already rotated; so why is there another rotation needed? And what does "unrotated principal component" mean?

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ Questions solely about how software works are off topic here, but you may have a real statistical question buried here. You may want to edit your question to clarify the underlying statistical issue. You may find that when you understand the statistical concepts involved, the software-specific elements are self-evident or at least easy to get from the documentation. $\endgroup$
    – gung - Reinstate Monica
    Jul 8, 2015 at 9:17
  • 1
    $\begingroup$ @gung - My question is not about software. May be I placed it wrongly. All what I wanted to know is that as per my understanding, we obtain principal components only when we rotate the axes in the line of maximum variance. Then what is unrotated principal component, a term which I found on various pages explaining about PCA. Kindly let me know if my question is still ambiguous. $\endgroup$
    – Srewashi Lahiri
    Jul 9, 2015 at 10:01
  • $\begingroup$ It certainly looks like it's about SAS. If it isn't, I would edit your Q to remove the references to SAS & re-explain your question in software-neutral terms. You may also be interested in reading this thread. $\endgroup$
    – gung - Reinstate Monica
    Jul 9, 2015 at 13:13
  • $\begingroup$ I mentioned SAS because I was carrying out the analysis in that software. Even if you discount the word, you can just provide me an explanation to my edited version of the question. Also I went through the thread. Kindly correct me if I am wrong. When we calculate principal components, it means that the axes are already rotated. So another varimax notation is not required. Is that so? I am really confused about this part. Many thanks in advance $\endgroup$
    – Srewashi Lahiri
    Jul 9, 2015 at 15:35
  • 2
    $\begingroup$ Srewashi, I have taken the liberty to substantially rewrite your question based on your clarifications in the comments. I think it is a good question, +1. Please check that my edits reflect your intentions! You can always edit more. Cc to @gung. $\endgroup$
    – amoeba
    Jul 10, 2015 at 13:37

1 Answer 1

Reset to default
19
$\begingroup$

This is going to be a non-technical answer.

You are right: PCA is essentially a rotation of the coordinate axes, chosen such that each successful axis captures as much variance as possible.

In some disciplines (such as e.g. psychology), people like to apply PCA in order to interpret the resulting axes. I.e. they want to be able to say that principal axis #1 (which is a certain linear combination of original variables) has some particular meaning. To guess this meaning they would look at the weights in the linear combination. However, these weights are often messy and no clear meaning can be discerned.

In these cases, people sometimes choose to tinker a bit with the vanilla PCA solution. They take certain number of principal axes (that are deemed "significant" by some criterion), and additionally rotate them, trying to achieve some "simple structure" --- that is, linear combinations that would be easier to interpret. There are specific algorithms that look for the simplest possible structure; one of them is called varimax. After varimax rotation, successive components do not anymore capture as much variance as possible! This feature of PCA gets broken by doing the additional varimax (or any other) rotation.

So before applying varimax rotation, you have "unrotated" principal components. And afterwards, you get "rotated" principal components. In other words, this terminology refers to the post-processing of the PCA results and not to the PCA rotation itself.

All of this is somewhat complicated by the fact that what gets rotated are loadings and not principal axes as such. However, for the mathematical details I refer you (and any interested reader) to my long answer here: Is PCA followed by a rotation (such as varimax) still PCA?

Improve this answer
$\endgroup$
5
  • $\begingroup$ I haven't yet come across a better and clearer explanation. I also went through the other link that you have provided but I am yet to decipher it in totality. If I understood right then unrotated principal components are already orthogonal and uncorrelated. Here I have a little confusion - as PCs correspond to successive maximum variance then is it necessary that after the first PC is found, the second maximum variance line (second PC) will be at 90 degree (orthogonal) to the first one and so forth? $\endgroup$
    – Srewashi Lahiri
    Jul 14, 2015 at 17:02
  • $\begingroup$ That's right: "unrotated" principal components are uncorrelated and "unrotated" principal axes are orthogonal. And yes, it is necessary that successive principal axes are orthogonal and principal components uncorrelated to the previous ones (one can prove it mathematically). By the way, if you think that this (or any other) answer settles the issue for you, you can "accept" it by clicking on the green tick on the left. Once you reach 15 reputation, you will also be able to upvote answers that you find useful (I think currently you are not able to upvote any answers). $\endgroup$
    – amoeba
    Jul 14, 2015 at 19:57
  • $\begingroup$ +1. what gets rotated are loadings and not principal axes as such I would add that this is a technical notion. Theoretically, these two rotation kinds are juxtapositional. In PCA we rotate to find the specific orthogonal basis (the one with steepest scree-plot of eigenvalues). In varimax, we rotate to find another specific orthogonal basis (with the interpetable-most structure). We could do any kind of orthogonal basis. $\endgroup$
    – ttnphns
    Jul 14, 2015 at 20:15
  • $\begingroup$ If possible can you explain it in layman terms what unrotated pc's mean? $\endgroup$
    – sai_636
    Feb 11, 2019 at 23:00
  • $\begingroup$ @sai_636 For layman terms please see stats.stackexchange.com/questions/2691. $\endgroup$
    – amoeba
    Feb 12, 2019 at 8:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.