Thanks to @amoeba, I learned that standardized eigenvectors are sometimes calculated, i.e., eigenvectors are divided by the square root of their eigenvalues.

Now, when I want to do principal component regression (PCR), I first calculate the components that I subsequently use in regression. There are two procedures that can be used:

  1. I calculate my components once as raw components, that is I do not standardize the eigenvectors.

  2. I calculate my components as components based on standardized eigenvectors.

Now, I do PCR once with the components in situation 1, and once with the components in situation 2.

My questions:

  • What are the implications (a) for the coefficients, (b) for the interpretation in the two cases?
  • What are the main differences?
  • I know that the variance of the components will differ, but how does that change the behaviour in regression, or does it at all?
  • Is there any recommendation to whether I should use the one but not the other?
  • 2
    $\begingroup$ maybe you will want to link to where amoeba has dropped the idea you are starting from? Because your question is unclear without that context. $\endgroup$ – ttnphns Apr 10 '15 at 13:45
  • $\begingroup$ I tried to make it clearer. She didn't really drop the idea, rather uncovered my knowledge lacks. $\endgroup$ – MaHo Apr 10 '15 at 17:58
  • $\begingroup$ In one case the components are standardized, in the other not. $\endgroup$ – user73344 Apr 12 '15 at 7:48
  • $\begingroup$ PCA eigenvectors can be multiplied (not divided!) by the square roots of the eigenvalues to obtain loadings. In principal component regression one uses principal components, i.e. projections on the eigenvectors, and not eigenvectors themselves, so I am not sure what exactly you mean by your approach #2. Perhaps you mean standardized principal components (scaled to unit variance). If so, then notice that any scaling of individual predictors does not influence the regression at all, so the answer to your question is that it does not matter. $\endgroup$ – amoeba Jul 8 '15 at 15:06
  • $\begingroup$ It's common to express eigenvectors as unit vectors. I don't know what dividing each component by its eigenvalue would mean. $\endgroup$ – duffymo Jul 8 '15 at 16:29

It might have something to do with whether your predictor variables are standardized or not (centered they must be!). If they are in original form, some eigenvector will be extremely large so standardizing them is a good idea (Why?). I don't see the point in standardizing them if your PCA used standardized (commensurating) variables already.

| cite | improve this answer | |
  • 1
    $\begingroup$ -1. That's not really an answer to the question. $\endgroup$ – amoeba Apr 14 '16 at 9:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.