I have a couple of questions involving doing a regression (logistic or linear) after principal component analysis.

  1. If I find principal components using Principal component analysis, can I use these components like regular variables to be used in linear and logistic regression?

  2. If not, do I have to perform an extra step like "rotate" the principal components? What is this "rotation", what is it actually doing, and why is it helpful for using in regression? (this is what happens in factor analysis, no?)

  3. If I can use these components, rotated or not, can I simply setup the model and bind it with the outcome data? Do I need to transform the outcome data in any way? Or can I use the original outcome data with the principal components?

  • 2
    $\begingroup$ (1) Yes. (2) No need to rotate. (3) No need to transform. $\endgroup$ – amoeba says Reinstate Monica Mar 18 '16 at 11:22
  1. Yes.

    That's exactly what principal component regression is: https://en.wikipedia.org/wiki/Principal_component_regression.

  2. No need to rotate.

    In fact, rotating would not make any difference as far as prediction is concerned; see Using varimax-rotated PCA components as predictors in linear regression. Of course if you want to interpret individual regression coefficients, then you'd need to have interpretable predictors; rotated PCs might be more interpretable (or not).

  3. No need to transform.

    The dependent ("outcome") variable should be left alone. Including it into PCA would be cheating. There is no transformation that you should use only because you use PCA.

In general, you might want to read How can top principal components retain the predictive power on a dependent variable (or even lead to better predictions)?.

  • 2
    $\begingroup$ I've copied this comment by @amoeba as a community wiki answer because the comment is, more or less, an answer to this question. We have a dramatic gap between answers and questions. At least part of the problem is that some questions are answered in comments: if comments which answered the question were answers instead, we would have fewer unanswered questions. $\endgroup$ – mkt - Reinstate Monica Aug 10 '18 at 12:23
  • $\begingroup$ BTW, I don't think it's necessary to post such copy-pasted comments as CW. I'd be only happy if you got reputation from upvotes here. (If you want, feel free to flag for mod attention to remove CW status.) $\endgroup$ – amoeba says Reinstate Monica Aug 10 '18 at 12:40
  • $\begingroup$ @amoeba Thanks! I appreciate the thought but I like the CW solution, since I don't feel like I've done enough here to earn rep. Just happy to help tidy up the unanswered question list. :) $\endgroup$ – mkt - Reinstate Monica Aug 10 '18 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.