I am studying a data set with one continuous response variable and 4 continuous predictor variables. The two predictors most correlated with the response also have a high (but not perfect) correlation with each other. A more parsimonious model might rely on 3 predictors, but I don't want to drop any of the variables.

I am considering two approaches. One is to do principal components analysis (PCA) on the space of predictors, and running a principal components regression (PCR), regressing the response on the first 3 principal components of the predictors' space.

The other alternative is to run a factor analysis (FA), attempting to discover three factors underlying the predictors' space that best explain the response; the method of FA would be PCA.

My question is whether these approaches are equivalent, or whether there is a fundamental conceptual difference in the goals of these methods. Thank you!

  • $\begingroup$ "The method of FA would be PCA" means that such an FA = PCA. Obviously, there is no difference then. $\endgroup$ – amoeba says Reinstate Monica Apr 27 '17 at 13:35
  • $\begingroup$ Just to be sure, can you clarify what you mean by "the method of FA would be PCA"? What software are you using and what "method" are you talking about exactly? $\endgroup$ – amoeba says Reinstate Monica Apr 27 '17 at 13:51
  • $\begingroup$ If so, @amoeba, I am curious why no article on PCR mentions its equivalence to PCA-FA, and no article on FA mentions the equivalence of PCA-FA to PCR. I am distinguishing between a FA performed using PCA versus one performed using another method like MLE. I am using R. I use prcomp for PCA. I am considering using the pls package for PCR, versus the psych package with factanal to do FA. $\endgroup$ – Hal M. Switkay Apr 27 '17 at 13:52
  • $\begingroup$ Wait a second - when you say that the other alternative is to run FA via PCA, you mean that you are later going to regress the response on the factor scores, right? Or something else? $\endgroup$ – amoeba says Reinstate Monica Apr 27 '17 at 13:54
  • $\begingroup$ You have it right, @amoeba. $\endgroup$ – Hal M. Switkay Apr 27 '17 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.