I have a linear model that requires a large number of interactions (there are as many interactions as there are IVs) and I want to reduce collinearity using PCA, then regressing the DV on the principal components (PCR).

lm(Y ~ NUM1 + NUM2 + NUM3 + NUM4 + NUM2*NUM3 + NUM2*NUM4 + FACT1*NUM1*NUM2)

FACT1 is a 10 levels categorical variable. All other IVs are numerical.

Identifying principal components on the full model specifications, including interactions, doesn't feel like the right approach since interactions inevitably introduce correlation among the model variables, that is between the basic predictors and the interactions themselves.

How should I conduct PCR in this case?

  • $\begingroup$ "Identifying principal components on the full model specifications, including interactions" - what does this mean? PCA has nothing to do with the model specification. $\endgroup$ – Aksakal May 27 '15 at 14:11
  • $\begingroup$ @Aksakal PCR has nothing to do with model specification? $\endgroup$ – Robert Kubrick May 27 '15 at 14:20
  • $\begingroup$ I see what you meant: to run PCA on the design matrix with all interactions or not. I don't think there's a rule that would prohibit you from doing this. $\endgroup$ – Aksakal May 27 '15 at 14:24
  • 1
    $\begingroup$ @Aksakal I've tried this myself and I wasn't entirely sure whether to trust the results. But I don't see why it wouldn't work. But if you're using a regression model it would make more sense to apply Principal Components Regression directly, or a penalized regression like the lasso, depending on your goals. The lasso, and it's cousin the elastic net, are known to outperform PCR when making predictions $\endgroup$ – shadowtalker May 27 '15 at 15:55
  • 1
    $\begingroup$ @Aksakal I'm saying that they should be combined in my opinion $\endgroup$ – shadowtalker May 27 '15 at 16:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.