0
$\begingroup$

Should I include covariates (i.e. age and sex) in the principal component regression as predictors? Or do I not need to do that because they were accounted for in the PCA?

Any help would be greatly appreciated.

Edit: Principal component analysis was used on the highly correlated independent variables (a,b,c,d,e,f,g). Also covariates age and sex were added in the principal component. from that, I got 4 principal components that accounted for most of the variation in the data. The rest of the components were not included in my analysis because they did not contribute much to the variance in the data, that included age and sex. Now I would like to perform a regression analysis on the principal components that explain most of the variance in the data. Should I only include PC1 and PC2 or should I also include PC1, PC2, age and sex? Would that be redundant to add them into the model once again?

Improve this question
$\endgroup$
3
  • $\begingroup$ How did you do the PCA? What are you using the regression to accomplish? $\endgroup$
    – Dave
    Commented Sep 12, 2020 at 20:24
  • $\begingroup$ @Dave I tried to give a more detailed example, please see above. $\endgroup$
    – J.Doe
    Commented Sep 12, 2020 at 20:40
  • 1
    $\begingroup$ What do you want to use the regression to accomplish? You've tagged this with hypothesis-testing but seem to want to do pure prediction. $\endgroup$
    – Dave
    Commented Sep 12, 2020 at 20:42

1 Answer 1

Reset to default
1
$\begingroup$

The answer depends on whether or not you want to penalize the contributions of age and sex to your outcome predictions or if you want to control strongly for them while penalizing the contributions of the other predictors.

If you think that age and sex are so important to your model that they need to be included with full weight, then you include them as full predictors in all the modeling and only do the principal-components part of the regression on the remaining predictors.

If you think that age and sex aren't necessarily any more important to control for than the other predictors, include them within the principal-components regression.

That choice is up to you, your knowledge of the subject matter, and the purposes of your modeling.

I see no reason at all to include age and sex with the principal-components analysis and also to include them separately as predictors in the model. If anything, that would tend to over-weight their contributions to outcome predictions.

Do note, also, that there are issues with relative weighting of categorical variables like sex in penalized methods like principal-components regression, ridge regression, and LASSO. See this page for some discussion.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.