0
$\begingroup$

I would like to run an prediction model and have a set of continuous independent variables. They are all important but highly correlated. How can I effectively reduce collinearity and still use these variables in my prediction model?

$\endgroup$
  • $\begingroup$ It depends on the nature of the model: could you be more specific? $\endgroup$ – whuber Jul 5 '18 at 11:41
  • $\begingroup$ Outcome: 3 categories (mild, mod, severe) - model is ordered logistic regression. Predictors: continuous measurement of lab values and a few demographics. The lab values are correlated to each other. $\endgroup$ – Lucy Jul 5 '18 at 18:53
  • $\begingroup$ Could you share the evidence you have that collinearity will create any problems with prediction? Are you sure you really want to focus on prediction as opposed to explanation or coefficient estimation? $\endgroup$ – whuber Jul 5 '18 at 20:41
  • $\begingroup$ If we have collinearity among predictors, one or both can have an inflated p-value, thus my selection method (keep all variables jointly significant at p<0.05) will be biased. $\endgroup$ – Lucy Jul 6 '18 at 2:32
  • 1
    $\begingroup$ The problem lies with your selection method, which misuses the p-value, rather than with the collinearity. $\endgroup$ – whuber Jul 7 '18 at 1:48
0
$\begingroup$

You can reduce multicollinearity using PCA. There are lots of questions/answers about how to implement PCA. This method allows you to group similar covariates into independent "Principal Components" which can give insight into the relative relatedness of your covariates.

Also, check into Variance Inflation Factor (VIF) protocols. There are ways to use stepwise VIF reduction to rid yourself of highly collinear variables in the dataset. However, if you need to keep every covariate for some reason, a clustering approach like PCA or PLS would do the trick.

$\endgroup$
  • 1
    $\begingroup$ It will do the trick provided the response doesn't depend on the components you throw away! $\endgroup$ – whuber Jul 7 '18 at 1:49
  • 1
    $\begingroup$ That is a good point. You always run that risk when doing any sort of variable-reduction techniques. Generally, I would stick with keeping my correlated covariates, running Principal Components Regression with an oblique transformation in my PCA routine to flex the independence criterion. $\endgroup$ – ERT Jul 7 '18 at 1:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.