how do you linearly combine predictors, such as adding them together to avoid multicollinearity? I have multicolinearity in my regression. say the function is lm(y ~ x + z) where x and z have collinearity. Is one solution to assess the individual regression mod1 = y ~ x and mod2= y ~ z then combine them. How would the combination work? how do you add these two together is it y = the coef x from mod1 + coef of z from mod2?

  • $\begingroup$ Welcome to CrossValidated! Can you tell us more about why you want to do this? I ask because there are many ways you could "solve" multicollinearity problems, and your choice of action might depend on the underlying reason you need to do it. $\endgroup$ Apr 18, 2019 at 15:50
  • $\begingroup$ Sure, I would like to synthetically build y, and believe x and z represent y. However, when I combine the 2 predictors in an equation (x and Z) they do not behave well and have VIF of 9 for x and z has 3. Also, when I regress x and z against y by themselves they are significant and in the multivariable regression x is not. $\endgroup$
    – Kat
    Apr 18, 2019 at 16:01
  • 1
    $\begingroup$ Following up on the question from @LouisCialdella how do you intend to use (or interpret) your model of y once you have built the model? For example, if the model is for predicting values of y in new cases then there might be little or no problem arising from collinearity. Please add that information to the original question as comments can occasionally be lost. $\endgroup$
    – EdM
    Apr 18, 2019 at 16:21
  • $\begingroup$ it's for predicting values of y in new cases $\endgroup$
    – Kat
    Apr 18, 2019 at 16:29

1 Answer 1


The usual goal in regression is to include all relevant predictors in the model. Omitting important predictors correlated with included predictors can lead to reversed signs of regression coefficients in standard linear regression (see Simpson's paradox). In logistic or Cox survival regressions, omitting a predictor related to outcome will even bias the coefficients of predictors with which it is not correlated. So starting with separate models for each of x and z is not a good way to proceed.

Multicollinearity (perhaps an over-statement in this case with only 2 predictors) gets a lot of attention but doesn't by itself necessarily lead to problems unless it's so extreme that you can't get reliable estimates of the coefficients. As this answer notes:

... the effects of collinearity are seen in the variances of the parameter estimates, not in the parameter estimates themselves.

The VIF, after all, is the Variance Inflation Factor for a coefficient.

What you are seeing is what you would expect if x and z are correlated with each other and if one or both are directly related to y. Then each of x and z could be significantly related to y individually (based on some p-value cutoff), but including both in a 2-predictor regression could lead to 1 of them appearing to be "not significant." It's quite possible in your 2-predictor model that x is still related to y in the same direction as it was individually, but that a high standard error for its regression coefficient (due to its correlation with z) made it fail to meet some p-value cutoff.

If your intent is prediction of y in new cases based on values of x and z, then there is no reason to omit either of x or z. In such a situation it's best to try to retain all variables related to outcome, even those that don't pass some particular test of statistical significance. Variable selection based on p-values or other relations to outcome "only makes things worse." You do have to take precautions against overfitting or use other modeling approaches when the ratio of cases to predictors gets low, but with only 2 predictors that shouldn't be a problem provided you have more than a couple of dozen cases.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.