Estimating effect of linear regression coefficients with multicollinearity

Question

As I didn't find a satisfying for that questions I try it here:

I have a multivariate Lineare Regression model with some correlated predictor variables. The "simple" question I want to answer is: "If I increase x1 by 1 unit, how much does the target value change?". I know I can use VIF to detect multicollinearity, but is there any approach how I can estimate how much the target is effected by a certain input variable considering the multicollinearity?

(I use elastic net / ridge at the moment, maybe this also is relevant).

Answer 1

In the case of multiple linear regression, which is what I think you meant, then the change in $y$ (your response or target) for a one unit change in $x$_$i$ ($i$^$th$ covariate) is estimated while holding all other covariates constant, whether or not there is collinearity - this is $\beta$$i$ (your coefficient for the $i$^$th$ covariate).

Regularised regression techniques (i.e. ridge, lasso or elastic net) help cope with collinearity by reducing the effect of influential points on the $k$-dimensional surface fitted through the data, where $k$ is the number of covariates you have. However, regularised regression still operates in the same manner as traditional multiple linear regression technically, as all that is changing is how the coefficient is calculated, not how the coefficient is used/interpreted. See this link for more on how ridge regression does this.

Answer 2

Multicollinearity does not affect much the level of the coefficient. So, "If I increase x1 by 1 unit, how much does the target value change?" will not be addressed directly by multicollinearity considerations.

Andre indicated that you can address multicollinearity by using regularization. And, you have done that. Watch out that regularization can dramatically change the explanatory power of your model. It can do that by reducing the relative influence of the most impactful variables and also at times by even changing the sign of some variables coefficients.

I think a better approach is to detect whether any of your variables are truly multicollinear. It is actually relatively rare that two variables are multicollinear. This is because they would have to be correlated at a level of 0.90 which corresponds to a Tolerance of close to 0.20 and a Variation Inflation Factor (VIF) of 5 times. The latter is a fairly common standard of two variables being multicollinear. If you do have multicollinear variables, the easiest way to fix that is to remove the variable that is less statistically significant and/or less explanatory. Then, you are fine. And, the regression coefficients of your resulting model directly answer your question in a robust way.