# Reducing the dependency among variables

I am trying to perform a multi linear regression model:

$$y_i = β_0 + β_1x_{i1} + β_2x_{i2} +... + β_px_{ip} + ε_i$$

where $$x_{i1}, x_{i2}, ..., x_{ip}$$ are highly correlated with each other (VIFs can be as low as 5 and high as 10).

I am just wondering if there exists a procedure with the following properties:

1) reduces the collinearity of the variables (e.g. VIFs should be lower than 5 after the procedure)

2) the variables after the procedure should maintain the original meanings/interpretations.. (so PCA and FA are out).

3) not dropping any of the variables. I should have all p original varaibles.. (So LASSO and RIDGE are out)

• Why do you want to reduce the correlation? Aug 2, 2018 at 19:08
• @Jonas I am trying to perform "orthogonal transformation" as explained in business.wvu.edu/files/d/d2fd65ec-6fe4-44b5-b3bd-9b84449f3d7e/… to remove the dependency among variables and decompose the variance of the dependent variable. Unfortunately, the similarity, as measured by correlation coefficient, between the original and transformed variable tends to be low if the original variable is highly correlated with other variables. Aug 2, 2018 at 19:11
• Ridge regression estimates don't drop variables... Aug 2, 2018 at 23:51
• @EricMittman you are right... do you have any suggestions? Aug 2, 2018 at 23:52

Lasso and Ridge regression do not drop the variables but rather it reduces the significance of highly correlated variable by reducing its coefficients.

Also, I need a clarification as to why should you not drop the highly correlated independent variables? As multi-colinearity occurs whenever two Independent variables are highly correlated, in other words, when both the Independent variables are giving you the same information. Having said that, why is it not okay to remove one of those two correlated Independent variable?