I am trying to perform a multi linear regression model:

$$y_i = β_0 + β_1x_{i1} + β_2x_{i2} +... + β_px_{ip} + ε_i$$

where $$x_{i1}, x_{i2}, ..., x_{ip}$$ are highly correlated with each other (VIFs can be as low as 5 and high as 30).

I am just wondering if there exists a procedure with the following properties:

1) extracts the pure part of each independent variable, where pure is defined as a part of the variable which is uncorrelated with other variables.

2) the pure part and the original variable should be similar to each other, whether similar is defined as moments such as mean and variance and correlation coefficient (ideally >0.90)

  • $\begingroup$ There really can't be such a method. If you have two variables in your model whose correlation is 90%, how would you extract two new variables with the stated properties? It would be impossible! $\endgroup$ – Matthew Drury Aug 3 '18 at 17:50
  • $\begingroup$ @MatthewDrury Well, that's why I asked because I couldn't find a suitable method.. $\endgroup$ – Jun Jang Aug 3 '18 at 17:51
  • $\begingroup$ Right! So, given that such a method can't really exist, what compromises would you like to make on your conditions? $\endgroup$ – Matthew Drury Aug 3 '18 at 18:07
  • $\begingroup$ It sounds like you might be asking about a procedure I have described at stats.stackexchange.com/a/46508/919. Could that be it? $\endgroup$ – whuber Aug 3 '18 at 18:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.